metamaker

Simple command line tool to train and deploy your machine learning models with AWS SageMaker

Features

metamaker enables you to:

• Build a docker image for training and inference with poetry and FastAPI
• Train your own machine learning model with SageMaker
• Deploy inference endpoint with SageMaker

Usage

1. Create poetry project and install metamaker

❯ poetry new your_module ❯ cd your_module ❯ poetry add metamaker

1. Define scripts for traning and inference in main.py

```main.py from pathlib import Path from typing import Any, Dict

from metamaker import MetaMaker

Import your model, and input/output data classs:

Model ... machine learning model class you want to use

Input ... input data class for inference

Output ... ouput data class for inference

Note that the Input and Output are used as type hints to

create API endpoint with FastAPI like below:

@fastapi_app.post("/invocations")

def predict(data: Input) -> Output:

...

from your_module import Model, Input, Output

app = MetaMakerModel, Input, Output

@app.trainer def train( datasetpath: Path, artifactpath: Path, hyperparameters: Dict[str, Any], ) -> None: model = Model(**hyperparameters) model.train(datasetpath / "train.csv") model.save(artifactpath / "model.tar.gz")

@app.loader def load(artifactpath: Path) -> Model: return Model.load(artifactpath / "model.tar.gz")

@app.predictor def predict(model: Model, data: Input) -> Output: return model.predict(data) ```

1. Write metamaker configs in metamaker.yaml

```metamaker.yaml

Specify metamaker handler like: path.to.module:app_name

handler: main:app

datasetpath and artifactpath should be directories and end with '/'

datasetpath: s3://your-bucket/path/to/dataset/ artifactpath: s3://your-bucket/path/to/artifacts/

hyperparameter_path: ./hparams.yaml

image: name: metamaker includes: - yourmodule/ - main.py excludes: - _pycache__/ - '*.py[cod]'

training: execution_role: arn:aws:iam::xxxxxxxxxxxx:role/SageMakerExecutionRole instance: type: ml.m5.large count: 1

inference: endpointname: yourendpoint instance: type: ml.t2.meduim count: 1 ```

1. Build docker image and push to ECR

metamaker build --deploy .

1. Train your model with SageMaker and deploy endpoint

metamaker sagemaker train --deploy