How to Use Amazon SageMaker Pipelines MLOps with Gretel Synthetic Data

Step 1: Set Up Your AWS Environment

!pip install git+https://github.com/gretelai/gretel-mlops.git

dataset:
  name: healthcare-stroke-data
  train_path: 's3://gretel-datasets/ml_ops/stroke/train.csv'
  validation_path: null
  test_path: null
  target_column: stroke
  drop_columns: id
ML:
  ml_task: classification
  objective: 'binary:logistic'
  objective_type: Maximize
  ml_eval_metric: f1
  ml_deployment_threshold: 0.6
gretel:
  strategy: balance
  generate_factor: 1
  mode: cloud
  sink_bucket: null

The ML section defines the machine learning task as classification or regression and sets the optimization function’s objective. Choose an evaluation metric to maximize or minimize. Set a deployment threshold to ensure only adequately performing models are registered in the SageMaker Model Registry.

The gretel section covers using synthetically generated data. Set the strategy parameter, and determine the amount of synthetic data with generate_factor. Choose the training mode to be cloud (Gretel Cloud) or hybrid (AWS). For hybrid training, you must provide sink_bucket configured in Gretel Hybrid deployment.

Step 3: Define and Run the SageMaker Pipeline

The following code block is used to define the pipeline:

from gretel_mlops.aws.sagemaker.pipeline import get_pipeline

model_package_group_name = f"GretelModelPackageGroup-{config['dataset']['name']}"
pipeline_name = f"GretelPipeline-{config['dataset']['name']}"
print(f"Initiating {pipeline_name}")

pipeline = get_pipeline(
    region=region,
    role=role,
    default_bucket=default_bucket,
    model_package_group_name=model_package_group_name,
    pipeline_name=pipeline_name,
    config=config,
)

Once defined, create or update the pipeline and start the pipeline execution:

pipeline.upsert(role_arn=role)
train_execution = pipeline.start()

Check the status of your pipeline execution using a workflow graph. Choose Home > Pipelines > your pipeline name > Graph tab to display the workflow graph. The graph created by the Gretel MLOps library is shown below.

Figure 3 – SageMaker Pipeline workflow graph using Gretel.

Step 4: Evaluate the Final Results

After the pipeline is successfully executed, retrieve the ML evaluation report. It shows machine learning metrics evaluated on the test set.

The following code block shows how to retrieve the evaluation report:

import json

s3_client = boto3.client('s3')
s3_path_report = f"{pipeline.steps[3].arguments['ProcessingOutputConfig']['Outputs'][0]['S3Output']['S3Uri']}/evaluation.json"
bucket_name = s3_path_report.replace("s3://", "").split('/', 1)[0]
file_key = s3_path_report.replace("s3://", "").split('/', 1)[1]

# Fetch the file from S3
response = s3_client.get_object(Bucket=bucket_name, Key=file_key)
content = response['Body'].read()

# Parse the JSON content
data = json.loads(content)

# Pretty print the JSON data
print(json.dumps(data, indent=4))

An example evaluation report is shown below:

{
  "metrics": {
    "auc": {
      "value": 0.7368312757201646
    },
    "aucpr": {
      "value": 0.12997917776590717
    },
    "precision": {
      "value": 0.136986301369863
    },
    "recall": {
      "value": 0.4
    },
    "f1": {
      "value": 0.2040816326530612
    },
    "confusion_matrix": {
      "value": [
        [846,126],
        [ 30, 20]
      ]
    }
  }
}

Deep Dive on Individual Steps

Preprocessing Step

This step from the Gretel MLOps library prepares the data for subsequent phases and includes:

• Data preparation: Loads data from configuration files and identifies feature and target columns. It also eliminates excluded columns.
• Data splitting and preprocessing: Splits data into training, validation, and test sets based on available paths. Note that the training data, intended for the Gretel phase, requires no transformation. It’s saved as a Gretel training source file.
• Data transformation: For the downstream ML model, numeric data undergoes imputation and scaling, while categorical data is subjected to imputation and one-hot encoding. A preprocessing model is fitted to the training data, applying these transformations across all datasets.

Gretel Step

This step focuses on training a generative model on the Gretel training source file and involves the following tasks:

• Hyperparameter tuning: Utilizes Gretel Tuner, an optional module of Gretel’s Python SDK, for tuning synthetic model parameters. This involves comprehensive parameter sweeps to find the optimal configuration. It’s tailored to the downstream ML task that uses an XGBoost classifier or regression model, similar to the final ML application.
• Generate and use synthetic data: Generates synthetic data to replace, augment, or balance the training data:
  • Replace: Uses solely synthetic data for ML training, ideal for privacy-sensitive scenarios.
  • Augment: Enhances real training data with synthetic records, enriching the model’s performance with diverse examples.
  • Balance: For classification tasks, creates synthetic data for underrepresented classes to address class imbalance and improve model accuracy and fairness.
• Synthetic data quality reports: Gretel provides reports with synthetic data quality metric that are accessible via:

sqs_scores = f”{pipeline.steps[1].arguments['ProcessingOutputConfig']['Outputs'][0]['S3Output']['S3Uri']}/report_quality_scores.txt

Model Training Step

In this step, the focus is on training the downstream ML model:

• XGBoost framework: Employs XGBoost for building efficient ML models for classification or regression.
• Hyperparameter Tuning: Fine tunes the XGBoost model by finding the hyperparameters that optimize its performance on the validation set. Target metrics depend on the task:
  • Classification task: accuracy, f1, auc, aucpr, precision, or recall
  • Regression task: mse, rmse, mae, or R2

Evaluation Step

This step assesses the optimal downstream ML model on the test set and compiles an evaluation report with relevant metrics.

Condition and Model Register Step

The last two steps are registering the trained ML model in the model registry, conditional on passing a predefined performance threshold.

Conclusion

The integration of Gretel with Amazon SageMaker Pipelines is a significant advance for machine learning practices. By using Gretel in ML workflows, you develop more robust machine learning models while ensuring compliance with stringent privacy regulations.

The use of Gretel within the AWS framework, especially in a hybrid environment, introduces an extra layer of data privacy protection. This protects sensitive information throughout the entire machine learning lifecycle.

Get started with the Gretel 101 Blueprint to learn the basics of the Gretel SDK and train generative models in the Gretel Cloud. For more advanced work, the Gretel Advanced Tabular Blueprint offers customizable model configurations and conditional synthetic data generation. Explore text generation with the Gretel Text Generation Blueprint, which is ideal for finetuning large language models. To train synthetic data in your own cloud environment, check out Gretel Hybrid.

You can find Gretel available on AWS Marketplace.

.

.

Gretel – AWS Partner Spotlight

Gretel is an AWS Partner that enables the development of domain-specific AI models for creating data that mirrors, boosts, or simulates real-world data without the privacy concerns.

Contact Gretel | Partner Overview | AWS Marketplace