Manufacturing MLOps at GS Caltex with Amazon SageMaker Canvas, half 1

GS Caltex makes use of Amazon SageMaker Canvas and Amazon SageMaker Pipelines to attach course of engineers’ fashions to manufacturing workflows. The South Korean power and chemical firm divides mannequin growth and operations between engineers with complementary abilities.

Course of engineers construct and validate fashions in SageMaker Canvas with out writing code, drawing on their course of information. Machine studying (ML) engineers configure SageMaker Pipelines workflows for deployment and scheduled inference.

Shifting property from SageMaker Canvas into SageMaker Pipelines nonetheless required integration work. ML engineers needed to set information paths for manufacturing, provide the mannequin configuration, and join preprocessing and inference jobs. GS Caltex automated this handoff whereas reusing the property created in SageMaker Canvas.

GS Caltex studies that its ML engineers now deploy fashions inside a couple of hours, in contrast with a couple of week earlier than. The corporate runs batch inference each half-hour, decreasing the necessity for repeatedly operating endpoints. GS Caltex studies that this has meaningfully lowered its inference prices.

This put up explains how GS Caltex connects SageMaker Canvas to SageMaker Pipelines and reuses validated preprocessing flows and fashions in manufacturing.

Resolution overview

The machine studying operations (MLOps) workflow has two paths: getting ready a brand new mannequin model for deployment, and operating batch inference with the deployed mannequin.

From a validated mannequin to a repeatable workflow

You need to use SageMaker Canvas to organize information visually, construct fashions, and generate batch predictions. You may as well register validated fashions in Amazon SageMaker Mannequin Registry from SageMaker Canvas. GS Caltex constructed a separate handoff course of to show these property into executable SageMaker Pipelines steps.

Course of engineers perceive the variables and information behind a course of and assess whether or not a mannequin’s predictions are helpful. ML engineers flip the ensuing property into manufacturing jobs and handle mannequin variations and execution historical past.

Reference structure

The next diagram reveals how course of engineers hand off SageMaker Canvas property to ML engineers for deployment and scheduled inference. ML engineers use Amazon Easy Storage Service (Amazon S3) for preprocessing flows and Mannequin Registry for mannequin variations, then schedule pipeline runs via Amazon EventBridge.

ML engineers obtain the preprocessing stream and mannequin from SageMaker Canvas and assessment the information codecs and prediction outcomes. After that assessment, they approve the mannequin in Mannequin Registry and deploy it. This preparation takes place when a brand new mannequin model is deployed. Subsequent inference runs reuse the deployed mannequin.

Amazon EventBridge begins every scheduled run. SageMaker Pipelines prepares the enter information from Amazon Easy Storage Service (Amazon S3), runs preprocessing and SageMaker Batch Remodel, and shops the ends in S3. ML engineers can examine step standing and Amazon CloudWatch logs to observe the workflow.

The manufacturing pipeline reuses the preprocessing logic and skilled mannequin created in SageMaker Canvas. ML engineers configure the enter and output paths, set the order of execution, and write the code that connects approval and scheduling to the workflow.

Conditions

Earlier than you start, put together the next:

The total reference implementation is obtainable within the GitHub repository. The code examples on this put up give attention to the connections between parts. The repository makes use of a pattern mortgage classification mannequin, so alter the configuration to match your mannequin’s enter and output codecs.

Technical implementation

The next steps cowl preprocessing reuse, mannequin approval and deployment, pipeline building, and scheduled inference.

1. Run the SageMaker Canvas preprocessing stream with SageMaker Processing

A preprocessing stream created with Knowledge Wrangler in SageMaker Canvas data the information sources and transformation steps. To arrange the reference implementation, export information to Amazon S3 utilizing SageMaker Processing, then retrieve the job settings and the .stream file utilized by that job.

Within the export settings, flip off Auto job configuration and choose SageMaker Processing. After the export begins, open the Processing job particulars on the Amazon SageMaker AI console. Document the container picture URI, the S3 URI of the enter named stream, the execution position, and the output settings. Use these values to configure the pipeline step that runs the .stream file.

The SageMaker execution position should have permissions to learn the required S3 information, write the outcomes, and entry the container picture. Select the occasion kind and rely based mostly on the amount of knowledge you have to course of.

Preserve the preprocessing logic validated by the method engineer, and alter the information sources, paths, and different settings for the manufacturing atmosphere. Test the container picture and output node in opposition to the unique job configuration.

Some .stream information include execution metadata resembling internal_metadata.dw_job.container_uri. Test the precise Processing job settings moderately than assuming that each export incorporates this area. Should you transfer the workflow to a different AWS Area, confirm that the container picture is obtainable there.

The reference implementation separates intermediate and ultimate outputs by execution below runs/<execution-id>/. Its PrepareInput step writes information to a set S3 location referenced by the .stream file. Forestall executions that share that enter path from overlapping.

To isolate enter information for every execution as properly, add a preparation step that updates the supply URI in a replica of the .stream file. Declaring a SageMaker Pipelines parameter doesn’t mechanically exchange a URI contained in the file.

2. Deal with variations between coaching and inference inputs

The goal column is one distinction between coaching and inference information. A stream constructed with coaching information can include transformations that anticipate the goal column within the enter schema. Inference information doesn’t include that identified end result.

The reference implementation provides a PrepareInput step earlier than preprocessing. If the goal column is lacking, this step provides it with a placeholder worth. After preprocessing, DropTargetColumn removes the column. Match the placeholder’s information kind and column place to the coaching schema, and move solely the anticipated options to the mannequin of their unique order.

PrepareInput -> PreprocessNewData -> DropTargetColumn
             -> BatchPredict -> AssembleResults

Use this strategy solely when the placeholder goal doesn’t have an effect on characteristic values. A stream that filters rows by the goal, derives options from it, or applies goal encoding wants extra therapy. In these circumstances, separate the transformations and discovered state required for inference.

Validation ought to cowl column order and rely, information varieties, missing-value dealing with, and row rely. After eradicating the goal, verify that the remaining columns match the characteristic order used throughout coaching.

3. Register, approve, and deploy the mannequin

Register the validated SageMaker Canvas mannequin model in Mannequin Registry. The registered model can embody the mannequin information, inference configuration, and analysis outcomes. ML engineers assessment this data earlier than altering the approval standing.

The mannequin file, resembling mannequin.tar.gz, should be used with the proper inference container and atmosphere settings. A mannequin package deal in Mannequin Registry teams this data collectively. On this batch structure, deployment creates a SageMaker mannequin useful resource from an accepted package deal for Batch Remodel to reference.

The next instance checks a package deal that has already been accepted, then creates the mannequin useful resource. Provide model_package_arn, model_name, and sagemaker_role_arn throughout deployment. This instance makes use of a singular mannequin title for every model.

import boto3

sm_client = boto3.shopper("sagemaker")
package deal = sm_client.describe_model_package(
    ModelPackageName=model_package_arn
)

if package deal["ModelPackageStatus"] != "Accomplished":
    elevate RuntimeError("Mannequin package deal isn't prepared")
if package deal["ModelApprovalStatus"] != "Authorised":
    elevate RuntimeError("Mannequin package deal isn't accepted")

sm_client.create_model(
    ModelName=model_name,
    ExecutionRoleArn=sagemaker_role_arn,
    PrimaryContainer={"ModelPackageName": model_package_arn},
)

Mannequin Registry data approval standing, however it doesn’t implement an approval verify on each inference request. Test that standing within the deployment code and retain the chosen package deal ARN and mannequin title within the pipeline configuration. If approval is revoked, replace the schedules and execution configurations that use the mannequin.

Registration and deployment happen when a mannequin model is launched. The pipeline that runs each half-hour creates Batch Remodel jobs utilizing that mannequin, so it doesn’t must register the mannequin once more on every run.

4. Join preprocessing, inference, and outcome meeting

The inference pipeline has 5 steps.

Configure every job earlier than connecting the steps. Within the following instance, prepare_args, preprocess_args, drop_target_args, and assemble_args maintain the step arguments returned by run() calls on processors that use a PipelineSession. The processors and transformer share the pipeline_session object. Configure the transformer with the beforehand deployed model_name.

from sagemaker.workflow.pipeline import Pipeline
from sagemaker.workflow.steps import ProcessingStep, TransformStep

put together = ProcessingStep(
    title="PrepareInput", step_args=prepare_args
)
preprocess = ProcessingStep(
    title="PreprocessNewData",
    step_args=preprocess_args,
    depends_on=[prepare],
)
drop_target = ProcessingStep(
    title="DropTargetColumn",
    step_args=drop_target_args,
    depends_on=[preprocess],
)

# The DropTargetColumn job defines an output named "options".
features_uri = (
    drop_target.properties.ProcessingOutputConfig
    .Outputs["features"].S3Output.S3Uri
)
predict = TransformStep(
    title="BatchPredict",
    step_args=transformer.rework(
        information=features_uri,
        content_type="textual content/csv",
        split_type="Line",
    ),
)
assemble = ProcessingStep(
    title="AssembleResults",
    step_args=assemble_args,
    depends_on=[predict],
)

pipeline = Pipeline(
    title="canvas-process-inference",
    steps=[prepare, preprocess, drop_target, predict, assemble],
    sagemaker_session=pipeline_session,
)
pipeline_arn = pipeline.upsert(role_arn=sagemaker_role_arn)["PipelineArn"]

The next execution graph reveals the 5 steps within the pattern mortgage mannequin’s inference workflow. They put together inputs, apply preprocessing, take away the goal column, run batch inference, and assemble outcomes. The execution was within the Executing state when this screenshot was taken.

Determine 2: SageMaker Pipelines execution graph for the pattern mortgage mannequin

Use depends_on to regulate execution order, and move information via step output properties. For instance, assemble_args should obtain the placement of the preprocessed information and predict.properties.TransformOutput.S3OutputPath. The output title "options" belongs to DropTargetColumn. The Knowledge Wrangler step makes use of the output ID outlined in its preprocessing stream.

Within the reference implementation, AssembleResults combines the preprocessed information and predictions by row place. Earlier than utilizing it in manufacturing, add checks that file order and row counts match, and cease outcome technology if they don’t. If preprocessing kinds or removes rows, preserve an express mapping between supply document IDs and predictions.

5. Schedule inference each half-hour

GS Caltex runs inference each half-hour. The next instance makes use of an Amazon EventBridge rule to schedule SageMaker Pipelines executions. Create the pipeline and confirm it with a guide run earlier than attaching the schedule.

import boto3

events_client = boto3.shopper("occasions")
rule_name = "canvas-process-inference-every-30-minutes"

events_client.put_rule(
    Title=rule_name,
    ScheduleExpression="fee(half-hour)",
    State="ENABLED",
)

outcome = events_client.put_targets(
    Rule=rule_name,
    Targets=[{
        "Id": "canvas-process-inference",
        "Arn": pipeline_arn,
        "RoleArn": eventbridge_role_arn,
    }],
)
if outcome["FailedEntryCount"]:
    elevate RuntimeError(outcome["FailedEntries"])

Set eventbridge_role_arn to a task that trusts the Amazon EventBridge service principal, occasions.amazonaws.com, and has sagemaker:StartPipelineExecution permission for the goal pipeline. The pipeline_arn worth identifies the pipeline created within the earlier step.

The expression fee(half-hour) schedules runs at 30-minute intervals. To run on the hour and half hour, use cron(0/30 * * * ? *). Resolve which information window every run processes within the enter preparation step, and document it with the execution.

Operational issues

Earlier than enabling scheduled runs, verify the full price, execution time, and dealing with of invalid or mismatched information.

Consider batch prices throughout the complete workflow

Amazon SageMaker Pipelines is a serverless workflow orchestration service. It manages execution order and dependencies with out a workflow server so that you can provision or preserve. Processing and Batch Remodel jobs use separate compute sources and are billed in response to occasion kind and length.

A provisioned real-time endpoint is an efficient match for requests that want a persistently low-latency response. Batch Remodel fits workloads that may course of a set of data collectively. It releases the job’s compute sources when the job finishes, so you possibly can run predictions with out preserving an inference endpoint operating repeatedly for every mannequin.

A 30-minute schedule produces 48 scheduled runs per day. Embrace preprocessing, occasion startup, inference, and postprocessing when estimating whole runtime. The associated fee comparability must also account for Processing, S3, CloudWatch, and different providers utilized by the workflow.

Assess the price of every workload utilizing its measured utilization, together with the supporting providers and the complete runtime of each scheduled job.

Test runtime and overlapping executions

A schedule that begins a run each half-hour doesn’t assure that every run finishes inside that interval. Measure the complete pipeline runtime with consultant information, together with occasion startup and the time wanted to retry failed jobs.

Resolve easy methods to deal with a scheduled run that arrives earlier than the earlier execution has completed. Controlling parallel steps inside a pipeline doesn’t stop separate pipeline executions from overlapping. Test lively executions, stop duplicate processing of the identical enter window, and separate the output paths for every run.

Document the enter information window, preprocessing stream model, mannequin package deal ARN, pipeline execution ARN, and output location for every run. These particulars assist reproduce outcomes and examine errors.

Validate outputs and failure conduct

Full the next checks earlier than you allow the schedule:

1. Run a validation dataset via SageMaker Canvas and Processing. Evaluate the output columns, information varieties, row counts, and key remodeled values.
2. Test that including a placeholder goal doesn’t change characteristic values, and that the goal is faraway from the mannequin enter.
3. Run Batch Remodel with the accepted mannequin model and confirm the affiliation between supply data and predictions.
4. Verify that an invalid enter schema or preprocessing failure prevents downstream inference and outcome publication.
5. Measure runtime and value, take a look at overlapping executions and reprocessing of the identical enter window, after which allow the 30-minute schedule.

Add CloudWatch alerts for job failures and processing delays, along with information high quality or mannequin monitoring as wanted. Set preliminary thresholds from measured runtime and information traits, and assessment them when the mannequin or information quantity modifications.

Adoption strategy and outcomes

With a staged rollout, a group can confirm the handoff earlier than standardizing it throughout fashions. The next sections describe an adoption strategy and the outcomes GS Caltex reported.

Transfer from mannequin growth to common operation

Begin by validating the trail to manufacturing for one mannequin, then standardize the steps that repeat throughout deployments. The next sequence is a urged adoption strategy based mostly on the GS Caltex expertise.

As automation expands, course of engineers proceed to evaluate whether or not the predictions are helpful for the method. ML engineers handle pipeline execution and modifications.

Outcomes at GS Caltex

GS Caltex decreased the repeated work wanted to attach SageMaker Canvas fashions to manufacturing. ML engineers now deploy fashions inside a couple of hours, in contrast with a couple of week beforehand. Additionally they reuse the property that course of engineers validated in SageMaker Canvas.

Course of engineers started proposing functions in different processes after seeing their fashions utilized in manufacturing. ML engineers spent much less time repeating integration work and will give attention to working and monitoring the pipelines.

Clear up take a look at sources

For a take a look at deployment, disable the Amazon EventBridge rule and take away its targets first. After lively pipeline executions and jobs have completed, delete the take a look at pipeline and its SageMaker mannequin sources.

Should you created a real-time endpoint for testing, delete it as properly. Take away enter, intermediate, and prediction information from S3, and clear up CloudWatch logs in response to your retention coverage. Test whether or not different executions or audit necessities nonetheless rely upon mannequin information and Mannequin Registry variations earlier than deleting them.

Conclusion

At GS Caltex, course of engineers create and validate fashions in SageMaker Canvas, and ML engineers use these property to run manufacturing workflows in SageMaker Pipelines. This handoff decreased repeated integration work. GS Caltex studies that ML engineers now full mannequin deployments inside a couple of hours, in contrast with a couple of week earlier than.

The handoff wants to incorporate the preprocessing stream, enter schema, inference configuration, and approval course of alongside the mannequin information. Confirm that the workflow reproduces the anticipated outcomes and might run reliably on the required schedule.

To do that strategy, begin with one mannequin you could have validated in SageMaker Canvas and use the GitHub repository. See the SageMaker Pipelines developer information for particulars on defining and operating workflows. Test preprocessing and batch prediction outcomes with a guide execution, then schedule runs on the interval your utility wants.

Partly 2, we present how GS Caltex used Amazon Bedrock to automate shifting present fashions onto this pipeline.

Concerning the authors