Source time Edition date
IndustryAWS Machine Learning Blog
SageMaker Inference now supports Multi-Instance GPU to maximize GPU utilization for model endpoints
SageMaker Inference 现支持多实例 GPU，以最大化模型端点的 GPU 利用率
SageMaker's MIG support lets teams slice one NVIDIA GPU into isolated partitions, but the gain depends on whether your models are small enough to fit the fractions you choose.
Judgment
SageMaker Inference users now face a partitioning trade-off: place different models in separate MIG slices, but only if each model fits the fixed compute and memory share, so the endpoint owner decides which models share one A100.
Basis
The AWS Machine Learning Blog post by Shantanu Tripathi, Apoorva Chandra, and Dmitry Soldatkin says SageMaker Inference now supports GPU partitioning via NVIDIA MIG, dividing a physical GPU into independent partitions with a fixed share of compute units and dedicated memory, isolated in hardware. The source's own example: an ml.p4d.24xlarge has eight A100s and holds eight models whether each needs 40 GB or 14 GB; previously the smallest request was one whole accelerator, so teams either under-served a model or paid for unused capability. MIG changes placement by allowing a different model per partition, but the model must fit the chosen fraction.
Unclear The source does not state supported instance types beyond ml.p4d.24xlarge, available partition sizes, pricing impact, or whether existing endpoints can be resized without redeployment.
Next Check the SageMaker Inference documentation for MIG-capable instance families and exact partition profiles before choosing a deployment shape.