Cluster

Source: src/AWS/SageMaker/Cluster.ts

An Amazon SageMaker HyperPod cluster — a resilient, persistent cluster of ML compute for distributed training and inference, orchestrated by Slurm or EKS, with automatic faulty-node recovery and deep health checks.

Provisioning a HyperPod cluster takes 10–25 minutes; instance groups are updated in place and removing a group from instanceGroups deletes it from the cluster.

Creating Clusters

Slurm-Orchestrated Cluster

import * as AWS from "alchemy/AWS";

const cluster = yield* AWS.SageMaker.Cluster("TrainingCluster", {
  instanceGroups: {
    controller: {
      InstanceType: "ml.t3.medium",
      InstanceCount: 1,
      ExecutionRole: role.roleArn,
      LifeCycleConfig: {
        SourceS3Uri: `s3://${bucket.bucketName}/lifecycle`,
        OnCreate: "on_create.sh",
      },
    },
  },
});

EKS-Orchestrated Cluster

// The EKS cluster must use the `API` (or `API_AND_CONFIG_MAP`)
// authentication mode — pass `accessConfig` explicitly, EKS's own
// CONFIG_MAP default is rejected. LifeCycleConfig is required for
// EKS-orchestrated instance groups too.
const hyperpod = yield* AWS.SageMaker.Cluster("EksHyperPod", {
  orchestrator: { Eks: { ClusterArn: eksCluster.clusterArn } },
  vpcConfig: {
    SecurityGroupIds: [securityGroupId],
    Subnets: network.privateSubnetIds,
  },
  instanceGroups: {
    workers: {
      InstanceType: "ml.g5.xlarge",
      InstanceCount: 2,
      ExecutionRole: role.roleArn,
      LifeCycleConfig: {
        SourceS3Uri: `s3://${bucket.bucketName}/lifecycle`,
        OnCreate: "on_create.sh",
      },
    },
  },
  nodeRecovery: "Automatic",
});

// The keys carry through to the attributes — typed per key:
const workers = hyperpod.instanceGroups.workers;

Running Workloads (Slurm)

# Slurm jobs are submitted on the cluster itself. Each node is an SSM
# target named sagemaker-cluster:<cluster-id>_<instance-group>-<instance-id>
# (list nodes with `aws sagemaker list-cluster-nodes`).
aws ssm start-session \
  --target sagemaker-cluster:6wl4at0i68c6_controller-i-0123456789abcdef0
# then, on the node:
sbatch --nodes=4 train.sbatch

Running Workloads (EKS)

Low level: apply any Kubernetes manifest to the orchestrator

// HyperPod nodes are ordinary EKS nodes — target them from a raw
// manifest (a PyTorchJob CRD, a batch/v1 Job, ...) with the well-known
// node labels.
const job = yield* AWS.EKS.Manifest("RawTrainJob", {
  cluster: eksCluster,
  manifest: {
    apiVersion: "batch/v1",
    kind: "Job",
    metadata: { name: "raw-train", namespace: "default" },
    spec: {
      template: {
        spec: {
          nodeSelector: {
            "sagemaker.amazonaws.com/node-health-status": "Schedulable",
            "sagemaker.amazonaws.com/instance-group-name": "workers",
          },
          containers: [{ name: "train", image: "ghcr.io/acme/train:v3" }],
          restartPolicy: "Never",
        },
      },
    },
  },
});

High level: an effectful Job pinned to HyperPod nodes

// AWS.EKS.Job / AWS.EKS.Deployment run on HyperPod via the orchestrating
// EKS cluster; the `hyperpod` prop derives the node selector, namespace,
// and Kueue labels — pass the ComputeQuota resource to submit through
// task governance.
const evaluate = yield* AWS.EKS.Job(
  "Evaluate",
  {
    cluster: eksCluster,
    main: import.meta.url,
    hyperpod: {
      instanceGroup: "workers",
      quota,
      priorityClass: "training",
    },
  },
  Effect.gen(function* () {
    const putItem = yield* AWS.DynamoDB.PutItem(resultsTable);
    return {
      run: Effect.gen(function* () {
        // evaluation logic; bindings land IAM on the pod-identity role
      }),
    };
  }).pipe(Effect.provide(AWS.DynamoDB.PutItemHttp)),
);

Task Governance

// Requires the amazon-sagemaker-hyperpod-taskgovernance EKS add-on.
const policy = yield* AWS.SageMaker.ClusterSchedulerConfig("Scheduler", {
  clusterArn: hyperpod.clusterArn,
  schedulerConfig: {
    PriorityClasses: [{ Name: "training", Weight: 90 }],
    FairShare: "Enabled",
  },
});

// Creates the hyperpod-ns-research namespace + Kueue LocalQueue that
// `hyperpod: { quota }` on an EKS Job/Deployment submits into.
const quota = yield* AWS.SageMaker.ComputeQuota("ResearchQuota", {
  clusterArn: hyperpod.clusterArn,
  computeQuotaTarget: { TeamName: "research", FairShareWeight: 10 },
  computeQuotaConfig: {
    ComputeQuotaResources: [
      { InstanceType: "ml.g5.xlarge", Count: 1 },
    ],
  },
});