VirtualCluster

Source: src/AWS/EMRContainers/VirtualCluster.ts

An Amazon EMR on EKS virtual cluster — a registration of a Kubernetes namespace on an EKS cluster as an EMR job-submission target. Virtual clusters consume no resources themselves (no cost while idle); jobs submitted to the virtual cluster run as pods in the mapped namespace.

The underlying EKS cluster must grant Amazon EMR on EKS access to the namespace (via EKS access entries when the cluster’s authentication mode includes API, or the legacy aws-auth ConfigMap). Everything except tags is immutable — changes replace the virtual cluster.

Creating Virtual Clusters

import * as AWS from "alchemy/AWS";

const cluster = yield* AWS.EKS.Cluster("Cluster", {
  roleArn: clusterRole.roleArn,
  resourcesVpcConfig: { subnetIds },
  accessConfig: { authenticationMode: "API_AND_CONFIG_MAP" },
});

const virtualCluster = yield* AWS.EMRContainers.VirtualCluster("Spark", {
  containerProvider: {
    id: cluster.clusterName,
    info: { eksInfo: { namespace: "emr" } },
  },
});
// virtualCluster.virtualClusterId is passed to StartJobRun