ServerlessCluster

Source: src/AWS/Kafka/ServerlessCluster.ts

An Amazon MSK (Managed Streaming for Apache Kafka) Serverless cluster.

MSK Serverless clusters use IAM authentication exclusively and scale broker capacity automatically — there is no broker count, instance type, or storage to configure. They are reachable only from inside a VPC. Creation takes roughly 5-10 minutes.

The provisioned (broker-count) MSK cluster is a separate, much slower (~20-40 minute) resource and is intentionally not modeled here.

Creating a Serverless Cluster

const cluster = yield* ServerlessCluster("Events", {
  subnetIds: [subnetA.subnetId, subnetB.subnetId],
  securityGroupIds: [kafkaSecurityGroup.securityGroupId],
});

Consuming from a Lambda Function

yield* Kafka.consumeKafkaTopic(cluster, { topics: ["orders"] }, (records) =>
  records.pipe(Stream.runForEach((r) => Effect.log(r.value))),
);