InvokeEndpointWithResponseStream

Source: src/AWS/SageMakerRuntime/InvokeEndpointWithResponseStream.ts

Runtime binding for sagemaker:InvokeEndpointWithResponseStream — the streaming variant of InvokeEndpoint. The inference response arrives incrementally as an event Stream of PayloadPart events carrying raw model-specific bytes; the container behind the endpoint must support inference streaming.

The binding takes one or more endpoint names and grants the function sagemaker:InvokeEndpoint (the IAM action the streaming operation authorizes against) scoped to exactly those endpoint ARNs.

Streaming an Endpoint Response

// init
const invokeStream = yield* AWS.SageMakerRuntime.InvokeEndpointWithResponseStream(
  "my-streaming-endpoint",
);

// runtime — Body is the raw payload the container expects
const result = yield* invokeStream({
  ContentType: "application/json",
  Body: JSON.stringify({ inputs: "Say hello." }),
});
const events = yield* Stream.runCollect(result.Body);
// each PayloadPart's Bytes is a model-specific chunk