QueryGateway

Source: src/Cloudflare/AI/QueryGateway.ts

Binding service that turns a Gateway resource into a typed QueryGatewayClient for Worker runtime code. Wraps the Cloudflare.AI. Gateway runtime binding so each operation returns an Effect tagged with GatewayError, exposes the raw Workers AI handle for ai.run(...), and provides a model(options) factory that produces an effect/unstable/ai LanguageModel Layer.

Bind a Gateway to a Worker and obtain the Effect-native AI Gateway client (run, getUrl, model, …).

QueryGateway is a single identifier that is simultaneously the binding’s Context tag, its type, and the callable — yield* Cloudflare.AI.QueryGateway(gateway).

Calling AI Gateway

Bind the gateway during the Worker’s init phase, then use run or getUrl from request handlers.

const aiGateway = yield* Cloudflare.AI.QueryGateway(gateway);

return {
  fetch: Effect.gen(function* () {
    return yield* aiGateway.run({
      provider: "workers-ai",
      endpoint: "@cf/meta/llama-3.1-8b-instruct",
      headers: { "content-type": "application/json" },
      query: { prompt: "Write a concise status update" },
    });
  }),
};

Driving Effect AI through the gateway

model(options) produces a Layer<LanguageModel, never, RuntimeContext> that translates LanguageModel.generateText / streamText calls (including tool calls and structured outputs) into ai.run(...) against the bound Workers AI model, routed through the gateway.

const aiGateway = yield* Cloudflare.AI.QueryGateway(gateway);

const languageModel = aiGateway.model({
  model: "@cf/meta/llama-3.1-8b-instruct",
  parameters: { temperature: 0.7, maxTokens: 1024 },
});

const response = yield* LanguageModel.generateText({ prompt }).pipe(
  Effect.provide(languageModel),
);

Provide QueryGatewayBinding in the worker’s runtime layer to resolve the underlying Cloudflare.AI. binding at request time.