Modal.Function reference

Function

Source: packages/modal/src/Function.ts

A Modal Function running your program: Modal scales its containers with demand (to zero when idle), and the function takes calls, runs on a schedule, or serves the web.

Alchemy bundles main, builds it into an image on Modal’s builder (no Docker), and deploys the function into its Modal.App. Modal starts each container with its Python entrypoint; Alchemy’s entrypoint starts your program and hands it work:

• the program’s RPC methods (every function it returns besides fetch) become the function’s methods: Modal.InvokeFunction calls them with typed arguments and results, and Modal queues, retries and autoscales the calls;
• with web, Modal routes the function’s *.modal.run URL to the program’s fetch handler;
• a schedule tick arrives at fetch as POST /__modal/schedule.

A function serves one of these. To serve methods next to fetch, use a Modal.Server. The program’s env, including the credentials its bindings need, goes into a Modal Secret, never into the image.

Function: Effect program

A function other runtimes call

src/Embed.ts

import * as Modal from "@alchemy.run/modal";
import * as Effect from "effect/Effect";
import { Inference } from "./Inference.ts";

export default class Embed extends Modal.Function<Embed>()(
  "Embed",
  { main: import.meta.url, app: Inference },
  Effect.gen(function* () {
    return {
      embed: Effect.fn(function* (text: string) {
        return { tokens: text.split(/\s+/).length };
      }),
    };
  }),
) {}

Call it from a Cloudflare Worker

const embed = yield* Modal.InvokeFunction(Embed);
const { tokens } = yield* embed.embed("hello modal");

Function: Async program

src/Embed.ts

import * as Modal from "@alchemy.run/modal";
import { Inference } from "./Inference.ts";

export const Embed = Modal.Function("Embed", {
  main: "./src/embed-server.ts",
  app: Inference,
  env: { MODEL: "small" },
});
export type EmbedEnv = Modal.InferEnv<typeof Embed>;

src/embed-server.ts

import type { EmbedEnv } from "./Embed.ts";

const env = process.env as unknown as EmbedEnv;

Bun.serve({
  port: Number(process.env.PORT),
  async fetch(request) {
    const { args } = (await request.json()) as { args: [string] };
    return Response.json({ model: env.MODEL, tokens: args[0].split(/\s+/).length });
  },
});

Function: Web endpoints

Serve fetch at a *.modal.run URL

export default class Api extends Modal.Function<Api>()(
  "Api",
  { main: import.meta.url, app: Inference, web: true },
  Effect.gen(function* () {
    return { fetch: Effect.succeed(HttpServerResponse.text("hello from Modal")) };
  }),
) {}

Behind Modal’s proxy auth

export default class Admin extends Modal.Function<Admin>()(
  "Admin",
  { main: import.meta.url, app: Inference, web: { requiresProxyAuth: true, label: "admin" } },
  program,
) {}

Function: Schedules

export default class Digest extends Modal.Function<Digest>()(
  "Digest",
  { main: import.meta.url, app: Inference, schedule: { cron: "0 9 * * *", timezone: "Europe/London" } },
  Effect.gen(function* () {
    return {
      fetch: Effect.gen(function* () {
        yield* Effect.log("sending the digest");
        return yield* HttpServerResponse.json(null);
      }),
    };
  }),
) {}

Function: GPUs, volumes and scaling

export default class Generate extends Modal.Function<Generate>()(
  "Generate",
  {
    main: import.meta.url,
    app: Inference,
    gpu: "L4",
    mounts: [{ volume: Weights, path: "/models" }],
    minContainers: 1,
    scaledownWindow: 300,
    maxConcurrentInputs: 8,
    timeout: "10 minutes",
  },
  program,
) {}

FunctionAutoscaler

Source: packages/modal/src/FunctionAutoscaler.ts

Autoscaling settings for a Modal Function deployed with Modal’s Python client (modal deploy): warm containers, a container cap, buffer containers and scale-down timing, changed without redeploying the app.

For a Modal.Function or Modal.Server of the Stack, set these on the resource itself. This resource adjusts a function deployed outside the Stack, and restores the settings it found when destroyed. Modal resets overrides like these when the app is redeployed, so redeploying the app and then this stack applies them again.

FunctionAutoscaler: Keeping containers warm

import * as Modal from "@alchemy.run/modal";

export const Warm = Modal.FunctionAutoscaler("Warm", {
  app: "inference",
  function: "Model.*",
  minContainers: 2,
  bufferContainers: 1,
  scaledownWindow: "5 minutes",
});

InvokeFunction

Source: packages/modal/src/InvokeFunction.ts

Call a Modal Function from any runtime: a Worker, a Lambda, a Durable Object, another Modal Function. Bind a Modal.Function of the Stack, or a function deployed with modal deploy by name.

A Modal.Function program’s RPC methods come back as a typed client: each call goes through Modal (queued, retried and autoscaled like any Modal call) to the method of that name. Arguments and results are JSON values, and a typed failure reaches the caller by its _tag. A method that returns a Stream arrives whole, once the call finishes. The binding gives the host its own key for the function, which every call carries: the program answers only callers bound to it.

A Python function, or an async program, gets a FunctionClient instead: call, map and spawn with CBOR arguments (Modal’s cross-language format). Payloads over 2 MiB go through Modal’s blob store.

InvokeFunction: Call a function

RPC methods of a Modal.Function

src/Embed.ts

import * as Modal from "@alchemy.run/modal";
import * as Effect from "effect/Effect";
import { Inference } from "./Inference.ts";

export default class Embed extends Modal.Function<Embed>()(
  "Embed",
  { main: import.meta.url, app: Inference },
  Effect.gen(function* () {
    return {
      embed: Effect.fn(function* (text: string) {
        return { tokens: text.split(/\s+/).length };
      }),
    };
  }),
) {}

Call it from a Cloudflare Worker

import * as Cloudflare from "@alchemy.run/cloudflare";
import * as Modal from "@alchemy.run/modal";
import * as Effect from "effect/Effect";
import * as HttpServerResponse from "effect/http/HttpServerResponse";
import Embed from "./Embed.ts";

export default class Api extends Cloudflare.Worker<Api>()(
  "Api",
  { main: import.meta.url, compatibility: { flags: ["nodejs_compat"] } },
  Effect.gen(function* () {
    const embed = yield* Modal.InvokeFunction(Embed);
    return {
      fetch: Effect.gen(function* () {
        const { tokens } = yield* embed.embed("hello world");
        return yield* HttpServerResponse.json({ tokens });
      }).pipe(Effect.orDie),
    };
  }).pipe(Effect.provide(Modal.InvokeFunctionHttp)),
) {}

A Python function deployed with modal deploy

const embed = yield* Modal.InvokeFunction({ app: "inference", function: "embed" });
const vector = yield* embed.call<number[]>(["hello world"]);

A method of a parameterized Python class

const model = yield* Modal.InvokeFunction({
  app: "inference",
  function: "Model.generate",
  parameters: { size: "7b" },
});
const text = yield* model.call<string>(["hello"]);

InvokeFunction: Fan out

const results = yield* embed.map<{ tokens: number }>([["a"], ["b"], ["c"]]);

InvokeFunction: Fire and forget

const job = yield* train.spawn([], { epochs: 3 });
const metrics = yield* train.result(job, { timeout: "30 minutes" });

InvokeFunctionHttp

Source: packages/modal/src/InvokeFunctionHttp.ts Kind: Layer · Provides: Modal.InvokeFunction

InvokeFunction over Modal’s API. Grants the host’s identity (a scoped Modal.ServiceUser) the contributor role on the function’s environment.

ReadFunction

Source: packages/modal/src/ReadFunction.ts

Read a Modal Function’s (or Server’s) live state from any runtime: backlog and running inputs, call counts, outcome counts, latency percentiles and metric series. Bind a resource of the Stack, or a function deployed with modal deploy by name.

ReadFunction: Watch the backlog

const embed = yield* Modal.ReadFunction(Embed);
const { backlog } = yield* embed.stats();
if (backlog > 100) return HttpServerResponse.text("busy", { status: 503 });

ReadFunction: History

const stats = yield* embed.timeRangeStats({ since: new Date(Date.now() - 3_600_000) });
yield* Effect.log(`${stats.inputFailureCount} failed of ${stats.inputSuccessCount}`);

ReadFunctionHttp

Source: packages/modal/src/ReadFunctionHttp.ts Kind: Layer · Provides: Modal.ReadFunction

ReadFunction over Modal’s API. Grants the host’s identity (a scoped Modal.ServiceUser) the viewer role on the function’s environment.

ScaleFunction

Source: packages/modal/src/ScaleFunction.ts

Scale a Modal Function or Server from a running program: keep containers warm ahead of a launch, or cap them at night. Overrides last until the next deploy; use the resource’s own props (or Modal.FunctionAutoscaler for a function deployed outside the Stack) for the steady state.

ScaleFunction: Warm up ahead of traffic

const generate = yield* Modal.ScaleFunction(Generate);
yield* generate.update({ minContainers: 4, bufferContainers: 2 });

ScaleFunctionHttp

Source: packages/modal/src/ScaleFunctionHttp.ts Kind: Layer · Provides: Modal.ScaleFunction

ScaleFunction over Modal’s API. Grants the host’s identity (a scoped Modal.ServiceUser) the contributor role on the function’s environment.