Modal.Server reference

FetchServer

Source: packages/modal/src/FetchServer.ts

Call a Modal.Server, a Modal.Function with web, or a model Modal.Endpoint from any runtime. Servers and endpoints require Modal’s proxy auth by default: the binding creates a Modal.WebhookToken for the host and sends its Modal-Key / Modal-Secret headers with every request.

A Modal.Server program’s RPC methods come back on the client too. Each call is a POST /__rpc__/<method> to the Server through Modal’s proxy, with the arguments and result as JSON; a method that returns a Stream streams. The binding gives the host its own key for the Server, which every method call carries: the Server answers methods only for callers bound to it. It serves them only behind proxy auth, so an unauthenticated Server answers them with 404.

FetchServer: Call a Server

import * as Cloudflare from "@alchemy.run/cloudflare";
import * as Modal from "@alchemy.run/modal";
import * as Effect from "effect/Effect";
import * as HttpClientRequest from "effect/http/HttpClientRequest";
import * as HttpServerResponse from "effect/http/HttpServerResponse";
import Api from "./Api.ts";

export default class Gateway extends Cloudflare.Worker<Gateway>()(
  "Gateway",
  { main: import.meta.url },
  Effect.gen(function* () {
    const api = yield* Modal.FetchServer(Api);
    return {
      fetch: Effect.gen(function* () {
        const response = yield* api.fetch(HttpClientRequest.get("/health"));
        return HttpServerResponse.text(yield* response.text, { status: response.status });
      }).pipe(Effect.orDie),
    };
  }).pipe(Effect.provide(Modal.FetchServerHttp)),
) {}

FetchServer: Call a Server’s methods

RPC methods next to fetch

src/Api.ts

export default class Api extends Modal.Server<Api>()(
  "Api",
  { main: import.meta.url, app: Backend },
  Effect.gen(function* () {
    return {
      greet: Effect.fn(function* (name: string) {
        return `hello ${name}`;
      }),
      fetch: Effect.succeed(HttpServerResponse.text("ok")),
    };
  }),
) {}

Call them from a Cloudflare Worker

const api = yield* Modal.FetchServer(Api);
const greeting = yield* api.greet("modal");

FetchServerHttp

Source: packages/modal/src/FetchServerHttp.ts Kind: Layer · Provides: Modal.FetchServer

FetchServer over HTTP. Creates one Modal.WebhookToken per host (<host>ProxyAuth) and binds its header pair into the host’s env.

ReadServer

Source: packages/modal/src/ReadServer.ts

Read a Modal.Server’s traffic from any runtime: request log, status codes, latency percentiles, metric series and live containers.

ReadServer: Error rate

const api = yield* Modal.ReadServer(Api);
const stats = yield* api.stats({ since: new Date(Date.now() - 600_000) });

ReadServer: Recent requests

const recent = yield* api.requests({ limit: 20 });

ReadServerHttp

Source: packages/modal/src/ReadServerHttp.ts Kind: Layer · Provides: Modal.ReadServer

ReadServer over Modal’s API. Grants the host’s identity (a scoped Modal.ServiceUser) the viewer role on the server’s environment.

Server

Source: packages/modal/src/Server.ts

A Modal Server: your program as a long-running HTTP server on Modal, with low-latency routing through one region and autoscaling on concurrent requests. Use it for APIs, model servers and anything that keeps connections open; use Modal.Function with web for simple endpoints.

Alchemy bundles main, builds it into an image on Modal’s builder (no Docker) and deploys the Server into its Modal.App. Each container starts the program, and Modal routes the Server’s URL to its fetch handler. Requests need Modal’s proxy auth headers unless unauthenticated is set: bind Modal.FetchServer(server) to call it from another runtime.

Server: Effect program

src/Api.ts

import * as Modal from "@alchemy.run/modal";
import * as Effect from "effect/Effect";
import * as HttpServerResponse from "effect/http/HttpServerResponse";
import { Backend } from "./Backend.ts";

export default class Api extends Modal.Server<Api>()(
  "Api",
  { main: import.meta.url, app: Backend, routingRegion: "us-east" },
  Effect.gen(function* () {
    return {
      fetch: Effect.succeed(HttpServerResponse.text("ok")),
    };
  }),
) {}

Server: Async program

src/Api.ts

import * as Modal from "@alchemy.run/modal";
import { Backend } from "./Backend.ts";

export const Api = Modal.Server("Api", {
  main: "./src/api-server.ts",
  app: Backend,
  unauthenticated: true,
  env: { GREETING: "hello" },
});
export type ApiEnv = Modal.InferEnv<typeof Api>;

src/api-server.ts

import type { ApiEnv } from "./Api.ts";

const env = process.env as unknown as ApiEnv;

Bun.serve({ port: Number(process.env.PORT), fetch: () => new Response(env.GREETING) });

Server: Calling it

const api = yield* Modal.FetchServer(Api);
const response = yield* api.fetch(HttpClientRequest.get("/"));

Server: Scaling

export default class Api extends Modal.Server<Api>()(
  "Api",
  {
    main: import.meta.url,
    app: Backend,
    minContainers: 2,
    targetConcurrency: 20,
    maxConcurrency: 50,
    exitGracePeriod: "30 seconds",
  },
  program,
) {}