Autoscaling

Modal scales a function’s containers with its calls, down to none when idle. A Modal.FunctionAutoscaler sets the bounds: containers kept warm, spare ones ready for the next spike, and a cap on how many may run.

Warm containers skip the cold start

With minContainers: 2 and bufferContainers: 1, two containers wait for calls and a third stands ready while they’re busy:

callwarm, waiting

The two minimum containers never stop. When both are busy, Modal starts a buffer container, so the next call doesn't wait for a cold start.

Keep containers warm

src/Scaling.ts

import * as Modal from "@alchemy.run/modal";

export const Warm = Modal.FunctionAutoscaler("Warm", {
  app: "inference",
  function: "Model.*",
  minContainers: 2,
  bufferContainers: 1,
  scaledownWindow: "5 minutes",
});

scaledownWindow is how long an idle container above the minimum waits before it stops.

Cap concurrency

src/Scaling.ts

export const Capped = Modal.FunctionAutoscaler("Capped", {
  app: "inference",
  function: "embed",
  maxContainers: 10,
});

Settings return when you destroy it

Destroying the resource restores the settings it found. modal deploy resets them, so deploy your Stack after redeploying the app.