Cloudflare.Domain reference

Backfill

Source: packages/cloudflare/src/Domain/Backfill.ts

The backfill of a Domain’s views and projections, run by reconciliation. The Durable Object platform declares it with the model’s consumers: you don’t declare it yourself. A deploy that adds a view or projection, or bumps a version, waits here until every new consumer has received the history recorded before it, and reports how far each one has got.

Backfill: What a deploy waits for

Each consumer generation (a view, or a view after a version bump) has one run, held by one Durable Object of a class only that generation has. The first deploy to reach it starts it; every later start finds it started, so there is never more than one, however many deploys ask. The run lives on Cloudflare and moves on its own: this resource starts it, waits for it, and reports its progress, but holds none of its state.

So a deploy is safe to repeat and to interrupt. Run it again after a failure, a timeout or a Ctrl-C, and it waits on the same runs, from where they are; once they are complete it returns at once. A deploy that changes no consumer plans no change here.

Backfill: Progress

While it waits, the deploy notes each run’s phase and how far through the directory it is:

Backfilling view:Deposits reading history 41% (1,690/4,096 subtrees, 410,211 instances) (48s) ...

A consumer keyed by its source reads its history in one pass; any other reads it, then applies it once every source’s history is in. refused counts calls Cloudflare turned down for the account’s load: the run backs off and sends them again.

Backfill: Rebuilding and cancelling

A run reads the sources its consumer read when it started. A deploy whose consumer reads other sources at the same version (a view that now reads a journal as well as accounts) fails, saying to bump the version: waiting on the run would report a history that misses the new source.

To run a backfill again, bump the consumer’s version: the new generation is a new class with its own run, and the same Worker upload deletes the old generation’s class, data and run. To stop a backfill, remove the consumer from the model and deploy: its classes, and the run with them, are deleted. Either is one Worker upload, so a consumer never has two runs at once.