Custom State Store

Alchemy's state management system is designed to be pluggable, allowing you to implement your own storage backends. This guide walks you through creating a custom state store implementation.

NOTE

This guide builds on the concepts from State Management. Familiarize yourself with how Alchemy handles state before creating a custom state store.

Understanding the StateStore Interface

All state stores in Alchemy implement the StateStore interface:

export interface StateStore {
  /** Initialize the state container if one is required */
  init?(): Promise<void>;
  
  /** Delete the state container if one exists */
  deinit?(): Promise<void>;
  
  /** List all resources in the given stage. */
  list(): Promise<string[]>;
  
  /** Return the number of items in this store */
  count(): Promise<number>;
  
  /** Get a state by key */
  get(key: string): Promise<State | undefined>;
  
  /** Get multiple states by their keys */
  getBatch(ids: string[]): Promise<Record<string, State>>;
  
  /** Get all states in the store */
  all(): Promise<Record<string, State>>;
  
  /** Set a state for a key */
  set(key: string, value: State): Promise<void>;
  
  /** Delete a state by key */
  delete(key: string): Promise<void>;
}

Consistency Requirements

Alchemy relies on consistent state operations, particularly for the get and set methods. When implementing a custom state store:

• Strong Consistency: Operations must be strongly consistent, especially between get and set. If you set a value and immediately get it, you should receive the updated value.
• Atomicity: State changes should be atomic to avoid partial updates that could corrupt the state.
• Durability: Once a state is set, it should be persisted reliably to avoid data loss.

These requirements ensure Alchemy correctly tracks resource state and makes appropriate decisions about resource lifecycle.

Serialization and Special Types

Alchemy uses a special serialization system to handle JavaScript objects, dates, secrets, and other complex types. Always use the provided serialize and deserialize functions from Alchemy to properly handle these types:

import { serialize, deserialize } from "alchemy/serde";

// When storing state:
const serializedData = await serialize(this.scope, value);

// When retrieving state:
const state = await deserialize(this.scope, rawData) as State;

IMPORTANT

For detailed information on Alchemy's serialization system, see the Serialization and Deserialization guide.

Creating a Custom State Store

Let's walk through implementing a custom state store using a cloud storage service. We'll follow the pattern used in Alchemy's built-in state stores.

Basic Structure

Your custom state store should:

1. Accept a scope in the constructor
2. Implement all required StateStore methods
3. Handle serialization and deserialization of state data

Here's a skeleton implementation:

import type { Scope } from "alchemy/scope";
import { deserialize, serialize } from "alchemy/serde";
import type { State, StateStore } from "alchemy/state";

export interface MyCustomStoreOptions {
  // Options specific to your storage backend
  endpoint?: string;
  apiKey?: string;
}

export class MyCustomStateStore implements StateStore {
  constructor(
    public readonly scope: Scope,
    private options: MyCustomStoreOptions
  ) {
    // Initialize any properties needed
  }

  async init(): Promise<void> {
    // Set up the storage backend if needed
  }

  async deinit(): Promise<void> {
    // Clean up resources if needed
  }

  async list(): Promise<string[]> {
    // List all state keys in the store
  }

  async count(): Promise<number> {
    // Return the count of items
    const keys = await this.list();
    return keys.length;
  }

  async get(key: string): Promise<State | undefined> {
    // Get state by key
  }

  async getBatch(ids: string[]): Promise<Record<string, State>> {
    // Get multiple states efficiently
  }

  async all(): Promise<Record<string, State>> {
    // Get all states
    const keys = await this.list();
    return this.getBatch(keys);
  }

  async set(key: string, value: State): Promise<void> {
    // Store state
  }

  async delete(key: string): Promise<void> {
    // Delete state
  }
}

Example: In-Memory State Store

Here's a simple in-memory state store implementation:

/**
 * A simple in-memory state store implementation
 * Note: This is for demonstration - it doesn't persist between runs
 */
export class InMemoryStateStore implements StateStore {
  // Map to store the state data in memory
  private stateMap: Map<string, any> = new Map();
  
  constructor(
    public readonly scope: Scope,
    private options: { namespace?: string } = {}
  ) {
    // Create a scope-specific namespace for the state
    this.namespace = options.namespace || scope.chain.join('/');
  }

  // Optional init method, not really needed for in-memory store
  async init(): Promise<void> {
    // Nothing to initialize for in-memory store
    console.log(`Initialized in-memory state store for scope: ${this.namespace}`);
  }

  // Optional cleanup method
  async deinit(): Promise<void> {
    // Clear all state for this scope
    const keyPrefix = `${this.namespace}/`;
    
    for (const key of this.stateMap.keys()) {
      if (key.startsWith(keyPrefix)) {
        this.stateMap.delete(key);
      }
    }
  }

  // List all resources in this scope
  async list(): Promise<string[]> {
    const keyPrefix = `${this.namespace}/`;
    const result: string[] = [];
    
    for (const key of this.stateMap.keys()) {
      if (key.startsWith(keyPrefix)) {
        // Remove the prefix and return the actual resource ID
        result.push(key.substring(keyPrefix.length));
      }
    }
    
    return result;
  }

  // Return the count of items in this scope
  async count(): Promise<number> {
    return (await this.list()).length;
  }

  // Get a state by key
  async get(key: string): Promise<State | undefined> {
    const fullKey = `${this.namespace}/${key}`;
    const serializedState = this.stateMap.get(fullKey);
    
    if (!serializedState) {
      return undefined;
    }
    
    // Deserialize the state
    const state = await deserialize(this.scope, serializedState) as State;
    
    // Ensure scope is set on output
    return {
      ...state,
      output: {
        ...(state.output || {}),
        Scope: this.scope,
      },
    };
  }

  // Get multiple states
  async getBatch(ids: string[]): Promise<Record<string, State>> {
    const result: Record<string, State> = {};
    
    for (const id of ids) {
      const state = await this.get(id);
      if (state) {
        result[id] = state;
      }
    }
    
    return result;
  }

  // Get all states in this scope
  async all(): Promise<Record<string, State>> {
    const keys = await this.list();
    return this.getBatch(keys);
  }

  // Set a state
  async set(key: string, value: State): Promise<void> {
    const fullKey = `${this.namespace}/${key}`;
    
    // Serialize the state to handle cycles
    const serializedData = await serialize(this.scope, value);
    
    // Store in the map
    this.stateMap.set(fullKey, serializedData);
  }

  // Delete a state
  async delete(key: string): Promise<void> {
    const fullKey = `${this.namespace}/${key}`;
    this.stateMap.delete(fullKey);
  }
}

Key Implementation Details

When implementing a custom state store, pay attention to these important details:

1. State Serialization

Always use Alchemy's serialize and deserialize functions to handle state data:

// When storing state:
const serializedData = await serialize(this.scope, value);

// When retrieving state:
const state = await deserialize(this.scope, rawData) as State;

These functions handle cycles in the state graph and encrypt/decrypt secrets.

2. Key Naming and Paths

State keys often include characters that may be problematic in certain storage systems (like slashes). Consider encoding/decoding keys:

private convertKeyForStorage(key: string): string {
  return key.replaceAll("/", ":");
}

private convertKeyFromStorage(key: string): string {
  return key.replaceAll(":", "/");
}

3. Scope Awareness

State stores should be aware of their scope and use it to organize data:

constructor(public readonly scope: Scope, options: Options) {
  const scopePath = scope.chain.join("/");
  this.namespace = `alchemy/${scopePath}`;
}

4. Error Handling

Be sure to handle common errors gracefully:

• Return undefined for missing states
• Throw clear errors for authentication/permission issues
• Consider implementing retry logic for transient failures

5. Setting Scope on Output

When returning a state, always ensure the scope is set on the output:

return {
  ...state,
  output: {
    ...(state.output || {}),
    Scope: this.scope,
  },
};

Using a Custom State Store

To use your custom state store, pass it to the Alchemy app initialization:

const app = await alchemy("my-app", {
  stage: "prod",
  phase: process.argv.includes("--destroy") ? "destroy" : "up",
  stateStore: (scope) => new InMemoryStateStore(scope)
});

// ... resource declarations ...

await app.finalize();

Testing Your State Store

It's important to thoroughly test your state store implementation:

// Create a test file for your state store
import { describe, expect } from "bun:test";
import { alchemy } from "alchemy";
import { InMemoryStateStore } from "./in-memory-state-store";

const test = alchemy.test(import.meta)

describe("InMemoryStateStore", () => {
  
  test("basic operations", async (scope) => {
    const store = new InMemoryStateStore(scope);
    await store.init();
    
    // Test basic operations
    await store.set("test-key", { /* sample state */ });
    const state = await store.get("test-key");
    expect(state).toBeDefined();
    
    // Test list and count
    const keys = await store.list();
    expect(keys).toContain("test-key");
    
    // Test delete
    await store.delete("test-key");
    const deletedState = await store.get("test-key");
    expect(deletedState).toBeUndefined();
    
    await store.deinit();
  });
});