What is Effect Cluster?

Effect Cluster is Effect’s library for stateful services spread across many processes. Its center is the entity: a small, long-lived program with an id and its own state, like one chat room. Effect Cluster runs each instance on one process at a time, delivers its messages in order, and moves it when processes come and go. The processes are runners, and they agree on who runs what through a Postgres database.

Take one small app: chat rooms. Each room is an instance of the Room entity, and an Api container calls a room by its name. Two runners, in a pool called Default, hold the rooms between them, and the database records which runner owns each room’s shard. With Alchemy all of it deploys from one Stack, here on a Docker swarm:

Docker swarmyour cloud

ApiDocker Service

DatabasePostgres

shard locks

300

messages

persisted

Defaultpool

Runner 1shards 1–150

lobbyRoom

Runner 2shards 151–300

kitchenRoom

Each room runs inside the runner that owns its shard. The database records the owners, so either runner can take over the other's shards.

Entities: a class and its instances

An entity starts from the messages it accepts, as Effect RPC schemas, the same arguments Effect’s Entity.make takes:

const Rooms = Entity.make("Room", [
  Rpc.make("Join", { payload: { name: Schema.String }, success: Schema.Int }),
  Rpc.make("Members", { success: Schema.Array(Schema.String) }),
]);

An instance is one room: the lobby, the kitchen. Every instance runs the same code against its own state.

See Entities.

One instance per id, one message at a time

You never create an instance. Every id exists from the start, and the first message to an id brings its instance to life:

src/Api.ts

import * as Docker from "@alchemy.run/docker";
import * as Effect from "effect/Effect";
import * as HttpServerResponse from "effect/http/HttpServerResponse";
import Room from "./Room.ts";

export default class Api extends Docker.Service<Api>()(
  "Api",
  { main: import.meta.url, port: 3000 },
  Effect.gen(function* () {
    const rooms = yield* Room;
    return {
      fetch: Effect.gen(function* () {
        const lobby = yield* rooms.getByName("lobby");
        return yield* HttpServerResponse.json({ members: yield* lobby.Join({ name: "sam" }) });
      }).pipe(Effect.orDie),
    };
  }),
) {}

However many callers message the lobby, from however many servers, they reach the same instance, and it handles their messages one at a time. That’s what lets a room keep its members in a plain Ref without locks: nothing else ever touches them.

An instance’s code and its state

The class takes the messages with the entity’s code, in two phases. The outer Effect runs once per runner and resolves bindings. The inner one runs once per instance, on its first message, and returns its handlers:

src/Room.ts

import * as Cluster from "@alchemy.run/effect-cluster";
import { Entity } from "effect/cluster";
import * as Effect from "effect/Effect";
import * as Ref from "effect/Ref";
import { Rpc } from "effect/rpc";
import * as Schema from "effect/Schema";
import { Default } from "./Cluster.ts";

const Rooms = Entity.make("Room", [
  Rpc.make("Join", { payload: { name: Schema.String }, success: Schema.Int }),
  Rpc.make("Members", { success: Schema.Array(Schema.String) }),
]);

export default class Room extends Cluster.Entity<Room>()(
  Rooms,
  { main: import.meta.url },
  Effect.gen(function* () {
    return Effect.gen(function* () {
      const members = yield* Ref.make(new Set<string>());
      return Rooms.of({
        Join: ({ payload }) =>
          Ref.modify(members, (all) => {
            const next = new Set(all).add(payload.name);
            return [next.size, next];
          }),
        Members: () => Effect.map(Ref.get(members), (all) => [...all]),
      });
    });
  }).pipe(Effect.provide(Default)),
) {}

An instance’s state lives in memory. When it sits idle (a minute, by default), when its runner restarts, or when a deploy replaces the runners, the instance stops and its state is gone. The next message starts it fresh. Keep anything that must survive in a database the instance reads on its first message.

Runners run the instances

Runners are the processes that host instances. A pool is a group of runners that run the same code, and Default is one:

src/Cluster.ts

import * as ClusterDocker from "@alchemy.run/effect-cluster/Docker";
import * as Layer from "effect/Layer";

export const Database = ClusterDocker.Database("Cluster");

export const Default = ClusterDocker.SwarmPool("Default", { replicas: 2 }).pipe(
  Layer.provide(Database),
);

Providing Default to the room’s code is what deploys it. Every class given the same pool is bundled into one program, built into one image, and run on every runner in the pool. A second pool is its own shard group: its classes only run on its runners.

See Pools.

Shards and the database

Effect Cluster hashes each instance’s id to one of 300 shards, and each runner owns some of the shards. A message to the lobby goes to whichever runner owns the lobby’s shard.

The runners agree on who owns what through the database: a Postgres they all reach. Each runner registers itself there and takes locks on its shards. When a runner stops, its locks lapse, and the others take its shards over, along with the instances in them.

An id always hashes to the same shard; the shard's owner can change.

See Databases.

Persisted messages

By default a message travels straight to its runner. Mark an RPC persisted, and the message is written to the database first, delivered at least once, and handled once per primary key:

const Rooms = Entity.make("Room", [
  Rpc.make("Post", {
    payload: { messageId: Schema.String, text: Schema.String },
    primaryKey: ({ messageId }) => messageId,
  }).annotate(ClusterSchema.Persisted, true),
]);

A persisted message survives its runner restarting: whichever runner owns the shard next delivers it.

The gateway

Callers outside the cluster, like a Worker or a Lambda function, don’t reach the database or the runners. They reach the gateway: each pool serves its classes over HTTP, behind a token every caller is given. On AWS, GCP and Hetzner a load balancer puts it on the internet; on Fly it’s the pool’s fly.dev address. On Cloudflare the pool is one Worker and each instance is a Durable Object, so there are no shards and no database.

Inside the cluster, one entity calling another skips the gateway: the call goes straight from runner to runner.

See Calling an entity.

Workflows, singletons and crons

Effect Cluster runs three more kinds of programs in the same pool, declared the same way as an entity. A workflow is a durable function: each step is persisted, it can sleep for days or wait for a signal, and it picks up where it left off on another runner. A singleton is one fiber running on exactly one runner. A cron is a job that runs on a schedule, once per scheduled time across the cluster:

src/Cleanup.ts

import * as Cluster from "@alchemy.run/effect-cluster";
import * as Effect from "effect/Effect";
import { Default } from "./Cluster.ts";

export default class Cleanup extends Cluster.Cron<Cleanup>()(
  "Cleanup",
  { main: import.meta.url, cron: "0 3 * * *" },
  Effect.gen(function* () {
    return Effect.logInfo("closing empty rooms");
  }).pipe(Effect.provide(Default)),
) {}

See Workflows, Singletons and Crons.

How @alchemy.run/effect-cluster models these concepts

Each cloud’s entry point, like @alchemy.run/effect-cluster/Docker, has a Database Layer that runs Postgres on that cloud and a pool Layer that runs the runners: SwarmPool, EcsPool, ComputeEnginePool, DeploymentPool, MachinePool or ServerPool. Cluster.Entity, Cluster.Workflow, Cluster.Singleton and Cluster.Cron are classes, and providing a pool to one deploys it. yield* Room from any runtime returns a client that reaches the pool’s gateway, with its URL and token given to that runtime. On Cloudflare, DurableObjectPool runs each instance as a Durable Object instead.

Next, Setup, then the tutorial.