Durable Workflows

A durable workflow is a long-running business process whose execution state can be safely persisted, suspended, and resumed — even across process restarts and deploys. You write the workflow body as ordinary Effect.gen code, but every side effect happens at a named, schema-backed boundary so the engine can record what already happened and replay the rest. The result is a coordinating process that can sleep for days, wait for a webhook or human approval, retry failed steps, and run compensation on failure — without you hand-rolling a state machine in a database.

Unstable API

Everything in this section is unstable and lives under effect/unstable/workflow. Import the modules from the barrel:

import {
  Workflow,
  Activity,
  DurableClock,
  DurableDeferred,
  DurableQueue,
  WorkflowEngine,
  WorkflowProxy,
  WorkflowProxyServer
} from "effect/unstable/workflow"

or directly from a module path, e.g. effect/unstable/workflow/Workflow. Unstable modules may change between minor releases.

The mental model

A durable workflow is assembled from a handful of cooperating pieces. The golden rule that ties them together: keep external side effects at activity (and durable primitive) boundaries, and keep all names / schemas / idempotency keys stable across deploys — they are persisted coordination state, not just labels.

• Workflow — a typed durable definition: a stable name, a struct payload schema, success and error schemas, and an idempotencyKey function. From the name + idempotency key the engine derives a deterministic executionId, so the same logical request always maps to the same execution (reusing it observes the existing run instead of starting a duplicate).
• Activity — a named, schema-backed boundary around a single side effect (call an API, write a row, enqueue a job). Activities encode their success/failure with schemas so a resumed workflow can replay them from persisted results instead of rerunning the effect.
• DurableClock — a workflow-safe sleep. Short delays run in-process; longer delays become a durable timer the engine schedules and resumes via a deferred.
• DurableDeferred — a named wait point whose result is stored as an encoded Exit. The workflow awaits it and suspends; an activity, worker, timer, or external callback completes it later via a token.
• DurableQueue — delegate work to persisted background workers. The workflow processes an item and suspends; a worker takes it, runs the handler, and reports the result back through a deferred.
• WorkflowEngine — the runtime boundary that persists, suspends, and resumes executions. WorkflowEngine.layerMemory is an in-process engine for tests and local development.
• WorkflowProxy / WorkflowProxyServer — derive RPC or HTTP API contracts from workflow definitions so external callers can start, discard, or resume workflows.

Determinism and idempotency

The execution ID is a hash of name + idempotencyKey(payload). This is what makes durable execution work:

• Replay safety — when the engine resumes a suspended workflow it re-runs the workflow body from the top, but activities and deferreds short-circuit to their persisted results, so only the not-yet-completed work actually executes again.
• De-duplication — executing a workflow with a payload that maps to an existing executionId joins the in-flight run rather than starting a second one.

Because of replay, the workflow body must be deterministic outside of activity boundaries: do all I/O, randomness, and clock reads inside activities or durable primitives, and never derive name / idempotencyKey / activity names from ambient state. Changing a persisted name or schema after a workflow is live is a migration, not a refactor.

End-to-end example

A small order-processing workflow: charge the card (an activity), wait an hour for a cancellation window (a durable clock), then send a confirmation email (another activity). We register it with Workflow.toLayer and run it on the in-memory engine.

import { Effect, Layer, Schema } from "effect"
import { Activity, DurableClock, Workflow, WorkflowEngine } from "effect/unstable/workflow"

// 1. Define the durable workflow: stable name, payload struct, result schemas,
//    and a deterministic idempotency key derived from the payload.
const ProcessOrder = Workflow.make({
  name: "ProcessOrder",
  payload: {
    orderId: Schema.String,
    email: Schema.String,
    amountCents: Schema.Number
  },
  success: Schema.String, // the confirmation id we return
  idempotencyKey: ({ orderId }) => orderId
})

// 2. Implement the workflow body. Use Effect.fnUntraced (not a plain fn that
//    returns Effect.gen) so each activity/clock runs at a durable boundary.
const ProcessOrderLayer = ProcessOrder.toLayer(
  Effect.fnUntraced(function* (payload) {
    // Side effect #1, replayable from its persisted result on resume.
    const chargeId = yield* Activity.make({
      name: "ChargeCard",
      success: Schema.String,
      execute: Effect.succeed(`charge_${payload.orderId}`)
    })

    // Durable sleep: short delays run in-process, longer ones become a
    // schedulable timer that survives suspension.
    yield* DurableClock.sleep({ name: "CancellationWindow", duration: "1 hour" })

    // Side effect #2.
    yield* Activity.make({
      name: "SendConfirmationEmail",
      execute: Effect.log(`Emailing ${payload.email} for charge ${chargeId}`)
    })

    return chargeId
  })
)

// 3. Provide the engine. layerMemory is for dev/tests only — see the production
//    note below.
const MainLayer = ProcessOrderLayer.pipe(Layer.provide(WorkflowEngine.layerMemory))

// 4. Execute the workflow with a typed payload.
const program = Effect.gen(function* () {
  const confirmationId = yield* ProcessOrder.execute({
    orderId: "order-123",
    email: "buyer@example.com",
    amountCents: 4999
  })
  yield* Effect.log(`done: ${confirmationId}`) // => done: charge_order-123
}).pipe(Effect.provide(MainLayer))

Production durability

WorkflowEngine.layerMemory keeps all state in process memory and is not durable — use it only for tests and local development. For production durability, run workflows on the cluster engine: ClusterWorkflowEngine.layer from effect/unstable/cluster/ClusterWorkflowEngine, which persists execution state and coordinates retries, resumes, and timers across the fleet. See Cluster.

The grab bag

The handful of things you reach for most when wiring up a workflow. Each module has its own page (linked below) with the complete reference; this is the shortlist.

Define and execute a workflow

import { Effect, Schema } from "effect"
import { Workflow } from "effect/unstable/workflow"

const SendEmail = Workflow.make({
  name: "SendEmail",
  payload: { to: Schema.String },
  success: Schema.String,
  idempotencyKey: ({ to }) => to // stable per logical request
})

// Start (or join) a run and get the typed success value
SendEmail.execute({ to: "a@example.com" }) // => Effect<"sent">

// Fire-and-forget: returns the executionId instead of waiting
SendEmail.execute({ to: "b@example.com" }, { discard: true }) // => Effect<string>

Register the body with toLayer

Use Effect.fnUntraced (not a plain function returning Effect.gen) so every activity and durable primitive runs at a replayable boundary.

import { Effect } from "effect"

const SendEmailLayer = SendEmail.toLayer(
  Effect.fnUntraced(function* (payload, executionId) {
    yield* Effect.log(`[${executionId}] sending to ${payload.to}`)
    return "sent"
  })
)
// => Layer<never, never, WorkflowEngine>

Wrap a side effect in an activity

An activity is an Effect, so you yield* it directly. Its result is persisted and replayed instead of rerun on resume.

import { Effect, Schema } from "effect"
import { Activity } from "effect/unstable/workflow"

const FetchUser = Activity.make({
  name: "FetchUser",
  success: Schema.Struct({ id: Schema.String }),
  execute: Effect.succeed({ id: "u1" })
})
// yield* FetchUser inside a workflow => { id: "u1" }

Sleep durably

Short delays run in-process; longer ones become a schedulable timer that survives suspension.

import { DurableClock } from "effect/unstable/workflow"

DurableClock.sleep({ name: "ReminderDelay", duration: "3 days" })
// => Effect<void> (suspends the workflow until the timer fires)

Wait for an external signal

A DurableDeferred is a named wait point. The workflow awaits it and suspends; a webhook or worker completes it later via a token.

import { Schema } from "effect"
import { DurableDeferred } from "effect/unstable/workflow"

const Approval = DurableDeferred.make("Approval", { success: Schema.Boolean })
DurableDeferred.await(Approval) // => suspends until completed

// From outside (e.g. a webhook handler), using the deferred's token:
declare const token: DurableDeferred.Token
DurableDeferred.succeed(Approval, { token, value: true })

Run it on an engine

WorkflowEngine.layerMemory is the in-process engine for tests and local dev. For production durability, use ClusterWorkflowEngine.layer — see Engine & execution.

import { Layer } from "effect"
import { WorkflowEngine } from "effect/unstable/workflow"

const MainLayer = SendEmailLayer.pipe(Layer.provide(WorkflowEngine.layerMemory))

In this section

• Workflows & Activities — define workflows with Workflow.make, register bodies with toLayer, run them with execute / poll / interrupt / resume, and wrap side effects in Activity.make with retries, idempotency keys, and compensation.
• Durable primitives — workflow-safe sleeps (DurableClock), named wait points (DurableDeferred) with tokens and durable races, and background workers (DurableQueue).
• Engine & execution — the WorkflowEngine runtime boundary, the execution lifecycle, the in-memory engine, the production cluster engine, and building a custom engine with makeUnsafe.
• Exposing workflows — derive RPC and HTTP API contracts from workflow definitions with WorkflowProxy and serve them with WorkflowProxyServer.