Persistence

The effect/unstable/persistence package provides a small stack of durable storage primitives: a low-level key/value store, a schema-aware store for request results, and three higher-level building blocks (a durable cache, a durable job queue, and a shared rate limiter). Each layer builds on the one below it, and a single Redis service can power all of the Redis-backed variants at once.

import {
  KeyValueStore,
  Persistable,
  PersistedCache,
  PersistedQueue,
  Persistence,
  RateLimiter,
  Redis
} from "effect/unstable/persistence"

Unstable API

Every module in this section lives under effect/unstable/persistence. The unstable namespace is not yet covered by the stability guarantees of the core effect package, so import paths and signatures may change between minor releases. Pin your Effect version if you depend on these modules in production.

What is the persistence package?

Think of the package as layers stacked from raw bytes up to ready-made features:

• KeyValueStore — the lowest level. A Map-like service over strings, numbers, JSON, Uint8Array, and schema-encoded values, backed by memory, the filesystem, SQL, or the Web Storage API. Use it for small, lightweight durable state.
• Persistence / Persistable — a schema-aware store that reads and writes encoded Exit values keyed by each request’s PrimaryKey. It is the layer that knows how to serialize a request result (success or error) with the request’s schemas and apply per-entry TTLs.
• PersistedCache — durable memoization. Looks up a Persistable request, returns the stored result if present, otherwise runs your lookup and persists the Exit. Survives restarts and is shared across processes that share a backing store.
• PersistedQueue — a durable, at-least-once job queue (an outbox). Offers are persisted before they are taken, so work survives crashes.
• RateLimiter — shared fixed-window / token-bucket limits enforced across fibers and processes (e.g. a cross-process Redis store).

Redis is not a feature module — it is the integration seam underneath all the Redis-backed layers. You adapt an existing Redis client into a Redis service, and the Persistence, PersistedQueue, and RateLimiter Redis layers all consume it.

Choosing a module

Lightweight durable state

Small string / number / JSON / binary values you want to read and write like a Map.

Use KeyValueStore.

Memoize expensive requests

Idempotent lookups that are slow or costly and should be reused across restarts and workers.

Use PersistedCache.

Durable background jobs

An outbox / job queue where each item must be processed at least once even if the process crashes.

Use PersistedQueue.

Cross-process quotas

Shared throttling / rate limits enforced across multiple fibers or processes.

Use RateLimiter.

Raw encoded Exit storage

Direct control over schema-encoded Exit values keyed by request primary key, beneath the cache.

Use Persistence.

Task	Module
Lightweight durable string/binary/JSON state	KeyValueStore
Memoize expensive idempotent requests across restarts	PersistedCache
Durable background jobs / outbox	PersistedQueue
Cross-process quotas / throttling	RateLimiter
Raw schema-encoded Exit storage	Persistence

Backing stores at a glance

Most modules are split into a high-level service plus a pluggable backing store layer that decides where data physically lives. Choose a backing layer and provide it (along with Redis, SqlClient, or a KeyValueStore when the chosen layer requires one).

Module	Memory	KeyValueStore	SQL	Redis	Other
KeyValueStore	layerMemory	—	layerSql	—	layerStorage (Web Storage), layerFileSystem (disk)
Persistence	layerMemory	layerKvs	layerSql, layerSqlMultiTable	layerRedis	—
PersistedQueue	layerStoreMemory	—	layerStoreSql	layerStoreRedis	—
RateLimiter	layerStoreMemory	—	—	layerStoreRedis	—

PersistedCache has no backing layer of its own — it is built on top of Persistence, so you provide one of the Persistence layers above.

storeIds are migration boundaries

The Persistence, PersistedCache, and PersistedQueue modules select storage by a storeId string (used as a key prefix, table name, or SQL partition). Changing a storeId, the request’s PrimaryKey format, or the success/error schemas is a data migration: old entries may stop being found or fail to decode. Keep them stable.

The Redis service adapter

The Redis module is small enough to document in full here. It defines the low-level service used by every Redis-backed persistence layer. It deliberately does not create or own connections — you adapt an existing client (such as node-redis or ioredis) by implementing a single send function. There is no built-in client layer; wiring your client up is the integration point.

Redis

A Context.Service exposing send(command, ...args) for ordinary Redis commands and eval(script) for cached Lua scripts. Higher-level layers require this service in their environment.

import { Effect } from "effect"
import { Redis } from "effect/unstable/persistence"

const program = Effect.gen(function* () {
  const redis = yield* Redis
  yield* redis.send("SET", "greeting", "hello")
  const value = yield* redis.send<string>("GET", "greeting")
  // => "hello"
  return value
})

make

Wraps a raw command sender into a Redis service. It builds an internal cache that loads each Lua script once via SCRIPT LOAD, remembers its SHA, and runs it with EVALSHA. This is how you turn a real client into the service — wrap the result in Layer.effect(Redis)(...).

import { Effect, Layer } from "effect"
import { Redis } from "effect/unstable/persistence"

// `client` here is your own connected Redis client (node-redis, ioredis, ...).
declare const client: {
  sendCommand: (args: ReadonlyArray<string>) => Promise<unknown>
}

const RedisLive = Layer.effect(Redis)(
  Redis.make({
    // Map every command + arg list onto your client, and turn any client or
    // network failure into a RedisError.
    send: <A = unknown>(command: string, ...args: ReadonlyArray<string>) =>
      Effect.tryPromise({
        try: () => client.sendCommand([command, ...args]) as Promise<A>,
        catch: (cause) => new Redis.RedisError({ cause })
      })
  })
)
// => Layer<Redis>

RedisError

A Schema.ErrorClass (tag "RedisError") carrying the underlying cause as a Schema.Defect. Raised by command and script execution; your send implementation should map client failures into it.

import { Redis } from "effect/unstable/persistence"

const error = new Redis.RedisError({ cause: new Error("ECONNREFUSED") })
error._tag // => "RedisError"

script

Constructs a typed Script descriptor. f maps your typed parameters to the ordered argument list passed to Lua, and numberOfKeys (a number or a function of the params) tells Redis how many leading arguments are KEYS versus ARGV. The result type defaults to void.

import { Redis } from "effect/unstable/persistence"

// INCRBY a counter key by an amount. The key is a Redis KEY; the amount is ARGV.
const incrBy = Redis.script(
  (key: string, amount: number) => [key, amount],
  {
    lua: `return redis.call("INCRBY", KEYS[1], ARGV[1])`,
    numberOfKeys: 1
  }
)
// => Script<{ params: [string, number]; result: void }>

Script / withReturnType

The Script<Config> interface holds the Lua source, the param-to-arg mapping, and the key count. Because Lua return types are opaque to TypeScript, refine the result type with .withReturnType<R>(), then run it via redis.eval(script).

import { Effect } from "effect"
import { Redis } from "effect/unstable/persistence"

const incrBy = Redis.script(
  (key: string, amount: number) => [key, amount],
  {
    lua: `return redis.call("INCRBY", KEYS[1], ARGV[1])`,
    numberOfKeys: 1
  }
).withReturnType<number>()
// => Script<{ params: [string, number]; result: number }>

const program = Effect.gen(function* () {
  const redis = yield* Redis
  const total = yield* redis.eval(incrBy)("visits", 5)
  // => number, e.g. 5
  return total
})

One Redis layer powers everything

The Persistence.layerRedis, PersistedQueue.layerStoreRedis, and RateLimiter.layerStoreRedis layers all depend on this same Redis service. Provide one Redis layer (built with make) and every Redis-backed persistence feature can share the same connection.