Skip to content

Models, repositories and migrations

A Model is a Schema that describes one row of a SQL table while generating the variants you actually need at runtime — a select variant for reading, an insert variant that omits database-generated columns, and an update variant for partial writes (plus JSON variants for an HTTP API). SqlModel.makeRepository turns a model into a ready-made repository with insert, update, findById, and delete, so the common CRUD operations are type-safe and decoded without hand-writing each query.

import { Schema } from "effect"
import { Model } from "effect/unstable/schema"
// Branded id keeps Person ids from being confused with other numeric ids.
export const PersonId = Schema.Number.pipe(Schema.brand("PersonId"))
export class Person extends Model.Class<Person>("Person")({
// FieldExcept(["insert"]): the column is present in the select and update
// variants but omitted from insert, because the database assigns the id.
id: PersonId.pipe(Model.FieldExcept(["insert"])),
name: Schema.String,
email: Schema.String,
// Timestamp fields filled with the current time on insert / update.
createdAt: Model.DateTimeInsertFromDate,
updatedAt: Model.DateTimeUpdateFromDate
}) {}
// `Person` — the select schema (full row)
// `Person.insert` — input for INSERT (no id, no timestamps to supply)
// `Person.update` — input for UPDATE
// `Person.json` — variant for serializing over an API

The field helpers encode intent: Model.FieldExcept(["insert"]) keeps a column in the select and update variants while omitting it from insert (ideal for a database-assigned, but still updatable, id), Model.DateTimeInsertFromDate / Model.DateTimeUpdateFromDate populate timestamps on write, and Model.Sensitive hides values from JSON output. Model.GeneratedByDb exists for read-only generated columns — it contributes only select and json variants, so it is not suitable for an id you also pass to update. Each variant is a real Schema, so models compose with SqlSchema, validation, and the rest of the ecosystem.

A model is built from six variants. The three database variants (VariantsDatabase = "select" | "insert" | "update") describe the shapes you encode to and decode from SQL, and the three JSON variants (VariantsJson = "json" | "jsonCreate" | "jsonUpdate") describe the shapes you expose over an HTTP API. The default variant — used when you reference the model directly — is select.

import { Schema } from "effect"
import { Model } from "effect/unstable/schema"
export const GroupId = Schema.Number.pipe(Schema.brand("GroupId"))
export class Group extends Model.Class<Group>("Group")({
id: Model.GeneratedByDb(GroupId),
name: Schema.String,
createdAt: Model.DateTimeInsertFromDate,
updatedAt: Model.DateTimeUpdateFromDate
}) {}
Group // select — the full row (default variant)
Group.insert // insert — no id (GeneratedByDb), no timestamps to supply
Group.update // update — no createdAt (insert-only)
Group.json // json — read shape for an API
Group.jsonCreate // jsonCreate — create payload
Group.jsonUpdate // jsonUpdate — update payload

Every variant is itself a Schema, so you can decode/encode with it directly or promote it into a Schema.Class to attach derived data:

import { Schema } from "effect"
// Turn the json variant into a class with extra computed members.
class GroupJson extends Schema.Class<GroupJson>("GroupJson")(Group.json) {
get upperName() {
return this.name.toUpperCase()
}
}

How a field maps onto variants is decided by the helper you use. A plain schema (e.g. name: Schema.String) is included in all variants. A Field-based helper opts the property into only the variants it declares — so Model.GeneratedByDb (select + json) disappears from insert and update, and Model.Sensitive (database variants only) disappears from every JSON variant.

SqlModel.makeRepository reads the SqlClient from context and returns the CRUD operations bound to a table. Wrap it in a service so the rest of your app depends on the repository, not on raw SQL.

import { Context, Effect, Layer } from "effect"
import { SqlClient, SqlModel } from "effect/unstable/sql"
import { Person, PersonId } from "./Person.js"
// makeRepository returns an Effect whose success value is the record of CRUD
// operations and whose only requirement is SqlClient. `Effect.Success` reads
// that shape back out so the service interface stays in sync.
const makePeopleRepository = SqlModel.makeRepository(Person, {
tableName: "people",
spanPrefix: "People", // names the tracing spans for each operation
idColumn: "id"
})
export class People extends Context.Service<People, Effect.Success<typeof makePeopleRepository>>()(
"app/People"
) {
// Layer.effect runs makePeopleRepository, so this layer still needs a
// SqlClient layer to be provided alongside it.
static readonly layer: Layer.Layer<People, never, SqlClient.SqlClient> =
Layer.effect(People, makePeopleRepository)
}
const program = Effect.gen(function*() {
const people = yield* People
// insert encodes Person.insert, runs INSERT … RETURNING *, and decodes the
// row back into a full `Person`.
const created = yield* people.insert(
Person.insert.make({ name: "Ada", email: "ada@example.com" })
)
// findById fails with NoSuchElementError when the row is absent.
const found = yield* people.findById(PersonId.make(created.id))
yield* people.update(
Person.update.make({ id: found.id, name: "Ada Lovelace", email: found.email })
)
})

makeRepository(Model, { tableName, spanPrefix, idColumn, softDeleteColumn? }) returns a record of six operations. Returned rows are decoded with the full model schema; insert/update inputs are encoded with the model’s dedicated input schemas. Each operation is wrapped in a tracing span prefixed with spanPrefix.

Encodes Model.insert, runs INSERT … RETURNING * (or an insert + follow-up select on MySQL), and decodes the new row into a full model value.

const created = yield* repo.insert(
Person.insert.make({ name: "Ada", email: "ada@example.com" })
)
// => Person (with the DB-assigned id and timestamps)

Like insert but discards the result — no RETURNING, no decode. Use it when you do not need the inserted row back.

yield* repo.insertVoid(Person.insert.make({ name: "Ada", email: "ada@example.com" }))
// => void

Encodes Model.update, runs UPDATE … WHERE id = ? RETURNING *, and decodes the updated row. The id is read from the idColumn field of the update payload.

const updated = yield* repo.update(
Person.update.make({ id, name: "Ada Lovelace", email })
)
// => Person (the updated row)

Like update but discards the returned row.

yield* repo.updateVoid(Person.update.make({ id, name: "Ada L.", email }))
// => void

Selects the row whose idColumn equals the supplied id and decodes it. Fails with NoSuchElementError when no row matches.

const person = yield* repo.findById(PersonId.make(1))
// => Person | fails NoSuchElementError if absent

Removes the row by id. With a softDeleteColumn set, it instead sets that column to CURRENT_TIMESTAMP rather than deleting.

yield* repo.delete(PersonId.make(1))
// => void

Pass softDeleteColumn to mark rows as deleted instead of removing them. Reads and updates then only see rows where that column IS NULL, and delete sets it to CURRENT_TIMESTAMP.

const makeRepo = SqlModel.makeRepository(Person, {
tableName: "people",
spanPrefix: "People",
idColumn: "id",
softDeleteColumn: "deletedAt" // delete sets deletedAt = CURRENT_TIMESTAMP
})
// findById / update / delete all add ...AND deletedAt IS NULL

When the same lookups happen across concurrent fibers (e.g. resolving fields in a GraphQL/HTTP API), SqlModel.makeResolvers returns the operations as RequestResolvers that coalesce concurrent requests into a single batched query. It requires SqlClient | Scope.

import { Effect } from "effect"
import { SqlModel, SqlResolver } from "effect/unstable/sql"
import { Person, PersonId } from "./Person.js"
const program = Effect.gen(function*() {
// makeResolvers returns { insert, insertVoid, findById, delete }
const resolvers = yield* SqlModel.makeResolvers(Person, {
tableName: "people",
spanPrefix: "People",
idColumn: "id"
})
// `SqlResolver.request` wraps the payload in a `SqlRequest` and runs it
// through the resolver. Two concurrent findById requests coalesce into one
// `select * from people where id in (1, 2)`.
const [a, b] = yield* Effect.all(
[
SqlResolver.request(PersonId.make(1), resolvers.findById),
SqlResolver.request(PersonId.make(2), resolvers.findById)
],
{ batching: true }
)
})

The resolver record contains insert (batched insert returning rows), insertVoid (batched insert, no rows), findById (batched WHERE id IN (...), failing NoSuchElementError per missing id), and delete (batched delete, or soft delete when softDeleteColumn is set). For the lower-level query and schema-binding primitives these resolvers build on, see Queries; for the request/resolver model itself, see Batching.

A migrator runs numbered migration effects exactly once. It ensures a migrations table exists (default effect_sql_migrations), reads the highest applied id, and runs every pending migration in order inside a transaction, recording each id so it never runs twice. Each migration is an Effect that uses the ambient SqlClient.

  1. Write migration files named <id>_<name>.ts, each default-exporting an Effect that creates or alters tables. (.js, .mjs, and .mts extensions are also recognized.)

    migrations/0001_create_people.ts
    import { Effect } from "effect"
    import { SqlClient } from "effect/unstable/sql"
    // The default export is the migration Effect. It runs with the SqlClient
    // already in context, inside the migrator's transaction.
    export default Effect.gen(function*() {
    const sql = yield* SqlClient.SqlClient
    yield* sql`
    CREATE TABLE people (
    id SERIAL PRIMARY KEY,
    name TEXT NOT NULL,
    email TEXT NOT NULL UNIQUE,
    created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
    updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
    )
    `
    })
  2. Provide a migrator layer. The driver’s Migrator re-exports the loaders and adds a layer that runs pending migrations during layer construction.

    import { Config, Layer } from "effect"
    import { PgClient, PgMigrator } from "@effect/sql-pg"
    import { NodeServices } from "@effect/platform-node"
    const PgLive = PgClient.layerConfig({
    url: Config.redacted("DATABASE_URL")
    })
    // `fromFileSystem` discovers <id>_<name>.ts files at runtime. Other loaders:
    // `fromGlob` (bundler import map) and `fromRecord` (in-memory, great for tests).
    const MigratorLive = PgMigrator.layer({
    loader: PgMigrator.fromFileSystem("migrations"),
    // Optional: dump the schema after successful migrations.
    schemaDirectory: "migrations"
    }).pipe(
    Layer.provide(PgLive),
    // fromFileSystem + schema dump need FileSystem, Path, and a process spawner,
    // which NodeServices.layer provides together.
    Layer.provide(NodeServices.layer)
    )

Because MigratorLive runs migrations as a side effect of building its layer, provide it once at application startup (typically merged with your other layers) and pending migrations apply before the rest of the program starts. Migration ids must be unique; only ids greater than the latest recorded id are considered pending, so editing an already-applied migration will not re-run it — add a new one instead.

The core lives in effect/unstable/sql’s Migrator module; each driver package (@effect/sql-pg, @effect/sql-sqlite-node, …) wires it to a dialect and adds a layer. The migrator returns the list of applied [id, name] pairs.

The constructor factory. make({ dumpSchema? }) returns a function taking MigratorOptions{ loader, schemaDirectory?, table? } — and yields an Effect that applies pending migrations. table defaults to effect_sql_migrations. Drivers call this for you; use it directly only when building a custom migrator.

import { Migrator } from "effect/unstable/sql"
// A migrator with no schema-dump hook.
const migrate = Migrator.make({})({
loader: Migrator.fromRecord({}),
table: "my_migrations"
})
// => Effect<ReadonlyArray<[id, name]>, MigrationError | SqlError, SqlClient>

Loader that reads a directory via FileSystem, importing files matching <id>_<name>.(js|ts|mjs|mts), sorted by id. Returns Loader<FileSystem>.

import { Migrator } from "effect/unstable/sql"
const loader = Migrator.fromFileSystem("migrations")
// => Loader<FileSystem> (needs a FileSystem layer, e.g. NodeServices.layer)

Loader built from a bundler glob record of dynamic import functions (keys are file paths), parsed and sorted by id. No FileSystem requirement — the bundler inlines the imports.

import { Migrator } from "effect/unstable/sql"
const loader = Migrator.fromGlob(
import.meta.glob("./migrations/*.ts") as Record<string, () => Promise<any>>
)
// => Loader

Loader for Babel-style glob records, where keys look like _0001_initTs. The values are the already-imported modules (not import functions).

import { Migrator } from "effect/unstable/sql"
const loader = Migrator.fromBabelGlob(babelGeneratedMigrations)
// => Loader

In-memory loader: a record keyed by <id>_<name> whose values are migration Effects. Ideal for tests — no filesystem, no bundler.

import { Effect } from "effect"
import { Migrator, SqlClient } from "effect/unstable/sql"
const loader = Migrator.fromRecord({
"0001_init": Effect.gen(function*() {
const sql = yield* SqlClient.SqlClient
yield* sql`CREATE TABLE people (id INTEGER PRIMARY KEY, name TEXT)`
})
})
// => Loader (pair with an in-memory SQLite layer for fast isolated runs)
  • MigratorOptions<R>{ loader: Loader<R>; schemaDirectory?: string; table?: string }.
  • Loader<R>Effect<ReadonlyArray<ResolvedMigration>, MigrationError, R>.
  • ResolvedMigration — the tuple [id: number, name: string, load: Effect<…, …, SqlClient>].
  • Migration — recorded metadata { id: number; name: string; createdAt: Date }.

A Data.TaggedError("MigrationError") raised while loading, validating, locking, or running migrations. Its kind is one of:

// "BadState" — migrations table in an inconsistent state
// "ImportError" — a file had no default export, or it was not an Effect
// "Failed" — a migration Effect threw / a directory could not be read
// "Duplicates" — two migrations share the same id
// "Locked" — another runner holds the migration lock

On PostgreSQL the migrations table is explicitly LOCKed (ACCESS EXCLUSIVE) during a run, so concurrent deployers coordinate safely and a blocked runner logs Migrations already running rather than failing. Other dialects rely on the migrations table’s primary-key constraint: a duplicate insert from a second runner surfaces as a MigrationError with kind: "Locked".

For the query-building primitives these models and migrations are built on, see Queries; for the section overview, see SQL.

Every helper below is exported from effect/unstable/schema’s Model module. Each one declares which variants a property contributes to.

Builds a variant field from a record of per-variant schemas. This is the primitive every other helper is built on — supply only the variants you want the property to appear in.

import { Schema } from "effect"
import { Model } from "effect/unstable/schema"
// Present in select + json, absent from insert/update/jsonCreate/jsonUpdate.
const id = Model.Field({
select: Schema.Number,
json: Schema.Number
})
// => a VariantSchema.Field usable as a model field

Applies a single schema to every variant except the listed ones. Ideal for a database-assigned id that you still update by.

import { Schema } from "effect"
import { Model } from "effect/unstable/schema"
// Present everywhere except insert (the DB assigns it on insert).
const id = Schema.Number.pipe(Model.FieldExcept(["insert"]))
// => select/update/json/jsonCreate/jsonUpdate have id; insert does not

The inverse of FieldExcept: applies a schema only to the listed variants.

import { Schema } from "effect"
import { Model } from "effect/unstable/schema"
// Only appears in the json read variant.
const etag = Schema.String.pipe(Model.FieldOnly(["json"]))
// => json has etag; all other variants omit it

Makes a field optional in all variants. Database variants accept null and decode to Option; JSON variants additionally allow the key to be missing.

import { Schema } from "effect"
import { Model } from "effect/unstable/schema"
const bio = Model.FieldOption(Schema.String)
// db variants: Option.none() <- null; Option.some("...") <- "..."
// json variants: also tolerate a missing key, decoding to Option.none()

Transforms the schemas inside an existing field (or plain schema) per variant — the building block FieldOption itself uses.

import { Schema } from "effect"
import { Model } from "effect/unstable/schema"
// Make only the insert variant of a field nullable.
const evolve = Model.fieldEvolve({
insert: (s: Schema.String) => Schema.NullOr(s)
})
// => a function that rewrites the named variants of a field

Returns the raw per-variant field definitions stored on a model or variant struct — useful for reflection or for re-composing fields.

const defs = Model.fields(Group)
// => { id, name, createdAt, updatedAt } field configs

Pulls a single generated variant schema out of a model or variant struct as a standalone Schema.Struct.

// The insert struct schema, equivalent to Group.insert.
const insertSchema = Model.extract(Group, "insert")
// => Schema.Struct of the insert variant

Builds a variant struct from field definitions without the Class machinery — when you want the variants but not a class.

import { Schema } from "effect"
import { Model } from "effect/unstable/schema"
const Point = Model.Struct({
x: Schema.Number,
y: Schema.Number
})
Point.insert // => Schema.Struct

Creates a union over the default and generated variants of several variant structs.

const Shape = Model.Union(Point, Group)
// => a variant union; Shape.insert is a union of each member's insert variant

Marks a value as an explicit override for a field that otherwise carries an overrideable default (e.g. the timestamp helpers). Use it inside a constructor call to supply the value yourself instead of letting the default fire.

import { Model } from "effect/unstable/schema"
// Bypass the "now" default and pin an exact timestamp.
Group.insert.make({
name: "Engineering",
createdAt: Model.Override(new Date("2024-01-01T00:00:00Z")) as any
})

A column the database generates (serial id, default). Contributes to select and json only, so it is absent from insert and update.

import { Schema } from "effect"
import { Model } from "effect/unstable/schema"
const id = Model.GeneratedByDb(Schema.Number)
// select: id present | insert: absent | update: absent | json: present

A value the application generates before insert (e.g. a client-side UUID). Contributes to select, insert, update, and json, but is omitted from the JSON create/update payloads.

import { Schema } from "effect"
import { Model } from "effect/unstable/schema"
const id = Model.GeneratedByApp(Schema.String)
// select/insert/update/json: present | jsonCreate/jsonUpdate: absent

A value present in the database variants but hidden from all JSON variants — for password hashes, tokens, etc.

import { Schema } from "effect"
import { Model } from "effect/unstable/schema"
const passwordHash = Model.Sensitive(Schema.String)
// select/insert/update: present | json/jsonCreate/jsonUpdate: absent

A standalone schema (not a field) for an optional key whose encoded value may be missing or null and whose decoded value is an Option. FieldOption uses this for its JSON variants.

import { Schema } from "effect"
import { Model } from "effect/unstable/schema"
const nickname = Model.optionalOption(Schema.String)
// decode: missing/null -> Option.none(); "x" -> Option.some("x")
// encode: Option.none() -> omitted/null

These helpers store DateTime.Utc values and differ in their encoded representation (string / JS Date / milliseconds) and in whether they auto-fill on insert, on update, or not at all.

A DateTime.Utc date-only value encoded as a YYYY-MM-DD string. The time component is removed on decode.

import { Model } from "effect/unstable/schema"
const birthday = Model.Date
// encode: DateTime.Utc -> "2024-05-30" | decode: "2024-05-30" -> DateTime.Utc

Like Model.Date, but overrideable with a constructor default of today (time removed).

import { Model } from "effect/unstable/schema"
const day = Model.DateWithNow
// constructor default => DateTime.removeTime(DateTime.now)

An overrideable DateTime.Utc encoded as an ISO string, defaulting to the current time.

const ts = Model.DateTimeWithNow
// default => DateTime.now ; encoded as "2024-05-30T12:00:00.000Z"

As above, but encoded as a JavaScript Date (for drivers that pass Date objects), defaulting to the current time.

const ts = Model.DateTimeFromDateWithNow
// encoded representation: JS Date ; default => DateTime.now

As above, but encoded as a number of milliseconds since the epoch, defaulting to the current time.

const ts = Model.DateTimeFromNumberWithNow
// encoded representation: number (ms) ; default => DateTime.now

Model.DateTimeInsert / …FromDate / …FromNumber

Section titled “Model.DateTimeInsert / …FromDate / …FromNumber”

Auto-timestamp insert fields: defaulted to the current time on insert, selectable, and omitted from updates. The three variants differ only in the database encoding (string / Date / number); JSON encodes as a string for the first two and a number for FromNumber.

import { Model } from "effect/unstable/schema"
Model.DateTimeInsert // db: string | insert default = now | no update
Model.DateTimeInsertFromDate // db: JS Date | insert default = now | no update
Model.DateTimeInsertFromNumber // db: number | insert default = now | no update

Model.DateTimeUpdate / …FromDate / …FromNumber

Section titled “Model.DateTimeUpdate / …FromDate / …FromNumber”

Auto-timestamp update fields: defaulted to the current time on both inserts and updates, and selectable. Same encoding split as the insert helpers.

import { Model } from "effect/unstable/schema"
Model.DateTimeUpdate // db: string | default = now on insert + update
Model.DateTimeUpdateFromDate // db: JS Date | default = now on insert + update
Model.DateTimeUpdateFromNumber // db: number | default = now on insert + update

Stores a structured value as text in the database (parsed/serialized via JSON) while exposing the object schema directly in the JSON variants.

import { Schema } from "effect"
import { Model } from "effect/unstable/schema"
const Prefs = Schema.Struct({ theme: Schema.String, beta: Schema.Boolean })
const preferences = Model.JsonFromString(Prefs)
// db variants: '{"theme":"dark","beta":true}' <-> { theme, beta }
// json variants: { theme, beta } used directly (no string wrapping)

A boolean stored as SQLite’s 0 | 1 bit in the database variants but exposed as a real boolean in the JSON variants.

import { Model } from "effect/unstable/schema"
const active = Model.BooleanSqlite
// db: 1 <-> true, 0 <-> false | json: true / false directly

A schema for binary Uint8Array values backed by an ArrayBuffer — the building block for the binary UUID helpers below.

import { Model } from "effect/unstable/schema"
const bytes = Model.Uint8Array
// => Schema.instanceOf<Uint8Array<ArrayBuffer>>

These build insert fields that generate a UUID by default while keeping the column selectable and updatable. Pass a branded schema so ids stay distinct.

Model.UuidV4Insert / Model.UuidV4WithGenerate

Section titled “Model.UuidV4Insert / Model.UuidV4WithGenerate”

A branded string UUID v4. UuidV4Insert is the field (default-generated on insert); UuidV4WithGenerate adds the constructor default to an existing branded string schema.

import { Schema } from "effect"
import { Model } from "effect/unstable/schema"
const UserId = Schema.String.pipe(Schema.brand("UserId"))
const id = Model.UuidV4Insert(UserId)
// select/update/json: branded string | insert: auto-generated UUID v4

Model.UuidV7Insert / Model.UuidV7WithGenerate

Section titled “Model.UuidV7Insert / Model.UuidV7WithGenerate”

A branded string UUID v7 (time-ordered, sortable) generated from the current Clock on insert.

import { Schema } from "effect"
import { Model } from "effect/unstable/schema"
const EventId = Schema.String.pipe(Schema.brand("EventId"))
const id = Model.UuidV7Insert(EventId)
// insert: auto-generated UUID v7 (uses Clock for the timestamp)

Model.UuidV4BytesInsert / Model.UuidV4BytesWithGenerate

Section titled “Model.UuidV4BytesInsert / Model.UuidV4BytesWithGenerate”

A branded binary (Uint8Array) UUID v4 — compact 16-byte storage. UuidV4BytesInsert is the field; UuidV4BytesWithGenerate adds the generating default to a branded Uint8Array schema.

import { Schema } from "effect"
import { Model } from "effect/unstable/schema"
const KeyId = Model.Uint8Array.pipe(Schema.brand("KeyId"))
const id = Model.UuidV4BytesInsert(KeyId)
// insert: auto-generated 16-byte UUID v4 | stored as binary