Transports and serialization

Unstable module

RPC lives under effect/unstable/rpc. Unstable modules are production usable but their API may change between minor releases without a major version bump. Import from effect/unstable/rpc/RpcSerialization, .../RpcServer, and .../RpcClient (or the convenience barrel effect/unstable/rpc).

RPC groups and the client/server runtimes are transport-agnostic. To actually move bytes you pick two things and provide them on both ends:

1. A serialization format (RpcSerialization) — how protocol messages sit on the wire, and whether the format frames each message itself.
2. A transport protocol (RpcServer.Protocol on the server, RpcClient.Protocol on the client) — who owns the byte boundary: an HTTP body, a WebSocket frame, a socket stream, stdin/stdout, or a worker message channel.

Mental model. RPC schemas encode payloads, exits, stream chunks, and defects into transport-safe values. Serialization chooses how those values are represented on the wire (JSON, NDJSON, JSON-RPC, MessagePack) and whether message boundaries are part of that representation. The Protocol service owns the actual byte boundary and the receive loop.

Both ends must agree

The two ends of a connection must use compatible serialization and framing. If the transport already frames each payload (an ordinary HTTP request/response body), use a non-framing format (json / jsonRpc). If the transport delivers a raw byte stream that can split or coalesce messages (sockets, workers, stdio, streaming HTTP), use a self-framing format (ndjson / ndJsonRpc / msgPack). Mixing them silently corrupts the stream.

Common case: HTTP + NDJSON

The everyday setup is HTTP transport with NDJSON framing. RpcServer.layerHttp mounts the group on an HTTP router path, and the matching client points RpcClient.layerProtocolHttp at the URL. The same RpcSerialization layer goes on both ends.

// ---- server ----
import { Layer } from "effect"
import { HttpRouter } from "effect/unstable/http"
import { RpcSerialization, RpcServer } from "effect/unstable/rpc"
import { UserRpcs, UserRpcsLayer } from "./handlers.ts"

const RpcRoute = RpcServer.layerHttp({
  group: UserRpcs,
  path: "/rpc",
  protocol: "http" // request/response; omit for the websocket default
})

export const ServerLayer = HttpRouter.serve(RpcRoute).pipe(
  Layer.provide(UserRpcsLayer),
  Layer.provide(RpcSerialization.layerNdjson) // <- shared wire format
  // ...plus an HttpServer layer
)

// ---- client ----
import { Context, Layer } from "effect"
import { FetchHttpClient } from "effect/unstable/http"
import { RpcClient, RpcSerialization } from "effect/unstable/rpc"
import { UserRpcs } from "./handlers.ts"

export class UserClient extends Context.Service<UserClient, RpcClient.FromGroup<typeof UserRpcs>>()(
  "app/UserClient"
) {
  static readonly layer = Layer.effect(UserClient)(RpcClient.make(UserRpcs))
}

export const ClientLayer = UserClient.layer.pipe(
  Layer.provide(RpcClient.layerProtocolHttp({ url: "http://localhost:3000/rpc" })),
  Layer.provide(RpcSerialization.layerNdjson), // <- same wire format
  Layer.provide(FetchHttpClient.layer)
)

Everything below is the exhaustive reference: every serialization format, every server protocol, and every client protocol, each with when-to-use notes and a short example.

Serialization

RpcSerialization is the boundary between encoded protocol messages and the bytes or strings a transport carries. It is a service with three members:

import { RpcSerialization } from "effect/unstable/rpc"

// class RpcSerialization extends Context.Service<RpcSerialization, {
//   makeUnsafe(): Parser        // create a (possibly stateful) parser
//   readonly contentType: string      // e.g. "application/ndjson"
//   readonly includesFraming: boolean // does the format delimit messages itself?
// }>()("effect/rpc/RpcSerialization") {}

A Parser does the actual byte work, in both directions:

// interface Parser {
//   readonly decode: (data: Uint8Array | string) => ReadonlyArray<unknown>
//   readonly encode: (response: unknown) => Uint8Array | string | undefined
// }

Parsers are stateful — one per stream

makeUnsafe() may return a stateful parser. NDJSON keeps a line buffer and MessagePack keeps partial-frame bytes, so a single decode call can return zero, one, or several messages as data arrives. Create a parser for the lifetime of the stream, connection, or request body you are decoding — never share one across connections. The built-in protocols do this for you.

The values exported by the module fall into two shapes: the bare RpcSerialization["Service"] value (e.g. json, ndjson, makeMsgPack(...)) which you can pass to RpcSerialization.of yourself, and the ready-made layer* values that you actually Layer.provide.

layerJson / json

Whole-message JSON with no framing. Use when the transport already frames each payload — ordinary HTTP request and response bodies. Each decode call expects one complete payload.

import { RpcSerialization } from "effect/unstable/rpc"

const layer = RpcSerialization.layerJson
// json.contentType        // => "application/json"
// json.includesFraming    // => false

const parser = RpcSerialization.json.makeUnsafe()
parser.encode({ _tag: "Ping" }) // => '{"_tag":"Ping"}'
parser.decode('{"_tag":"Pong"}') // => [{ _tag: "Pong" }]

layerNdjson / ndjson

Newline-delimited JSON — JSON values separated by \n, so the format frames each message itself. Use for sockets, workers, stdio, and streaming HTTP, where a chunk may contain a partial line or several lines. The parser buffers across calls.

import { RpcSerialization } from "effect/unstable/rpc"

const layer = RpcSerialization.layerNdjson
// ndjson.contentType     // => "application/ndjson"
// ndjson.includesFraming // => true

const p = RpcSerialization.ndjson.makeUnsafe()
p.encode({ _tag: "Ping" }) // => '{"_tag":"Ping"}\n'

// A chunk that splits mid-line: the first decode buffers, the second completes it.
p.decode('{"_tag":"Po') // => []  (incomplete line held in the buffer)
p.decode('ng"}\n') // => [{ _tag: "Pong" }]

layerJsonRpc / jsonRpc

JSON-RPC 2.0 encoding with no framing; each RPC request becomes a { jsonrpc: "2.0", method, params, id } envelope and batches map to JSON-RPC arrays. Use when interoperating with JSON-RPC tooling over a body-framed transport (HTTP). jsonRpc is a function so you can override the contentType.

import { RpcSerialization } from "effect/unstable/rpc"

const layer = RpcSerialization.layerJsonRpc()
const layerCustom = RpcSerialization.layerJsonRpc({ contentType: "application/jsonrequest" })

const p = RpcSerialization.jsonRpc().makeUnsafe()
// default contentType // => "application/json", includesFraming => false
p.encode({ _tag: "Request", id: "1", tag: "GetUser", payload: { id: "1" }, headers: [] })
// => '{"jsonrpc":"2.0","method":"GetUser","params":{"id":"1"},"id":1,...}'

layerNdJsonRpc / ndJsonRpc

Newline-delimited JSON-RPC 2.0 — the JSON-RPC envelopes above, but self-framed with trailing newlines (it wraps an internal ndjson parser). Use for streaming or socket transports that need JSON-RPC interop. Default content type is application/json-rpc.

import { RpcSerialization } from "effect/unstable/rpc"

const layer = RpcSerialization.layerNdJsonRpc()
// ndJsonRpc().contentType     // => "application/json-rpc"
// ndJsonRpc().includesFraming // => true

const p = RpcSerialization.ndJsonRpc().makeUnsafe()
p.encode({ _tag: "Request", id: "1", tag: "Ping", payload: null, headers: [] })
// => '{"jsonrpc":"2.0","method":"Ping",...}\n'

layerMsgPack / msgPack / makeMsgPack

Compact binary MessagePack encoding (via msgpackr), self-framed and able to carry binary data natively without a schema encoding. Use for sockets and workers when payload size or binary fields matter. The parser keeps partial-frame bytes across decode calls.

import { RpcSerialization } from "effect/unstable/rpc"

// Ready-made layer (uses { useRecords: true }):
const layer = RpcSerialization.layerMsgPack
// msgPack.contentType     // => "application/msgpack"
// msgPack.includesFraming // => true

// Custom msgpackr options:
const custom = RpcSerialization.makeMsgPack({ useRecords: false })
const layerCustom = Layer.succeed(RpcSerialization.RpcSerialization)(custom)

const p = RpcSerialization.msgPack.makeUnsafe()
const bytes = p.encode({ _tag: "Ping" }) // => Uint8Array (packed)
p.decode(bytes as Uint8Array) // => [{ _tag: "Ping" }]

Picking a format

• HTTP request/response body → layerJson (or layerJsonRpc for JSON-RPC interop).
• Sockets / workers / stdio / streaming HTTP → layerNdjson (debuggable text) or layerMsgPack (compact, binary-friendly).
• JSON is human-inspectable but needs schema encodings for arbitrary binary values; MessagePack is smaller and carries binary data more naturally.

Server protocols

On the server the transport boundary is the RpcServer.Protocol service. It owns how encoded client messages arrive, how encoded responses are written, and which capabilities the channel has. RpcServer.make / RpcServer.layer (see Client and server) combine that Protocol with the handler context.

The Protocol service exposes three capability flags that the server runtime reads — they change behaviour transparently:

• supportsAck — the channel can deliver client acknowledgements between stream chunks, enabling back-pressure on streaming RPCs. HTTP request/response does not support it; sockets, stdio, and workers do.
• supportsTransferables — the channel can move Transferable objects (e.g. ArrayBuffers) without copying. Only the worker protocol sets this.
• supportsSpanPropagation — the channel preserves tracing span context across the boundary. HTTP does not; sockets, stdio, and workers do.

HTTP (POST)

RpcServer.layerHttp with protocol: "http" mounts a single POST route. Lower level, makeProtocolHttp / layerProtocolHttp build the Protocol and register the route on the current HttpRouter.

import { HttpRouter } from "effect/unstable/http"
import { RpcServer } from "effect/unstable/rpc"

// One POST route on the current HttpRouter:
const ProtocolLayer = RpcServer.layerProtocolHttp({ path: "/rpc" })
// requires RpcSerialization + HttpRouter; supportsAck/Transferables/Spans = false

WebSocket

The default for layerHttp. makeProtocolWebsocket / layerProtocolWebsocket register a GET upgrade route. A live channel means streaming back-pressure and span propagation work.

import { RpcServer } from "effect/unstable/rpc"

const ProtocolLayer = RpcServer.layerProtocolWebsocket({ path: "/rpc" })
// requires RpcSerialization + HttpRouter; use a framed format (layerNdjson/...)

Socket server

makeProtocolSocketServer / layerProtocolSocketServer accept raw socket connections from a SocketServer (TCP, etc.).

import { RpcServer } from "effect/unstable/rpc"

const ProtocolLayer = RpcServer.layerProtocolSocketServer
// requires RpcSerialization + SocketServer; supportsAck + spans = true

layerHttp

The easy path. Mounts the group on an HttpRouter path and provides the matching HTTP or WebSocket protocol (websocket by default, protocol: "http" for request/response). Returns a server Layer, so it bundles RpcServer.layer and the protocol together.

import { RpcServer } from "effect/unstable/rpc"
import { UserRpcs } from "./handlers.ts"

const RpcRoute = RpcServer.layerHttp({
  group: UserRpcs,
  path: "/rpc",
  protocol: "websocket", // default; "http" for POST request/response
  concurrency: "unbounded"
})
// requires RpcSerialization + HttpRouter + the handler/middleware/services context

makeProtocolHttp / layerProtocolHttp

Lower-level HTTP request/response protocol: registers a POST route on the current HttpRouter and returns the Protocol. Pair with a non-framing format for a buffered body, or a framing format to stream the response.

import { Effect } from "effect"
import { HttpRouter } from "effect/unstable/http"
import { RpcServer } from "effect/unstable/rpc"

const program = Effect.gen(function*() {
  const protocol = yield* RpcServer.makeProtocolHttp({ path: "/rpc" })
  // protocol.supportsAck // => false
})

// Or as a layer:
const ProtocolLayer = RpcServer.layerProtocolHttp({ path: "/rpc" })

makeProtocolWebsocket / layerProtocolWebsocket

WebSocket protocol: registers a GET upgrade route on the current HttpRouter. The persistent connection enables streaming acknowledgements and span propagation. Use a self-framing serialization.

import { RpcServer } from "effect/unstable/rpc"

const ProtocolLayer = RpcServer.layerProtocolWebsocket({ path: "/rpc" })
// requires RpcSerialization + HttpRouter

makeProtocolSocketServer / layerProtocolSocketServer

Protocol backed by the current SocketServer — accept TCP (or other socket) connections and route decoded RPC messages. Each connection is a distinct client id.

import { RpcServer } from "effect/unstable/rpc"

const ProtocolLayer = RpcServer.layerProtocolSocketServer
// requires RpcSerialization + SocketServer; supportsAck + supportsSpanPropagation = true

makeProtocolStdio / layerProtocolStdio

Protocol that reads RPC messages from Stdio.stdin and writes responses to Stdio.stdout. Ideal for child-process RPC: spawn a process and talk to it over its standard streams. See child processes for spawning. There is a single client (id 0).

import { RpcServer } from "effect/unstable/rpc"

const ProtocolLayer = RpcServer.layerProtocolStdio
// requires RpcSerialization + Stdio; supportsAck + supportsSpanPropagation = true
// use a framed format (layerNdjson / layerMsgPack) since stdin is a raw stream

makeProtocolWorkerRunner / layerProtocolWorkerRunner

Protocol for code running inside a worker: backed by the current WorkerRunnerPlatform, it routes worker messages to the RPC server and responses back. This is the worker-side counterpart to the client layerProtocolWorker. See workers and concurrency.

import { RpcServer } from "effect/unstable/rpc"

const ProtocolLayer = RpcServer.layerProtocolWorkerRunner
// requires WorkerRunnerPlatform; supportsAck + transferables + spans = true
// note: no RpcSerialization needed — worker messages are structured-cloned

makeProtocolWithHttpEffect / makeProtocolWithHttpEffectWebsocket

For mounting RPC on a hand-rolled HTTP app instead of an HttpRouter. Each returns { protocol, httpEffect }: the Protocol to provide to RpcServer.make, plus an Effect<HttpServerResponse, never, Scope | HttpServerRequest> you mount wherever you like. ...Websocket upgrades the request; the non-suffixed one handles request/response.

import { Effect } from "effect"
import { RpcServer } from "effect/unstable/rpc"

const program = Effect.gen(function*() {
  const { protocol, httpEffect } = yield* RpcServer.makeProtocolWithHttpEffect
  // provide `protocol` to RpcServer.make, mount `httpEffect` at any route
})

toHttpEffect / toHttpEffectWebsocket

Convenience: start an RPC server for a group and hand back just the httpEffect (server already forked into the scope). Use to expose an RPC group as a standalone HttpApp without wiring the Protocol yourself.

import { Effect } from "effect"
import { RpcServer } from "effect/unstable/rpc"
import { UserRpcs } from "./handlers.ts"

const program = Effect.gen(function*() {
  // request/response HTTP app:
  const app = yield* RpcServer.toHttpEffect(UserRpcs)
  // websocket app:
  const wsApp = yield* RpcServer.toHttpEffectWebsocket(UserRpcs)
  // requires Scope + RpcSerialization + handler/middleware/services context
})

Already-decoded channels

RpcServer.makeNoSerialization builds a server for a channel where another layer already owns serialization (you supply an onFromServer callback that receives decoded FromServer messages). This is what RpcTest and in-process transports build on; you rarely call it directly.

Client protocols

On the client the transport is the RpcClient.Protocol service:

import { RpcClient } from "effect/unstable/rpc"

// class Protocol extends Context.Service<Protocol, {
//   readonly run: (clientId, f) => Effect<never>   // receive loop
//   readonly send: (clientId, request, transferables?) => Effect<void, RpcClientError>
//   readonly supportsAck: boolean
//   readonly supportsTransferables: boolean
// }>()("effect/rpc/RpcClient/Protocol") {}

RpcClient.make(group) (Client and server) reads this service. HTTP sends one request per call and cannot acknowledge stream chunks; socket and worker protocols keep a live receive loop, so they support acknowledgements, interruption, and protocol-level failures.

makeProtocolHttp / layerProtocolHttp

HTTP transport: each call is one POST against the configured URL, decoded with the current RpcSerialization. layerProtocolHttp is the common entry point; makeProtocolHttp takes an explicit HttpClient. transformClient lets you add auth headers, retries, base URLs, etc. — see HTTP client.

import { RpcClient } from "effect/unstable/rpc"
import { HttpClient, HttpClientRequest } from "effect/unstable/http"

const ProtocolLayer = RpcClient.layerProtocolHttp({
  url: "http://localhost:3000/rpc",
  transformClient: (client) =>
    HttpClient.mapRequest(client, HttpClientRequest.setHeader("authorization", "Bearer ..."))
})
// requires RpcSerialization + HttpClient; supportsAck = false

makeProtocolSocket / layerProtocolSocket

Socket transport over the current Socket: a persistent connection with a ping/pong keep-alive and an automatic reconnect retry policy. retryTransientErrors suppresses errors from transient socket-open failures while retrying; makeProtocolSocket additionally accepts a custom retryPolicy Schedule. Supports acknowledgements (streaming back-pressure).

import { Schedule } from "effect"
import { RpcClient } from "effect/unstable/rpc"

const ProtocolLayer = RpcClient.layerProtocolSocket({ retryTransientErrors: true })

// Lower-level, with a custom reconnect schedule:
const custom = RpcClient.makeProtocolSocket({
  retryTransientErrors: true,
  retryPolicy: Schedule.exponential(500)
})
// requires Socket + RpcSerialization (and Scope); supportsAck = true

makeProtocolWorker / layerProtocolWorker

Worker transport: routes RPC requests to a pool of workers. The pool options are either a fixed pool ({ size, concurrency?, targetUtilization? }) or a dynamic TTL pool ({ minSize, maxSize, timeToLive, concurrency?, targetUtilization? }). Supports acknowledgements and transferables. See workers and concurrency for pool semantics.

import { RpcClient } from "effect/unstable/rpc"

// Fixed-size pool:
const Fixed = RpcClient.layerProtocolWorker({ size: 4, concurrency: 10 })

// Dynamic pool with TTL:
const Dynamic = RpcClient.layerProtocolWorker({
  minSize: 1,
  maxSize: 8,
  timeToLive: "30 seconds",
  concurrency: 10
})
// requires WorkerPlatform + Spawner; supportsAck + supportsTransferables = true

ConnectionHooks

Optional service letting you run effects when a socket or worker transport opens and closes the underlying channel. The socket protocol runs onConnect on each (re)connect and onDisconnect on close; the worker protocol runs onConnect once the first worker is acquired. Provide it as a layer alongside the protocol.

import { Effect, Layer } from "effect"
import { RpcClient } from "effect/unstable/rpc"

const HooksLayer = Layer.succeed(RpcClient.ConnectionHooks)(
  RpcClient.ConnectionHooks.of({
    onConnect: Effect.log("rpc transport connected"),
    onDisconnect: Effect.log("rpc transport disconnected")
  })
)

const ClientTransport = RpcClient.layerProtocolSocket({ retryTransientErrors: true }).pipe(
  Layer.provide(HooksLayer)
)

In-process clients

RpcClient.makeNoSerialization builds a client over an already-decoded channel (you supply an onFromClient callback and feed server messages back with the returned write). It is the basis for in-memory testing — most apps reach for RpcTest instead.

Setting request headers

Independent of transport, RpcClient.withHeaders merges headers into every outgoing request for the wrapped effect (it updates the CurrentHeaders reference). Per-call headers options are merged on top.

import { Effect } from "effect"
import { RpcClient } from "effect/unstable/rpc"
import { UserClient } from "./client.ts"

const program = Effect.gen(function*() {
  const client = yield* UserClient
  yield* client.GetUser({ id: "1" })
}).pipe(RpcClient.withHeaders({ authorization: "Bearer token" }))

Choosing a pairing

Transport	Server protocol	Client protocol	Serialization	Notes
HTTP request/response	layerProtocolHttp / layerHttp({ protocol: "http" })	layerProtocolHttp	layerJson / layerJsonRpc	Simplest. No streaming back-pressure (no ack).
WebSocket	layerProtocolWebsocket / layerHttp	layerProtocolSocket (+ a WebSocket Socket)	layerNdjson / layerMsgPack	Live channel: ack, interruption, spans.
TCP / raw socket	layerProtocolSocketServer	layerProtocolSocket	layerNdjson / layerMsgPack	Multiple clients server-side.
Child process (stdio)	layerProtocolStdio	layerProtocolSocket over the process socket / stdio	layerNdjson / layerMsgPack	See child processes.
Worker	layerProtocolWorkerRunner (in worker)	layerProtocolWorker (pool)	none (structured clone)	Transferables + ack supported.

The constant across every row: provide a matching serialization layer on both ends (where one is used), and let the framing of the format match whether the transport frames messages for you.