# Transports and serialization

**Unstable module:** RPC lives under `effect/unstable/rpc`. Unstable modules are production usable
  but their API may change between minor releases without a major version bump.
  Import from `effect/unstable/rpc/RpcSerialization`, `.../RpcServer`, and
  `.../RpcClient` (or the convenience barrel `effect/unstable/rpc`).

[RPC groups](https://effect.plants.sh/rpc/defining-rpcs/) and the [client/server runtimes](https://effect.plants.sh/rpc/client-and-server/)
are transport-agnostic. To actually move bytes you pick **two** things and
provide them on both ends:

1. A **serialization format** (`RpcSerialization`) — how protocol messages sit on
   the wire, and whether the format frames each message itself.
2. A **transport protocol** (`RpcServer.Protocol` on the server, `RpcClient.Protocol`
   on the client) — who owns the byte boundary: an HTTP body, a WebSocket frame, a
   socket stream, stdin/stdout, or a worker message channel.

**Mental model.** RPC *schemas* encode payloads, exits, stream chunks, and
defects into transport-safe values. *Serialization* chooses how those values are
represented on the wire (JSON, NDJSON, JSON-RPC, MessagePack) and whether message
boundaries are part of that representation. The *Protocol* service owns the actual
byte boundary and the receive loop.
**Both ends must agree:** The two ends of a connection must use **compatible serialization and framing**.
If the transport already frames each payload (an ordinary HTTP request/response
body), use a non-framing format (`json` / `jsonRpc`). If the transport delivers a
raw byte stream that can split or coalesce messages (sockets, workers, stdio,
streaming HTTP), use a self-framing format (`ndjson` / `ndJsonRpc` / `msgPack`).
Mixing them silently corrupts the stream.

## Common case: HTTP + NDJSON

The everyday setup is HTTP transport with NDJSON framing. `RpcServer.layerHttp`
mounts the group on an [HTTP router](https://effect.plants.sh/http-api/) path, and the matching client
points `RpcClient.layerProtocolHttp` at the URL. The same `RpcSerialization`
layer goes on both ends.

```ts
// ---- server ----
import { Layer } from "effect"
import { HttpRouter } from "effect/unstable/http"
import { RpcSerialization, RpcServer } from "effect/unstable/rpc"
import { UserRpcs, UserRpcsLayer } from "./handlers.ts"

const RpcRoute = RpcServer.layerHttp({
  group: UserRpcs,
  path: "/rpc",
  protocol: "http" // request/response; omit for the websocket default
})

export const ServerLayer = HttpRouter.serve(RpcRoute).pipe(
  Layer.provide(UserRpcsLayer),
  Layer.provide(RpcSerialization.layerNdjson) // <- shared wire format
  // ...plus an HttpServer layer
)
```

```ts
// ---- client ----
import { Context, Layer } from "effect"
import { FetchHttpClient } from "effect/unstable/http"
import { RpcClient, RpcSerialization } from "effect/unstable/rpc"
import { UserRpcs } from "./handlers.ts"

export class UserClient extends Context.Service<UserClient, RpcClient.FromGroup<typeof UserRpcs>>()(
  "app/UserClient"
) {
  static readonly layer = Layer.effect(UserClient)(RpcClient.make(UserRpcs))
}

export const ClientLayer = UserClient.layer.pipe(
  Layer.provide(RpcClient.layerProtocolHttp({ url: "http://localhost:3000/rpc" })),
  Layer.provide(RpcSerialization.layerNdjson), // <- same wire format
  Layer.provide(FetchHttpClient.layer)
)
```

Everything below is the exhaustive reference: every serialization format, every
server protocol, and every client protocol, each with when-to-use notes and a
short example.

## Serialization

`RpcSerialization` is the boundary between encoded protocol messages and the
bytes or strings a transport carries. It is a [service](https://effect.plants.sh/services-and-layers/)
with three members:

```ts
import { RpcSerialization } from "effect/unstable/rpc"

// class RpcSerialization extends Context.Service<RpcSerialization, {
//   makeUnsafe(): Parser        // create a (possibly stateful) parser
//   readonly contentType: string      // e.g. "application/ndjson"
//   readonly includesFraming: boolean // does the format delimit messages itself?
// }>()("effect/rpc/RpcSerialization") {}
```

A `Parser` does the actual byte work, in both directions:

```ts
// interface Parser {
//   readonly decode: (data: Uint8Array | string) => ReadonlyArray<unknown>
//   readonly encode: (response: unknown) => Uint8Array | string | undefined
// }
```
**Parsers are stateful — one per stream:** `makeUnsafe()` may return a **stateful** parser. NDJSON keeps a line buffer and
MessagePack keeps partial-frame bytes, so a single `decode` call can return zero,
one, or several messages as data arrives. Create a parser for the lifetime of the
stream, connection, or request body you are decoding — never share one across
connections. The built-in protocols do this for you.

The values exported by the module fall into two shapes: the bare
`RpcSerialization["Service"]` value (e.g. `json`, `ndjson`, `makeMsgPack(...)`)
which you can pass to `RpcSerialization.of` yourself, and the ready-made
`layer*` values that you actually `Layer.provide`.

### layerJson / json

Whole-message JSON with **no framing**. Use when the transport already frames
each payload — ordinary HTTP request and response bodies. Each `decode` call
expects one complete payload.

```ts
import { RpcSerialization } from "effect/unstable/rpc"

const layer = RpcSerialization.layerJson
// json.contentType        // => "application/json"
// json.includesFraming    // => false

const parser = RpcSerialization.json.makeUnsafe()
parser.encode({ _tag: "Ping" }) // => '{"_tag":"Ping"}'
parser.decode('{"_tag":"Pong"}') // => [{ _tag: "Pong" }]
```

### layerNdjson / ndjson

Newline-delimited JSON — JSON values separated by `\n`, so the format **frames
each message itself**. Use for sockets, workers, stdio, and streaming HTTP, where
a chunk may contain a partial line or several lines. The parser buffers across
calls.

```ts
import { RpcSerialization } from "effect/unstable/rpc"

const layer = RpcSerialization.layerNdjson
// ndjson.contentType     // => "application/ndjson"
// ndjson.includesFraming // => true

const p = RpcSerialization.ndjson.makeUnsafe()
p.encode({ _tag: "Ping" }) // => '{"_tag":"Ping"}\n'

// A chunk that splits mid-line: the first decode buffers, the second completes it.
p.decode('{"_tag":"Po') // => []  (incomplete line held in the buffer)
p.decode('ng"}\n') // => [{ _tag: "Pong" }]
```

### layerJsonRpc / jsonRpc

[JSON-RPC 2.0](https://www.jsonrpc.org/specification) encoding with **no framing**;
each RPC request becomes a `{ jsonrpc: "2.0", method, params, id }` envelope and
batches map to JSON-RPC arrays. Use when interoperating with JSON-RPC tooling over
a body-framed transport (HTTP). `jsonRpc` is a function so you can override the
`contentType`.

```ts
import { RpcSerialization } from "effect/unstable/rpc"

const layer = RpcSerialization.layerJsonRpc()
const layerCustom = RpcSerialization.layerJsonRpc({ contentType: "application/jsonrequest" })

const p = RpcSerialization.jsonRpc().makeUnsafe()
// default contentType // => "application/json", includesFraming => false
p.encode({ _tag: "Request", id: "1", tag: "GetUser", payload: { id: "1" }, headers: [] })
// => '{"jsonrpc":"2.0","method":"GetUser","params":{"id":"1"},"id":1,...}'
```

### layerNdJsonRpc / ndJsonRpc

Newline-delimited JSON-RPC 2.0 — the JSON-RPC envelopes above, but self-framed
with trailing newlines (it wraps an internal `ndjson` parser). Use for streaming
or socket transports that need JSON-RPC interop. Default content type is
`application/json-rpc`.

```ts
import { RpcSerialization } from "effect/unstable/rpc"

const layer = RpcSerialization.layerNdJsonRpc()
// ndJsonRpc().contentType     // => "application/json-rpc"
// ndJsonRpc().includesFraming // => true

const p = RpcSerialization.ndJsonRpc().makeUnsafe()
p.encode({ _tag: "Request", id: "1", tag: "Ping", payload: null, headers: [] })
// => '{"jsonrpc":"2.0","method":"Ping",...}\n'
```

### layerMsgPack / msgPack / makeMsgPack

Compact binary [MessagePack](https://msgpack.org/) encoding (via `msgpackr`),
**self-framed** and able to carry binary data natively without a schema encoding.
Use for sockets and workers when payload size or binary fields matter. The parser
keeps partial-frame bytes across `decode` calls.

```ts
import { RpcSerialization } from "effect/unstable/rpc"

// Ready-made layer (uses { useRecords: true }):
const layer = RpcSerialization.layerMsgPack
// msgPack.contentType     // => "application/msgpack"
// msgPack.includesFraming // => true

// Custom msgpackr options:
const custom = RpcSerialization.makeMsgPack({ useRecords: false })
const layerCustom = Layer.succeed(RpcSerialization.RpcSerialization)(custom)

const p = RpcSerialization.msgPack.makeUnsafe()
const bytes = p.encode({ _tag: "Ping" }) // => Uint8Array (packed)
p.decode(bytes as Uint8Array) // => [{ _tag: "Ping" }]
```
**Picking a format:** - HTTP request/response body → `layerJson` (or `layerJsonRpc` for JSON-RPC interop).
- Sockets / workers / stdio / streaming HTTP → `layerNdjson` (debuggable text) or
  `layerMsgPack` (compact, binary-friendly).
- JSON is human-inspectable but needs schema encodings for arbitrary binary
  values; MessagePack is smaller and carries binary data more naturally.

## Server protocols

On the server the transport boundary is the `RpcServer.Protocol` service. It owns
how encoded client messages arrive, how encoded responses are written, and which
capabilities the channel has. `RpcServer.make` / `RpcServer.layer`
([see Client and server](https://effect.plants.sh/rpc/client-and-server/)) combine that `Protocol` with
the handler context.

The `Protocol` service exposes three capability flags that the server runtime
reads — they change behaviour transparently:

- **`supportsAck`** — the channel can deliver client acknowledgements between
  stream chunks, enabling back-pressure on streaming RPCs. HTTP request/response
  does **not** support it; sockets, stdio, and workers do.
- **`supportsTransferables`** — the channel can move `Transferable` objects
  (e.g. `ArrayBuffer`s) without copying. Only the worker protocol sets this.
- **`supportsSpanPropagation`** — the channel preserves tracing span context
  across the boundary. HTTP does not; sockets, stdio, and workers do.

`RpcServer.layerHttp` with `protocol: "http"` mounts a single POST route. Lower
level, `makeProtocolHttp` / `layerProtocolHttp` build the `Protocol` and register
the route on the current [`HttpRouter`](https://effect.plants.sh/http-api/).

```ts
import { HttpRouter } from "effect/unstable/http"
import { RpcServer } from "effect/unstable/rpc"

// One POST route on the current HttpRouter:
const ProtocolLayer = RpcServer.layerProtocolHttp({ path: "/rpc" })
// requires RpcSerialization + HttpRouter; supportsAck/Transferables/Spans = false
```

The default for `layerHttp`. `makeProtocolWebsocket` / `layerProtocolWebsocket`
register a GET upgrade route. A live channel means streaming back-pressure and
span propagation work.

```ts
import { RpcServer } from "effect/unstable/rpc"

const ProtocolLayer = RpcServer.layerProtocolWebsocket({ path: "/rpc" })
// requires RpcSerialization + HttpRouter; use a framed format (layerNdjson/...)
```

`makeProtocolSocketServer` / `layerProtocolSocketServer` accept raw socket
connections from a [`SocketServer`](https://effect.plants.sh/platform/sockets/) (TCP, etc.).

```ts
import { RpcServer } from "effect/unstable/rpc"

const ProtocolLayer = RpcServer.layerProtocolSocketServer
// requires RpcSerialization + SocketServer; supportsAck + spans = true
```

### layerHttp

The easy path. Mounts the group on an `HttpRouter` path and provides the matching
HTTP **or** WebSocket protocol (websocket by default, `protocol: "http"` for
request/response). Returns a server `Layer`, so it bundles `RpcServer.layer` and
the protocol together.

```ts
import { RpcServer } from "effect/unstable/rpc"
import { UserRpcs } from "./handlers.ts"

const RpcRoute = RpcServer.layerHttp({
  group: UserRpcs,
  path: "/rpc",
  protocol: "websocket", // default; "http" for POST request/response
  concurrency: "unbounded"
})
// requires RpcSerialization + HttpRouter + the handler/middleware/services context
```

### makeProtocolHttp / layerProtocolHttp

Lower-level HTTP request/response protocol: registers a **POST** route on the
current `HttpRouter` and returns the `Protocol`. Pair with a non-framing format
for a buffered body, or a framing format to stream the response.

```ts
import { Effect } from "effect"
import { HttpRouter } from "effect/unstable/http"
import { RpcServer } from "effect/unstable/rpc"

const program = Effect.gen(function*() {
  const protocol = yield* RpcServer.makeProtocolHttp({ path: "/rpc" })
  // protocol.supportsAck // => false
})

// Or as a layer:
const ProtocolLayer = RpcServer.layerProtocolHttp({ path: "/rpc" })
```

### makeProtocolWebsocket / layerProtocolWebsocket

WebSocket protocol: registers a **GET** upgrade route on the current
`HttpRouter`. The persistent connection enables streaming acknowledgements and
span propagation. Use a self-framing serialization.

```ts
import { RpcServer } from "effect/unstable/rpc"

const ProtocolLayer = RpcServer.layerProtocolWebsocket({ path: "/rpc" })
// requires RpcSerialization + HttpRouter
```

### makeProtocolSocketServer / layerProtocolSocketServer

Protocol backed by the current [`SocketServer`](https://effect.plants.sh/platform/sockets/) — accept TCP
(or other socket) connections and route decoded RPC messages. Each connection is
a distinct client id.

```ts
import { RpcServer } from "effect/unstable/rpc"

const ProtocolLayer = RpcServer.layerProtocolSocketServer
// requires RpcSerialization + SocketServer; supportsAck + supportsSpanPropagation = true
```

### makeProtocolStdio / layerProtocolStdio

Protocol that reads RPC messages from `Stdio.stdin` and writes responses to
`Stdio.stdout`. Ideal for **child-process RPC**: spawn a process and talk to it
over its standard streams. See [child processes](https://effect.plants.sh/platform/child-processes/) for
spawning. There is a single client (id `0`).

```ts
import { RpcServer } from "effect/unstable/rpc"

const ProtocolLayer = RpcServer.layerProtocolStdio
// requires RpcSerialization + Stdio; supportsAck + supportsSpanPropagation = true
// use a framed format (layerNdjson / layerMsgPack) since stdin is a raw stream
```

### makeProtocolWorkerRunner / layerProtocolWorkerRunner

Protocol for code running **inside** a worker: backed by the current
`WorkerRunnerPlatform`, it routes worker messages to the RPC server and responses
back. This is the worker-side counterpart to the client `layerProtocolWorker`.
See [workers](https://effect.plants.sh/platform/workers/) and [concurrency](https://effect.plants.sh/concurrency/).

```ts
import { RpcServer } from "effect/unstable/rpc"

const ProtocolLayer = RpcServer.layerProtocolWorkerRunner
// requires WorkerRunnerPlatform; supportsAck + transferables + spans = true
// note: no RpcSerialization needed — worker messages are structured-cloned
```

### makeProtocolWithHttpEffect / makeProtocolWithHttpEffectWebsocket

For mounting RPC on a hand-rolled HTTP app instead of an `HttpRouter`. Each
returns `{ protocol, httpEffect }`: the `Protocol` to provide to `RpcServer.make`,
plus an `Effect<HttpServerResponse, never, Scope | HttpServerRequest>` you mount
wherever you like. `...Websocket` upgrades the request; the non-suffixed one
handles request/response.

```ts
import { Effect } from "effect"
import { RpcServer } from "effect/unstable/rpc"

const program = Effect.gen(function*() {
  const { protocol, httpEffect } = yield* RpcServer.makeProtocolWithHttpEffect
  // provide `protocol` to RpcServer.make, mount `httpEffect` at any route
})
```

### toHttpEffect / toHttpEffectWebsocket

Convenience: start an RPC server for a group and hand back just the `httpEffect`
(server already forked into the scope). Use to expose an RPC group as a standalone
[HttpApp](https://effect.plants.sh/http-server/) without wiring the `Protocol` yourself.

```ts
import { Effect } from "effect"
import { RpcServer } from "effect/unstable/rpc"
import { UserRpcs } from "./handlers.ts"

const program = Effect.gen(function*() {
  // request/response HTTP app:
  const app = yield* RpcServer.toHttpEffect(UserRpcs)
  // websocket app:
  const wsApp = yield* RpcServer.toHttpEffectWebsocket(UserRpcs)
  // requires Scope + RpcSerialization + handler/middleware/services context
})
```
**Already-decoded channels:** `RpcServer.makeNoSerialization` builds a server for a channel where another layer
already owns serialization (you supply an `onFromServer` callback that receives
decoded `FromServer` messages). This is what [`RpcTest`](https://effect.plants.sh/rpc/client-and-server/)
and in-process transports build on; you rarely call it directly.

## Client protocols

On the client the transport is the `RpcClient.Protocol` service:

```ts
import { RpcClient } from "effect/unstable/rpc"

// class Protocol extends Context.Service<Protocol, {
//   readonly run: (clientId, f) => Effect<never>   // receive loop
//   readonly send: (clientId, request, transferables?) => Effect<void, RpcClientError>
//   readonly supportsAck: boolean
//   readonly supportsTransferables: boolean
// }>()("effect/rpc/RpcClient/Protocol") {}
```

`RpcClient.make(group)` ([Client and server](https://effect.plants.sh/rpc/client-and-server/)) reads this
service. HTTP sends one request per call and cannot acknowledge stream chunks;
socket and worker protocols keep a live receive loop, so they support
acknowledgements, interruption, and protocol-level failures.

### makeProtocolHttp / layerProtocolHttp

HTTP transport: each call is one POST against the configured URL, decoded with
the current `RpcSerialization`. `layerProtocolHttp` is the common entry point;
`makeProtocolHttp` takes an explicit `HttpClient`. `transformClient` lets you add
auth headers, retries, base URLs, etc. — see [HTTP client](https://effect.plants.sh/http-client/).

```ts
import { RpcClient } from "effect/unstable/rpc"
import { HttpClient, HttpClientRequest } from "effect/unstable/http"

const ProtocolLayer = RpcClient.layerProtocolHttp({
  url: "http://localhost:3000/rpc",
  transformClient: (client) =>
    HttpClient.mapRequest(client, HttpClientRequest.setHeader("authorization", "Bearer ..."))
})
// requires RpcSerialization + HttpClient; supportsAck = false
```

### makeProtocolSocket / layerProtocolSocket

Socket transport over the current [`Socket`](https://effect.plants.sh/platform/sockets/): a persistent
connection with a ping/pong keep-alive and an automatic reconnect retry policy.
`retryTransientErrors` suppresses errors from transient socket-open failures while
retrying; `makeProtocolSocket` additionally accepts a custom `retryPolicy`
`Schedule`. Supports acknowledgements (streaming back-pressure).

```ts
import { Schedule } from "effect"
import { RpcClient } from "effect/unstable/rpc"

const ProtocolLayer = RpcClient.layerProtocolSocket({ retryTransientErrors: true })

// Lower-level, with a custom reconnect schedule:
const custom = RpcClient.makeProtocolSocket({
  retryTransientErrors: true,
  retryPolicy: Schedule.exponential(500)
})
// requires Socket + RpcSerialization (and Scope); supportsAck = true
```

### makeProtocolWorker / layerProtocolWorker

Worker transport: routes RPC requests to a **pool of workers**. The pool options
are either a fixed pool (`{ size, concurrency?, targetUtilization? }`) or a
dynamic TTL pool (`{ minSize, maxSize, timeToLive, concurrency?, targetUtilization? }`).
Supports acknowledgements and transferables. See [workers](https://effect.plants.sh/platform/workers/)
and [concurrency](https://effect.plants.sh/concurrency/) for pool semantics.

```ts
import { RpcClient } from "effect/unstable/rpc"

// Fixed-size pool:
const Fixed = RpcClient.layerProtocolWorker({ size: 4, concurrency: 10 })

// Dynamic pool with TTL:
const Dynamic = RpcClient.layerProtocolWorker({
  minSize: 1,
  maxSize: 8,
  timeToLive: "30 seconds",
  concurrency: 10
})
// requires WorkerPlatform + Spawner; supportsAck + supportsTransferables = true
```

### ConnectionHooks

Optional service letting you run effects when a socket or worker transport
**opens** and **closes** the underlying channel. The socket protocol runs
`onConnect` on each (re)connect and `onDisconnect` on close; the worker protocol
runs `onConnect` once the first worker is acquired. Provide it as a layer
alongside the protocol.

```ts
import { Effect, Layer } from "effect"
import { RpcClient } from "effect/unstable/rpc"

const HooksLayer = Layer.succeed(RpcClient.ConnectionHooks)(
  RpcClient.ConnectionHooks.of({
    onConnect: Effect.log("rpc transport connected"),
    onDisconnect: Effect.log("rpc transport disconnected")
  })
)

const ClientTransport = RpcClient.layerProtocolSocket({ retryTransientErrors: true }).pipe(
  Layer.provide(HooksLayer)
)
```
**In-process clients:** `RpcClient.makeNoSerialization` builds a client over an already-decoded channel
(you supply an `onFromClient` callback and feed server messages back with the
returned `write`). It is the basis for in-memory testing — most apps reach for
[`RpcTest`](https://effect.plants.sh/rpc/client-and-server/) instead.

### Setting request headers

Independent of transport, `RpcClient.withHeaders` merges headers into every
outgoing request for the wrapped effect (it updates the `CurrentHeaders`
reference). Per-call `headers` options are merged on top.

```ts
import { Effect } from "effect"
import { RpcClient } from "effect/unstable/rpc"
import { UserClient } from "./client.ts"

const program = Effect.gen(function*() {
  const client = yield* UserClient
  yield* client.GetUser({ id: "1" })
}).pipe(RpcClient.withHeaders({ authorization: "Bearer token" }))
```

## Choosing a pairing

| Transport | Server protocol | Client protocol | Serialization | Notes |
| --- | --- | --- | --- | --- |
| HTTP request/response | `layerProtocolHttp` / `layerHttp({ protocol: "http" })` | `layerProtocolHttp` | `layerJson` / `layerJsonRpc` | Simplest. No streaming back-pressure (no ack). |
| WebSocket | `layerProtocolWebsocket` / `layerHttp` | `layerProtocolSocket` (+ a WebSocket `Socket`) | `layerNdjson` / `layerMsgPack` | Live channel: ack, interruption, spans. |
| TCP / raw socket | `layerProtocolSocketServer` | `layerProtocolSocket` | `layerNdjson` / `layerMsgPack` | Multiple clients server-side. |
| Child process (stdio) | `layerProtocolStdio` | `layerProtocolSocket` over the process socket / stdio | `layerNdjson` / `layerMsgPack` | See [child processes](https://effect.plants.sh/platform/child-processes/). |
| Worker | `layerProtocolWorkerRunner` (in worker) | `layerProtocolWorker` (pool) | none (structured clone) | Transferables + ack supported. |

The constant across every row: provide a **matching** serialization layer on both
ends (where one is used), and let the framing of the format match whether the
transport frames messages for you.