Chat

The Chat module turns one-shot generation into a stateful conversation. A chat session keeps the running Prompt history in a Ref; each call combines that history with your new message, invokes the language model, and appends the response back into history. You focus on the current turn — the session manages context for you. This is the foundation for assistants and agentic loops.

Creating a session and chatting

Create a session with Chat.empty, seed one with Chat.fromPrompt (to set a system prompt or prior messages), or restore one from a previous export with Chat.fromJson. A session’s generateText, streamText, and generateObject mirror the LanguageModel API, but read and update history automatically.

import { OpenAiClient, OpenAiLanguageModel } from "@effect/ai-openai"
import { Config, Context, Effect, Layer, Ref, Schema } from "effect"
import { AiError, Chat, Prompt } from "effect/unstable/ai"
import { FetchHttpClient } from "effect/unstable/http"

const OpenAiClientLayer = OpenAiClient.layerConfig({
  apiKey: Config.redacted("OPENAI_API_KEY")
}).pipe(Layer.provide(FetchHttpClient.layer))

class AiAssistantError extends Schema.TaggedErrorClass<AiAssistantError>()("AiAssistantError", {
  reason: AiError.AiErrorReason
}) {
  static fromAiError(error: AiError.AiError) {
    return new AiAssistantError({ reason: error.reason })
  }
}

class AiAssistant extends Context.Service<AiAssistant, {
  chat(message: string): Effect.Effect<string, AiAssistantError>
}>()("acme/AiAssistant") {
  static readonly layer = Layer.effect(
    AiAssistant,
    Effect.gen(function*() {
      // The model to drive every turn. Captured into this Layer's requirements.
      const modelLayer = yield* OpenAiLanguageModel.model("gpt-5.2").captureRequirements

      // Seed a session with a system prompt. `Chat.fromPrompt` accepts a string,
      // a list of messages, or a Prompt built with the Prompt module.
      const session = yield* Chat.fromPrompt(Prompt.empty.pipe(
        Prompt.setSystem("You are a helpful assistant that answers questions.")
      ))

      const chat = Effect.fn("AiAssistant.chat")(
        function*(message: string) {
          // Each call adds a turn: the user message is merged with the running
          // history, sent to the model, and the reply is appended automatically.
          const response = yield* session.generateText({ prompt: message }).pipe(
            // The model is supplied per call, so you can even switch models
            // mid-conversation.
            Effect.provide(modelLayer)
          )

          // Inspect accumulated history at any time via the `history` Ref.
          const history = yield* Ref.get(session.history)
          yield* Effect.logInfo(`Conversation has ${history.content.length} messages`)

          return response.text
        },
        Effect.mapError(AiAssistantError.fromAiError)
      )

      return AiAssistant.of({ chat })
    })
  ).pipe(Layer.provide(OpenAiClientLayer))
}

Each call to session.generateText is a new turn. The session prepends the full history to your prompt, so you only ever pass the new message — the model sees the whole conversation.

Persisting and restoring sessions

A session is just history plus helpers, so it serializes cleanly. Use exportJson to snapshot a conversation and Chat.fromJson to restore it later — handy for resuming a chat across requests, processes, or storage.

import { Chat } from "effect/unstable/ai"
import { Effect } from "effect"

const resume = Effect.gen(function*() {
  const session = yield* Chat.empty
  yield* session.generateText({ prompt: "Remember the number 42." })

  // Snapshot the conversation as a JSON string.
  const json = yield* session.exportJson

  // ...store `json` somewhere, then later rebuild the exact same session.
  const restored = yield* Chat.fromJson(json)
  return restored
})

For a fully managed lifecycle — load by id, save automatically after each turn — use the persistence-backed variants described in Persisted chats below.

Streaming and history

streamText records the assistant’s reply into history only when the stream finalizes. If you want the full response to become part of the conversation, consume the stream to completion. You can write to session.history directly, but that bypasses the encode/decode/export helpers — prefer the generation methods.

Agentic loops with tools

Combine a session with a toolkit to build an agent. Pass the toolkit to generateText; if the model calls tools, the session appends their results to history automatically. Loop until the model returns a final answer with no further tool calls.

import { OpenAiClient, OpenAiLanguageModel } from "@effect/ai-openai"
import { Config, Context, DateTime, Effect, Layer, Schema } from "effect"
import { AiError, Chat, Tool, Toolkit } from "effect/unstable/ai"
import { FetchHttpClient } from "effect/unstable/http"

const OpenAiClientLayer = OpenAiClient.layerConfig({
  apiKey: Config.redacted("OPENAI_API_KEY")
}).pipe(Layer.provide(FetchHttpClient.layer))

// A single tool the agent can call. Note: no Date.now — we use the Clock-backed
// DateTime service inside the handler.
const Tools = Toolkit.make(Tool.make("getCurrentTime", {
  description: "Get the current time in ISO format",
  parameters: Schema.Struct({ id: Schema.String }),
  success: Schema.String
}))

const ToolsLayer = Tools.toLayer(Effect.gen(function*() {
  return Tools.of({
    getCurrentTime: Effect.fn("Tools.getCurrentTime")(function*(_) {
      const now = yield* DateTime.now
      return DateTime.formatIso(now)
    })
  })
}))

class AgentError extends Schema.TaggedErrorClass<AgentError>()("AgentError", {
  reason: AiError.AiErrorReason
}) {}

class Agent extends Context.Service<Agent, {
  ask(question: string): Effect.Effect<string, AgentError>
}>()("acme/Agent") {
  static readonly layer = Layer.effect(
    Agent,
    Effect.gen(function*() {
      const modelLayer = yield* OpenAiLanguageModel.model("gpt-5.2").captureRequirements
      const tools = yield* Tools

      const ask = Effect.fn("Agent.ask")(
        function*(question: string) {
          // Seed a fresh session per question with a system prompt + the question.
          const session = yield* Chat.fromPrompt([
            { role: "system", content: "You can use tools to answer questions." },
            { role: "user", content: question }
          ])

          while (true) {
            const response = yield* session.generateText({
              // Empty prompt: the model already has the full history.
              prompt: [],
              toolkit: tools
            }).pipe(Effect.provide(modelLayer))

            // Tools were called: the session already appended their results to
            // history, so loop again to let the model continue.
            if (response.toolCalls.length > 0) continue

            // No tool calls means the model produced its final answer.
            return response.text
          }
        },
        // Map the AI error to our domain error; die on any unexpected error
        // (e.g. a tool handler failure) so the channel narrows to AgentError.
        Effect.catchTag(
          "AiError",
          (error) => Effect.fail(new AgentError({ reason: error.reason })),
          (error) => Effect.die(error)
        )
      )

      return Agent.of({ ask })
    })
  ).pipe(Layer.provide([OpenAiClientLayer, ToolsLayer]))
}

The loop is the whole agent: generate, and if the response contains tool calls, keep going — the session has already merged the tool results into history — until the model is done and answers in plain text.

The session interface

A Chat session exposes:

• generateText, streamText, generateObject — same options as LanguageModel, but history-aware.
• history — a Ref<Prompt.Prompt> holding the conversation; read it to inspect the messages so far.
• export / exportJson — serialize the conversation for storage or transport.

Constructors: Chat.empty, Chat.fromPrompt, Chat.fromJson, Chat.fromExport, plus persistence-backed variants (makePersisted / layerPersisted) for sessions that save to a backing store after each turn.

History lives in the session, the model does not

The session stores history, but the language model is still supplied through the Effect environment when a generation runs. That is why each generateText call in the examples is provided a model Layer — and why you can use different models for different turns of the same conversation.

API reference

Everything below is exported from effect/unstable/ai’s Chat module. The common case is covered above; this section enumerates every public export.

Chat

The Context.Service tag for a stateful conversation session. Yield it to obtain a Chat.Service. (Chat.empty and Chat.fromPrompt already hand you a service; use the tag itself when you want to provide a session through a Layer.)

import { Chat } from "effect/unstable/ai"
import { Effect } from "effect"

const program = Effect.gen(function*() {
  const session = yield* Chat // resolve the service from context
  return yield* session.generateText({ prompt: "Hi!" })
})
// => Effect requiring Chat | LanguageModel

Service

The interface a session implements. Use it as the contract when writing functions that accept any chat session.

import type { Chat } from "effect/unstable/ai"

declare const session: Chat.Service
// session.history / session.generateText / session.streamText /
// session.generateObject / session.export / session.exportJson

Constructors

Chat.empty

An Effect that yields a brand-new session with empty history. The most common way to start a fresh conversation.

import { Chat } from "effect/unstable/ai"
import { Effect, Ref } from "effect"

const program = Effect.gen(function*() {
  const session = yield* Chat.empty
  const history = yield* Ref.get(session.history)
  return history.content.length
  // => 0
})

Chat.fromPrompt

Creates a session seeded with an initial Prompt. Accepts a Prompt.RawInput: a string, an array of messages, or a Prompt built with the Prompt module. Use it to set a system prompt or replay prior messages.

import { Chat } from "effect/unstable/ai"
import { Effect, Ref } from "effect"

const program = Effect.gen(function*() {
  const session = yield* Chat.fromPrompt([
    { role: "system", content: "You are concise." },
    { role: "user", content: "Hello" }
  ])
  const history = yield* Ref.get(session.history)
  return history.content.map((m) => m.role)
  // => ["system", "user"]
})

Chat.fromExport

Rebuilds a session from data produced by session.export (a structured object, not a string). Fails with Schema.SchemaError if the data is not valid history.

import { Chat } from "effect/unstable/ai"
import { Effect } from "effect"

const program = Effect.gen(function*() {
  const original = yield* Chat.fromPrompt("hi")
  const exported = yield* original.export // structured object
  const restored = yield* Chat.fromExport(exported)
  return restored
}).pipe(
  Effect.catchTag("SchemaError", () => Chat.empty) // fall back on bad data
)

Chat.fromJson

Rebuilds a session from a JSON string produced by session.exportJson. The most convenient round-trip for storage. Fails with Schema.SchemaError on malformed JSON.

import { Chat } from "effect/unstable/ai"
import { Effect } from "effect"

const program = Effect.gen(function*() {
  const original = yield* Chat.fromPrompt("hi")
  const json = yield* original.exportJson // string, e.g. '{"content":[...]}'
  const restored = yield* Chat.fromJson(json)
  return restored
}).pipe(
  Effect.catchTag("SchemaError", () => Chat.empty)
)

Session methods

These live on every Chat.Service instance.

history

A Ref<Prompt.Prompt> holding the full conversation. Read it with Ref.get to inspect messages. Writing to it directly works but bypasses the encode/decode/save helpers, so prefer the generation methods for normal turns.

import { Chat } from "effect/unstable/ai"
import { Effect, Ref } from "effect"

const program = Effect.gen(function*() {
  const session = yield* Chat.fromPrompt("first message")
  const prompt = yield* Ref.get(session.history)
  return prompt.content.length
  // => 1
})

generateText

Generates a text response. Merges the running history with options.prompt, calls the model, then appends both the user message and the reply to history. Pass a toolkit to enable function calling. Requires a LanguageModel in the environment.

import { Chat } from "effect/unstable/ai"
import { Effect } from "effect"

const program = Effect.gen(function*() {
  const session = yield* Chat.empty
  const response = yield* session.generateText({
    prompt: "What is the capital of France?"
  })
  return response.text
  // => "Paris."  (history now has the user turn + this reply)
})
// => Effect requiring LanguageModel

streamText

Like generateText, but returns a Stream of response parts as the model produces them. The assistant reply is written into history only once the stream finalizes, so consume it to completion.

import { Chat } from "effect/unstable/ai"
import { Effect, Stream } from "effect"

const program = Effect.gen(function*() {
  const session = yield* Chat.empty
  const stream = session.streamText({ prompt: "Write a haiku." })
  yield* Stream.runForEach(stream, (part) =>
    part.type === "text-delta"
      ? Effect.sync(() => process.stdout.write(part.delta))
      : Effect.void)
  // history is updated after the stream completes
})

generateObject

Generates a structured result that conforms to a Schema, while still tracking the exchange in history. Returns a response whose object is typed by the schema.

import { Chat } from "effect/unstable/ai"
import { Effect, Schema } from "effect"

const Contact = Schema.Struct({
  name: Schema.String,
  email: Schema.String
})

const program = Effect.gen(function*() {
  const session = yield* Chat.empty
  const response = yield* session.generateObject({
    prompt: "Extract: John Doe, john@example.com",
    schema: Contact
  })
  return response.object
  // => { name: "John Doe", email: "john@example.com" }
})

export

Serializes the conversation into a structured (non-string) value, ready to feed back into Chat.fromExport. Fails with AiError if encoding fails.

import { Chat } from "effect/unstable/ai"
import { Effect } from "effect"

const program = Effect.gen(function*() {
  const session = yield* Chat.fromPrompt("hello")
  const data = yield* session.export
  return data
  // => { content: [ { role: "user", content: [...] } ], ... }
})

exportJson

Serializes the conversation to a JSON string, ready for Chat.fromJson. Fails with AiError if encoding fails.

import { Chat } from "effect/unstable/ai"
import { Effect } from "effect"

const program = Effect.gen(function*() {
  const session = yield* Chat.fromPrompt("hello")
  const json = yield* session.exportJson
  return typeof json
  // => "string"
})

Persisted chats

For long-lived assistants, let Effect manage the load/save lifecycle. A persisted session is a Chat.Service plus an id and a save effect; it writes its history to a backing store after every generation, so you never serialize by hand. Provide the Chat.Persistence service (which itself sits on top of a BackingPersistence) and look chats up by id.

import { OpenAiClient, OpenAiLanguageModel } from "@effect/ai-openai"
import { Config, Effect, Layer } from "effect"
import { Chat } from "effect/unstable/ai"
import { FetchHttpClient } from "effect/unstable/http"
import { Persistence } from "effect/unstable/persistence"

const OpenAiClientLayer = OpenAiClient.layerConfig({
  apiKey: Config.redacted("OPENAI_API_KEY")
}).pipe(Layer.provide(FetchHttpClient.layer))

// Wire a chat-persistence layer on top of a backing store. Here the store is
// in-memory; swap `layerBackingMemory` for the SQL / Redis / KVS backing layers
// in production.
const ChatPersistenceLayer = Chat.layerPersisted({ storeId: "assistant-chats" }).pipe(
  Layer.provide(Persistence.layerBackingMemory)
)

const program = Effect.gen(function*() {
  const persistence = yield* Chat.Persistence
  const modelLayer = yield* OpenAiLanguageModel.model("gpt-5.2").captureRequirements

  // Load the chat for this user, creating + saving an empty one if it is new.
  const session = yield* persistence.getOrCreate("user-123")

  // Generation auto-saves the updated history to the backing store.
  const response = yield* session.generateText({
    prompt: "Pick up where we left off."
  }).pipe(Effect.provide(modelLayer))

  return { id: session.id, text: response.text }
}).pipe(Effect.provide([ChatPersistenceLayer, OpenAiClientLayer]))

Chat.Persistence

The Context.Service tag for chat persistence. Its service exposes get (fail with ChatNotFoundError when missing) and getOrCreate (create + save an empty session when missing). Both accept an optional timeToLive.

import { Chat } from "effect/unstable/ai"
import { Effect } from "effect"

const program = Effect.gen(function*() {
  const persistence = yield* Chat.Persistence
  // Throws ChatNotFoundError if "abc" was never stored:
  const existing = yield* persistence.get("abc")
  // Always succeeds — creates an empty, saved chat if needed:
  const ensured = yield* persistence.getOrCreate("xyz", {
    timeToLive: "1 hour"
  })
  return [existing.id, ensured.id]
})

Chat.Persistence (namespace)

The Persistence.Service interface, describing the get / getOrCreate contract a backing implementation must satisfy. Reference it when typing code that consumes the service.

import type { Chat } from "effect/unstable/ai"

declare const persistence: Chat.Persistence.Service
// persistence.get(chatId, options?) / persistence.getOrCreate(chatId, options?)

Persisted

A Chat.Service extended with persistence affordances: a stable id and a save effect that flushes current history to the backing store. Returned by persistence.get / getOrCreate.

import { Chat } from "effect/unstable/ai"
import { Effect } from "effect"

const program = Effect.gen(function*() {
  const persistence = yield* Chat.Persistence
  const session = yield* persistence.getOrCreate("user-123")
  // Force a manual save (generation already saves automatically):
  yield* session.save
  return session.id
  // => "user-123"
})

makePersisted

Builds the Chat.Persistence service directly as an Effect, given a storeId. Requires a BackingPersistence in the environment. Use this when you want the service value rather than a Layer.

import { Chat } from "effect/unstable/ai"
import { Effect } from "effect"
import { Persistence } from "effect/unstable/persistence"

const program = Effect.gen(function*() {
  const persistence = yield* Chat.makePersisted({ storeId: "chats" })
  return yield* persistence.getOrCreate("user-1")
}).pipe(Effect.provide(Persistence.layerBackingMemory))

layerPersisted

The Layer form of makePersisted: provides Chat.Persistence from a BackingPersistence. The idiomatic way to wire persistence into an application.

import { Chat } from "effect/unstable/ai"
import { Layer } from "effect"
import { Persistence } from "effect/unstable/persistence"

const ChatPersistenceLayer = Chat.layerPersisted({ storeId: "chats" }).pipe(
  Layer.provide(Persistence.layerBackingMemory)
)
// => Layer<Chat.Persistence>

Errors

ChatNotFoundError

A tagged error raised by Chat.Persistence.get when no stored chat matches the requested id. Carries the chatId. Catch it with Effect.catchTag.

import { Chat } from "effect/unstable/ai"
import { Effect } from "effect"

const program = Effect.gen(function*() {
  const persistence = yield* Chat.Persistence
  return yield* persistence.get("missing")
}).pipe(
  Effect.catchTag("ChatNotFoundError", (error) =>
    Effect.succeed(`No chat with id ${error.chatId}`))
  // => "No chat with id missing"
)

Related

• Language Model — the underlying generation API.
• Tools — define the tools your agent calls.
• Persistence — the backing store behind persisted chats.
• State Management — Ref, the primitive behind session history.