Chat

The Chat module turns one-shot generation into a stateful conversation. A chat session keeps the running Prompt history in a Ref; each call combines that history with your new message, invokes the language model, and appends the response back into history. You focus on the current turn — the session manages context for you. This is the foundation for assistants and agentic loops.

Creating a session and chatting

Create a session with Chat.empty, seed one with Chat.fromPrompt (to set a system prompt or prior messages), or restore one from a previous export with Chat.fromJson. A session’s generateText, streamText, and generateObject mirror the LanguageModel API, but read and update history automatically.

import { OpenAiClient, OpenAiLanguageModel } from "@effect/ai-openai"
import { Config, Context, Effect, Layer, Ref, Schema } from "effect"
import { AiError, Chat, Prompt } from "effect/unstable/ai"
import { FetchHttpClient } from "effect/unstable/http"

const OpenAiClientLayer = OpenAiClient.layerConfig({
  apiKey: Config.redacted("OPENAI_API_KEY")
}).pipe(Layer.provide(FetchHttpClient.layer))

class AiAssistantError extends Schema.TaggedErrorClass<AiAssistantError>()("AiAssistantError", {
  reason: AiError.AiErrorReason
}) {
  static fromAiError(error: AiError.AiError) {
    return new AiAssistantError({ reason: error.reason })
  }
}

class AiAssistant extends Context.Service<AiAssistant, {
  chat(message: string): Effect.Effect<string, AiAssistantError>
}>()("acme/AiAssistant") {
  static readonly layer = Layer.effect(
    AiAssistant,
    Effect.gen(function*() {
      // The model to drive every turn. Captured into this Layer's requirements.
      const modelLayer = yield* OpenAiLanguageModel.model("gpt-5.2").captureRequirements

      // Seed a session with a system prompt. `Chat.fromPrompt` accepts a string,
      // a list of messages, or a Prompt built with the Prompt module.
      const session = yield* Chat.fromPrompt(Prompt.empty.pipe(
        Prompt.setSystem("You are a helpful assistant that answers questions.")
      ))

      const chat = Effect.fn("AiAssistant.chat")(
        function*(message: string) {
          // Each call adds a turn: the user message is merged with the running
          // history, sent to the model, and the reply is appended automatically.
          const response = yield* session.generateText({ prompt: message }).pipe(
            // The model is supplied per call, so you can even switch models
            // mid-conversation.
            Effect.provide(modelLayer)
          )

          // Inspect accumulated history at any time via the `history` Ref.
          const history = yield* Ref.get(session.history)
          yield* Effect.logInfo(`Conversation has ${history.content.length} messages`)

          return response.text
        },
        Effect.mapError(AiAssistantError.fromAiError)
      )

      return AiAssistant.of({ chat })
    })
  ).pipe(Layer.provide(OpenAiClientLayer))
}

Each call to session.generateText is a new turn. The session prepends the full history to your prompt, so you only ever pass the new message — the model sees the whole conversation.

Persisting and restoring sessions

A session is just history plus helpers, so it serializes cleanly. Use exportJson to snapshot a conversation and Chat.fromJson to restore it later — handy for resuming a chat across requests, processes, or storage.

import { Chat } from "effect/unstable/ai"
import { Effect } from "effect"

const resume = Effect.gen(function*() {
  const session = yield* Chat.empty
  yield* session.generateText({ prompt: "Remember the number 42." })

  // Snapshot the conversation as a JSON string.
  const json = yield* session.exportJson

  // ...store `json` somewhere, then later rebuild the exact same session.
  const restored = yield* Chat.fromJson(json)
  return restored
})

Streaming and history

streamText records the assistant’s reply into history only when the stream finalizes. If you want the full response to become part of the conversation, consume the stream to completion. You can write to session.history directly, but that bypasses the encode/decode/export helpers — prefer the generation methods.

Agentic loops with tools

Combine a session with a toolkit to build an agent. Pass the toolkit to generateText; if the model calls tools, the session appends their results to history automatically. Loop until the model returns a final answer with no further tool calls.

import { OpenAiClient, OpenAiLanguageModel } from "@effect/ai-openai"
import { Config, Context, DateTime, Effect, Layer, Schema } from "effect"
import { AiError, Chat, Tool, Toolkit } from "effect/unstable/ai"
import { FetchHttpClient } from "effect/unstable/http"

const OpenAiClientLayer = OpenAiClient.layerConfig({
  apiKey: Config.redacted("OPENAI_API_KEY")
}).pipe(Layer.provide(FetchHttpClient.layer))

// A single tool the agent can call. Note: no Date.now — we use the Clock-backed
// DateTime service inside the handler.
const Tools = Toolkit.make(Tool.make("getCurrentTime", {
  description: "Get the current time in ISO format",
  parameters: Schema.Struct({ id: Schema.String }),
  success: Schema.String
}))

const ToolsLayer = Tools.toLayer(Effect.gen(function*() {
  return Tools.of({
    getCurrentTime: Effect.fn("Tools.getCurrentTime")(function*(_) {
      const now = yield* DateTime.now
      return DateTime.formatIso(now)
    })
  })
}))

class AgentError extends Schema.TaggedErrorClass<AgentError>()("AgentError", {
  reason: AiError.AiErrorReason
}) {}

class Agent extends Context.Service<Agent, {
  ask(question: string): Effect.Effect<string, AgentError>
}>()("acme/Agent") {
  static readonly layer = Layer.effect(
    Agent,
    Effect.gen(function*() {
      const modelLayer = yield* OpenAiLanguageModel.model("gpt-5.2").captureRequirements
      const tools = yield* Tools

      const ask = Effect.fn("Agent.ask")(
        function*(question: string) {
          // Seed a fresh session per question with a system prompt + the question.
          const session = yield* Chat.fromPrompt([
            { role: "system", content: "You can use tools to answer questions." },
            { role: "user", content: question }
          ])

          while (true) {
            const response = yield* session.generateText({
              // Empty prompt: the model already has the full history.
              prompt: [],
              toolkit: tools
            }).pipe(Effect.provide(modelLayer))

            // Tools were called: the session already appended their results to
            // history, so loop again to let the model continue.
            if (response.toolCalls.length > 0) continue

            // No tool calls means the model produced its final answer.
            return response.text
          }
        },
        // Map the AI error (the only thing in the error channel) to our domain error.
        Effect.catchTag(
          "AiError",
          (error) => Effect.fail(new AgentError({ reason: error.reason }))
        )
      )

      return Agent.of({ ask })
    })
  ).pipe(Layer.provide([OpenAiClientLayer, ToolsLayer]))
}

The loop is the whole agent: generate, and if the response contains tool calls, keep going — the session has already merged the tool results into history — until the model is done and answers in plain text.

The session interface

A Chat session exposes:

• generateText, streamText, generateObject — same options as LanguageModel, but history-aware.
• history — a Ref<Prompt.Prompt> holding the conversation; read it to inspect the messages so far.
• exportJson — serialize the conversation to a JSON string.

Constructors: Chat.empty, Chat.fromPrompt, Chat.fromJson, Chat.fromExport, plus persistence-backed variants (makePersisted / layerPersisted) for sessions that save to a backing store after each turn.

History lives in the session, the model does not

The session stores history, but the language model is still supplied through the Effect environment when a generation runs. That is why each generateText call in the examples is provided a model Layer — and why you can use different models for different turns of the same conversation.

Related

• Language Model — the underlying generation API.
• Tools — define the tools your agent calls.
• State Management — Ref, the primitive behind session history.