Language Model

LanguageModel is the central abstraction of Effect’s AI modules. It exposes three operations — generateText, generateObject, and streamText — that all work against whichever provider you supply through a Layer. Your code depends on the abstract LanguageModel.LanguageModel service; the concrete model is injected at the edges.

import { Effect } from "effect"
import { LanguageModel } from "effect/unstable/ai"

const program = Effect.gen(function*() {
  const response = yield* LanguageModel.generateText({
    prompt: "Explain quantum computing in one sentence"
  })

  console.log(response.text) // => the generated text
  return response
})

Setting up a provider

A provider is wired up in two pieces: a client Layer that holds credentials and an HttpClient, and a model Layer that selects a specific model. Both come from a provider package such as @effect/ai-openai.

import { OpenAiClient } from "@effect/ai-openai"
import { Config, Layer } from "effect"
import { FetchHttpClient } from "effect/unstable/http"

// The client Layer holds your API key (read from Config, so it never gets
// hard-coded) and depends on an HttpClient implementation. Providers let you
// choose which HttpClient to use — here the platform-agnostic Fetch client.
const OpenAiClientLayer = OpenAiClient.layerConfig({
  apiKey: Config.redacted("OPENAI_API_KEY")
}).pipe(Layer.provide(FetchHttpClient.layer))

OpenAiLanguageModel.model("gpt-5.2") returns a Model.Model Layer that provides a concrete LanguageModel (plus the Model.ProviderName and Model.ModelName metadata services) and requires the OpenAiClient. You provide that model Layer to the effect that runs the generation.

A complete service

Here is the idiomatic shape: a domain service that wraps the model behind your own API and your own error type. It generates text, decodes a structured object, and streams a response — all three operations in one place.

import { AnthropicClient, AnthropicLanguageModel } from "@effect/ai-anthropic"
import { OpenAiClient, OpenAiLanguageModel } from "@effect/ai-openai"
import { Config, Context, Effect, ExecutionPlan, Layer, Schema, Stream } from "effect"
import { AiError, LanguageModel, Model, type Response } from "effect/unstable/ai"
import { FetchHttpClient } from "effect/unstable/http"

// The schema we'll ask the model to fill in. Generated objects are decoded and
// validated through this schema, so a successful response is fully typed.
class LaunchPlan extends Schema.Class<LaunchPlan>("LaunchPlan")({
  audience: Schema.Literals(["developers", "operators", "platform teams"]),
  channels: Schema.Array(Schema.String),
  launchDate: Schema.String,
  summary: Schema.String,
  keyRisks: Schema.Array(Schema.String)
}) {}

// Client Layers for both providers.
const OpenAiClientLayer = OpenAiClient.layerConfig({
  apiKey: Config.redacted("OPENAI_API_KEY")
}).pipe(Layer.provide(FetchHttpClient.layer))

const AnthropicClientLayer = AnthropicClient.layerConfig({
  apiKey: Config.redacted("ANTHROPIC_API_KEY")
}).pipe(Layer.provide(FetchHttpClient.layer))

// A domain error. `AiError.AiErrorReason` is itself a Schema, so we can embed
// it directly in a tagged error of our own.
class AiWriterError extends Schema.TaggedErrorClass<AiWriterError>()("AiWriterError", {
  reason: AiError.AiErrorReason
}) {
  static fromAiError(error: AiError.AiError) {
    return new AiWriterError({ reason: error.reason })
  }
}

// An ExecutionPlan tries a cheaper OpenAI model first (up to 3 attempts), then
// falls back to an Anthropic model (up to 2 attempts) if the first keeps failing.
const DraftPlan = ExecutionPlan.make(
  { provide: OpenAiLanguageModel.model("gpt-5.2"), attempts: 3 },
  { provide: AnthropicLanguageModel.model("claude-opus-4-6"), attempts: 2 }
)

class AiWriter extends Context.Service<AiWriter, {
  draftAnnouncement(product: string): Effect.Effect<string, AiWriterError>
  extractLaunchPlan(notes: string): Effect.Effect<LaunchPlan, AiWriterError>
  streamReleaseHighlights(version: string): Stream.Stream<string, AiWriterError>
}>()("docs/AiWriter") {
  static readonly layer = Layer.effect(
    AiWriter,
    Effect.gen(function*() {
      // `captureRequirements` moves the plan's requirements (the AI clients)
      // into this Layer's requirements, so they get provided at the bottom.
      const draftsModel = yield* DraftPlan.captureRequirements
      // A single model for the other operations.
      const model = yield* OpenAiLanguageModel.model("gpt-4.1").captureRequirements

      // --- generateText: plain text plus rich metadata --------------------
      const draftAnnouncement = Effect.fn("AiWriter.draftAnnouncement")(
        function*(product: string) {
          const provider = yield* Model.ProviderName
          const response = yield* LanguageModel.generateText({
            prompt: `Write a short launch announcement for ${product}.`
          })

          // The response carries convenience accessors so you don't have to walk
          // the raw content parts: `text`, `finishReason`, `usage`, `toolCalls`...
          yield* Effect.logInfo(
            `${provider} finished: ${response.finishReason}, ` +
              `outputTokens=${response.usage.outputTokens.total}`
          )

          return response.text
        },
        // Apply the ExecutionPlan so failures fall back across providers.
        Effect.withExecutionPlan(draftsModel),
        Effect.mapError(AiWriterError.fromAiError)
      )

      // --- generateObject: schema-validated structured output -------------
      const extractLaunchPlan = Effect.fn("AiWriter.extractLaunchPlan")(
        function*(notes: string) {
          const response = yield* LanguageModel.generateObject({
            // `objectName` names the structure for the provider.
            objectName: "launch_plan",
            prompt: `Convert these notes into a launch plan:\n${notes}`,
            // The model output is decoded through this schema before you see it.
            schema: LaunchPlan
          })

          // `response.value` is a fully decoded, typed `LaunchPlan`.
          return response.value
        },
        Effect.provide(model),
        Effect.mapError(AiWriterError.fromAiError)
      )

      // --- streamText: incremental output ---------------------------------
      const streamReleaseHighlights = (version: string) =>
        LanguageModel.streamText({
          prompt: `Write release highlights for version ${version} as bullets.`
        }).pipe(
          // streamText emits typed parts; keep only the incremental text deltas.
          Stream.filter((part): part is Response.TextDeltaPart => part.type === "text-delta"),
          Stream.map((part) => part.delta),
          Stream.provide(model),
          Stream.mapError(AiWriterError.fromAiError)
        )

      return AiWriter.of({ draftAnnouncement, extractLaunchPlan, streamReleaseHighlights })
    })
  ).pipe(
    // This Layer needs both clients, since the plan spans both providers.
    Layer.provide([OpenAiClientLayer, AnthropicClientLayer])
  )
}

Selecting and providing a model

Every generation needs a concrete model in its environment. Provider packages give you a model(...) helper returning a Model.Model Layer:

Effect.provide(model) / Stream.provide(model) — run a single effect or stream with that model.
model.captureRequirements — within a Layer.effect, capture the model’s requirements (the client) into the surrounding Layer, as shown above.
Effect.withExecutionPlan(plan) — apply an ExecutionPlan built from several models for retries and cross-provider fallback.

Model.ProviderName and Model.ModelName are services the model Layer also provides, so any code running under a model can read which provider and model answered.

import { Effect } from "effect"
import { LanguageModel, Model } from "effect/unstable/ai"

const reportProvider = Effect.gen(function*() {
  const provider = yield* Model.ProviderName // => e.g. "openai"
  const modelName = yield* Model.ModelName // => e.g. "gpt-4.1"
  const response = yield* LanguageModel.generateText({ prompt: "Hi" })
  return `${provider}/${modelName}: ${response.text}`
})

Handling errors

All operations fail with AiError.AiError, a wrapper carrying module, method, and a reason describing what went wrong (rate limits, auth failures, malformed output, …). Because AiErrorReason is a Schema, you can embed it directly in your own tagged error, as AiWriterError does. Map at the boundary with Effect.mapError, or branch on the reason with Effect.catchTag.

import { Effect } from "effect"
import { AiError, LanguageModel } from "effect/unstable/ai"

const safe = LanguageModel.generateText({ prompt: "Hello" }).pipe(
  // Every AiError is tagged "AiError", so catchTag narrows the whole channel.
  Effect.catchTag("AiError", (error: AiError.AiError) =>
    Effect.succeed(`request failed: ${error.reason._tag}`)
  )
)

When a toolkit is attached, the error channel also widens to include the tools’ handler errors — see Tools.

API reference

The remainder of this page enumerates every public export of the LanguageModel module, plus the relevant Model and Response types it returns.

`LanguageModel.LanguageModel`

The Context.Service tag for the abstract language model. Yield it to get the Service interface (generateText, generateObject, streamText) when you want to make several calls against the same instance.

import { Effect } from "effect"
import { LanguageModel } from "effect/unstable/ai"

const program = Effect.gen(function*() {
  const model = yield* LanguageModel.LanguageModel
  const a = yield* model.generateText({ prompt: "Name a color" })
  const b = yield* model.generateText({ prompt: "Name a fruit" })
  return [a.text, b.text] // => e.g. ["Blue", "Mango"]
})

`LanguageModel.Service`

The interface implemented by every provider. It declares the three methods (generateText, generateObject, streamText) with the same signatures as the free functions below, minus the LanguageModel requirement (you already hold the instance). You rarely reference this type by name — it is what make returns.

`LanguageModel.generateText`

Runs a single, non-streaming generation and resolves to a GenerateTextResponse. Requires a LanguageModel in the environment.

import { Effect } from "effect"
import { LanguageModel } from "effect/unstable/ai"

const program = Effect.gen(function*() {
  const response = yield* LanguageModel.generateText({
    prompt: "Write a haiku about programming",
    toolChoice: "none"
  })

  console.log(response.text) // => the haiku
  console.log(response.usage.inputTokens.total) // => e.g. 12
  return response
})

`LanguageModel.generateObject`

Asks the model to produce a value matching a schema, then decodes and validates the JSON output through that schema. Resolves to a GenerateObjectResponse whose .value is the fully typed, decoded object. A decoding failure surfaces as an AiError, so a successful effect always carries a valid object.

import { Effect, Schema } from "effect"
import { LanguageModel } from "effect/unstable/ai"

const EventSchema = Schema.Struct({
  title: Schema.String,
  date: Schema.String,
  location: Schema.String
})

const program = Effect.gen(function*() {
  const response = yield* LanguageModel.generateObject({
    prompt: "Extract event info: Tech Conference on March 15th in San Francisco",
    schema: EventSchema,
    objectName: "event"
  })

  console.log(response.value)
  // => { title: "Tech Conference", date: "March 15th", location: "San Francisco" }
  return response.value
})

`LanguageModel.streamText`

Returns a Stream of typed Response.StreamParts emitted as the model produces them, rather than a single value. Filter for the part types you care about and map them into your own shape.

import { Console, Effect, Stream } from "effect"
import { LanguageModel } from "effect/unstable/ai"

const program = LanguageModel.streamText({
  prompt: "Write a story about a space explorer"
}).pipe(
  Stream.runForEach((part) => {
    if (part.type === "text-delta") {
      return Console.log(part.delta) // => incremental text chunks
    }
    return Effect.void
  })
)

The part taxonomy (text-delta, reasoning-delta, tool-call, finish, …) is documented under Tools. See Streaming for operators to consume the stream.

`LanguageModel.make`

Builds a Service from provider-specific generation hooks. You only call this when authoring a provider integration; application code uses the provider package’s model(...) helper instead.

import { Effect, Stream } from "effect"
import { LanguageModel } from "effect/unstable/ai"
import type { Response } from "effect/unstable/ai"

const service = LanguageModel.make({
  // Return the final, encoded content parts for a completed generation.
  generateText: (_options) => Effect.succeed<Array<Response.PartEncoded>>([]),
  // Emit encoded stream parts as the provider produces them.
  streamText: (_options) => Stream.empty,
  // Optional: rewrite structured-output schemas for the provider.
  // codecTransformer: LanguageModel.defaultCodecTransformer
})
// => Effect.Effect<LanguageModel.Service>

make prepares a ProviderOptions for each request (normalized prompt, tools, tool choice, response format, tracing span, incremental fields) and calls your hooks. Structured output uses the generateText hook plus the configured codecTransformer, defaulting to defaultCodecTransformer.

`GenerateTextOptions`

The option object accepted by generateText and streamText. Only prompt is required.

Field	Type	Description
`prompt`	`Prompt.RawInput`	The prompt — a string, message array, or `Prompt`.
`toolkit`	`ToolkitInput`	A toolkit (tools + handler) the model may call.
`toolChoice`	`ToolChoice`	Constrains whether/which tools are called (default `"auto"`).
`concurrency`	`Concurrency`	Concurrency used when resolving tool calls.
`disableToolCallResolution`	`boolean`	Pass tool definitions but resolve calls yourself.

import { Effect } from "effect"
import { LanguageModel } from "effect/unstable/ai"

const program = LanguageModel.generateText({
  // A bare string is the simplest prompt; you can also pass message arrays.
  prompt: "Summarize the plot of Hamlet in two sentences",
  toolChoice: "none",
  concurrency: "unbounded"
}) // => Effect<GenerateTextResponse<{}>, AiError, LanguageModel>

`GenerateObjectOptions`

Extends GenerateTextOptions with the fields that drive structured output.

Field	Type	Description
`schema`	`Schema.Top`	The structure to generate; the result is decoded through it.
`objectName`	`string`	Optional name passed to the provider for guidance.

import { Schema } from "effect"
import { LanguageModel } from "effect/unstable/ai"

const Person = Schema.Struct({ name: Schema.String, age: Schema.Number })

const program = LanguageModel.generateObject({
  prompt: "Describe a fictional person",
  schema: Person,
  objectName: "person"
}) // => Effect<GenerateObjectResponse<{}, { name: string; age: number }>, ...>

When objectName is omitted, the schema’s identifier is used (falling back to "generateObject").

`ToolChoice`

The union that constrains tool usage. ToolName is inferred from the toolkit.

"auto" (default) — the model decides whether and which tools to call.
"required" — the model must call some tool.
"none" — the model must not call a tool.
{ tool: "<name>" } — the model must call exactly this tool.
{ mode?: "auto" | "required"; oneOf: ["<name>", ...] } — restrict to a subset.

import type { LanguageModel } from "effect/unstable/ai"

const auto: LanguageModel.ToolChoice<"getWeather"> = "auto"
const forced: LanguageModel.ToolChoice<"getWeather"> = { tool: "getWeather" }
const subset: LanguageModel.ToolChoice<"getWeather" | "getTime"> = {
  mode: "required",
  oneOf: ["getWeather", "getTime"]
}

`GenerateTextResponse`

The result of generateText. It wraps the raw content parts and exposes convenience accessors so you rarely walk the parts by hand.

Accessor	Type	Description
`content`	`Array<Response.Part>`	The raw, decoded response parts.
`text`	`string`	All `text` parts concatenated.
`reasoning`	`Array<Response.ReasoningPart>`	Reasoning parts, if the model emits them.
`reasoningText`	`string \| undefined`	Concatenated reasoning text, or `undefined`.
`toolCalls`	`Array<...>`	Tool-call parts (populated with a toolkit).
`toolResults`	`Array<...>`	Resolved tool-result parts.
`finishReason`	`Response.FinishReason`	Why generation stopped (see below).
`usage`	`Response.Usage`	Token usage statistics (see below).

import { Effect } from "effect"
import { LanguageModel } from "effect/unstable/ai"

const program = Effect.gen(function*() {
  const response = yield* LanguageModel.generateText({ prompt: "Explain photosynthesis" })

  console.log(response.text) // => the explanation
  console.log(response.finishReason) // => "stop"
  console.log(response.reasoningText) // => undefined (unless a reasoning model)
  console.log(response.usage.outputTokens.total) // => e.g. 84
  return response
})

`GenerateObjectResponse`

Extends GenerateTextResponse (so it has all the accessors above) and adds value: the parsed, schema-validated object.

import { Effect, Schema } from "effect"
import { LanguageModel } from "effect/unstable/ai"

const User = Schema.Struct({ name: Schema.String, email: Schema.String })

const program = Effect.gen(function*() {
  const response = yield* LanguageModel.generateObject({
    prompt: "Create user: John Doe, john@example.com",
    schema: User
  })

  console.log(response.value) // => { name: "John Doe", email: "john@example.com" }
  console.log(response.text) // => the raw JSON text the model produced
  return response.value
})

`Response.FinishReason`

The literal union returned by response.finishReason: "stop", "length", "content-filter", "tool-calls", "error", "pause", "other", or "unknown".

import { Effect } from "effect"
import { LanguageModel } from "effect/unstable/ai"

const program = Effect.gen(function*() {
  const response = yield* LanguageModel.generateText({ prompt: "Write 5000 words" })
  if (response.finishReason === "length") {
    // The model hit its output-token limit before finishing.
  }
  return response.finishReason
})

`Response.Usage`

Token accounting attached to every finished generation. Every field is number | undefined (providers report different subsets).

import { Effect } from "effect"
import { LanguageModel } from "effect/unstable/ai"

const program = Effect.gen(function*() {
  const { usage } = yield* LanguageModel.generateText({ prompt: "Hi" })

  usage.inputTokens.total // => e.g. 10
  usage.inputTokens.uncached // => non-cached prompt tokens
  usage.inputTokens.cacheRead // => cached prompt tokens read
  usage.inputTokens.cacheWrite // => cached prompt tokens written
  usage.outputTokens.total // => e.g. 4
  usage.outputTokens.text // => text tokens
  usage.outputTokens.reasoning // => reasoning tokens
  return usage
})

`ProviderOptions`

The normalized request a provider hook receives from make: the prompt (a Prompt.Prompt), tools, toolChoice, responseFormat ({ type: "text" } or { type: "json"; objectName; schema }), the tracing span, and the incremental fields previousResponseId / incrementalPrompt. You read these inside a custom provider; application code never constructs one.

`CodecTransformer`

A function (schema) => { codec; jsonSchema } that rewrites a structured-output codec’s encoded side to satisfy a provider’s JSON-schema constraints while preserving the decoded type. Supplied to make via codecTransformer.

import type { LanguageModel } from "effect/unstable/ai"

declare const transform: LanguageModel.CodecTransformer
// const { codec, jsonSchema } = transform(MySchema)

`defaultCodecTransformer`

The CodecTransformer used when make is given no codecTransformer. It passes the codec through unchanged, resolves a top-level $ref, and copies definitions into $defs — appropriate for providers that accept the schema generated from an Effect codec as-is.

import { LanguageModel } from "effect/unstable/ai"

LanguageModel.defaultCodecTransformer // => CodecTransformer

Toolkit utility types

These type-level helpers let callers and provider authors describe and inspect the toolkit attached to a request. They have no runtime presence.

ToolkitOption<Tools, E, R> — the distributive form of an accepted toolkit: a Toolkit.WithHandler<Tools> or an Effect yielding one. Used for inference over toolkitA | toolkitB unions.
ToolkitInput<Tools, E, R> — the non-distributive form used for call-site assignability of the toolkit option.
ExtractTools<Options> — extracts the tool record from an options object’s toolkit, or {} when there is none.
ExtractError<Options> — infers the failure channel: AiError plus the toolkit’s handler errors (unless disableToolCallResolution: true).
ExtractServices<Options> — infers the services the toolkit’s handlers require.

import type { LanguageModel } from "effect/unstable/ai"

declare const options: { prompt: string }
type Tools = LanguageModel.ExtractTools<typeof options> // => {}
type Err = LanguageModel.ExtractError<typeof options> // => AiError

Tools — let the model call functions you define, and the Response.StreamPart taxonomy in full.
Chat — keep conversation history across many turns.
Streaming — operators for working with streamText output.
Schema — define the structures behind generateObject.
Tagged errors — wrap AiError in your own domain error.

Language Model

Setting up a provider

A complete service

Selecting and providing a model

Handling errors

API reference

LanguageModel.LanguageModel

LanguageModel.Service

LanguageModel.generateText

LanguageModel.generateObject

LanguageModel.streamText

LanguageModel.make

GenerateTextOptions

GenerateObjectOptions

ToolChoice

GenerateTextResponse

GenerateObjectResponse

Response.FinishReason

Response.Usage

ProviderOptions

CodecTransformer

defaultCodecTransformer