Language Model

LanguageModel is the central abstraction of Effect’s AI modules. It exposes three operations — generateText, generateObject, and streamText — that all work against whichever provider you supply through a Layer. Your code depends on the abstract LanguageModel.LanguageModel service; the concrete model is injected at the edges.

Setting up a provider

A provider is wired up in two pieces: a client Layer that holds credentials and an HttpClient, and a model Layer that selects a specific model. Both come from a provider package such as @effect/ai-openai.

import { OpenAiClient } from "@effect/ai-openai"
import { Config, Layer } from "effect"
import { FetchHttpClient } from "effect/unstable/http"

// The client Layer holds your API key (read from Config, so it never gets
// hard-coded) and depends on an HttpClient implementation. Providers let you
// choose which HttpClient to use — here the platform-agnostic Fetch client.
const OpenAiClientLayer = OpenAiClient.layerConfig({
  apiKey: Config.redacted("OPENAI_API_KEY")
}).pipe(Layer.provide(FetchHttpClient.layer))

OpenAiLanguageModel.model("gpt-5.2") returns a Layer that provides a concrete LanguageModel (plus Model.ProviderName and Model.ModelName metadata) and requires the OpenAiClient. You provide that model Layer to the effect that runs the generation.

A complete service

Here is the idiomatic shape: a domain service that wraps the model behind your own API and your own error type. It generates text, decodes a structured object, and streams a response — all three operations in one place.

import { AnthropicClient, AnthropicLanguageModel } from "@effect/ai-anthropic"
import { OpenAiClient, OpenAiLanguageModel } from "@effect/ai-openai"
import { Config, Context, Effect, ExecutionPlan, Layer, Schema, Stream } from "effect"
import { AiError, LanguageModel, Model, type Response } from "effect/unstable/ai"
import { FetchHttpClient } from "effect/unstable/http"

// The schema we'll ask the model to fill in. Generated objects are decoded and
// validated through this schema, so a successful response is fully typed.
class LaunchPlan extends Schema.Class<LaunchPlan>("LaunchPlan")({
  audience: Schema.Literals(["developers", "operators", "platform teams"]),
  channels: Schema.Array(Schema.String),
  launchDate: Schema.String,
  summary: Schema.String,
  keyRisks: Schema.Array(Schema.String)
}) {}

// Client Layers for both providers.
const OpenAiClientLayer = OpenAiClient.layerConfig({
  apiKey: Config.redacted("OPENAI_API_KEY")
}).pipe(Layer.provide(FetchHttpClient.layer))

const AnthropicClientLayer = AnthropicClient.layerConfig({
  apiKey: Config.redacted("ANTHROPIC_API_KEY")
}).pipe(Layer.provide(FetchHttpClient.layer))

// A domain error. `AiError.AiErrorReason` is itself a Schema, so we can embed
// it directly in a tagged error of our own.
class AiWriterError extends Schema.TaggedErrorClass<AiWriterError>()("AiWriterError", {
  reason: AiError.AiErrorReason
}) {
  static fromAiError(error: AiError.AiError) {
    return new AiWriterError({ reason: error.reason })
  }
}

// An ExecutionPlan tries a cheaper OpenAI model first (up to 3 attempts), then
// falls back to an Anthropic model (up to 2 attempts) if the first keeps failing.
const DraftPlan = ExecutionPlan.make(
  { provide: OpenAiLanguageModel.model("gpt-5.2"), attempts: 3 },
  { provide: AnthropicLanguageModel.model("claude-opus-4-6"), attempts: 2 }
)

class AiWriter extends Context.Service<AiWriter, {
  draftAnnouncement(product: string): Effect.Effect<string, AiWriterError>
  extractLaunchPlan(notes: string): Effect.Effect<LaunchPlan, AiWriterError>
  streamReleaseHighlights(version: string): Stream.Stream<string, AiWriterError>
}>()("docs/AiWriter") {
  static readonly layer = Layer.effect(
    AiWriter,
    Effect.gen(function*() {
      // `captureRequirements` moves the plan's requirements (the AI clients)
      // into this Layer's requirements, so they get provided at the bottom.
      const draftsModel = yield* DraftPlan.captureRequirements
      // A single model for the other operations.
      const model = yield* OpenAiLanguageModel.model("gpt-4.1").captureRequirements

      // --- generateText: plain text plus rich metadata --------------------
      const draftAnnouncement = Effect.fn("AiWriter.draftAnnouncement")(
        function*(product: string) {
          const provider = yield* Model.ProviderName
          const response = yield* LanguageModel.generateText({
            prompt: `Write a short launch announcement for ${product}.`
          })

          // The response carries convenience accessors so you don't have to walk
          // the raw content parts: `text`, `finishReason`, `usage`, `toolCalls`...
          yield* Effect.logInfo(
            `${provider} finished: ${response.finishReason}, ` +
              `outputTokens=${response.usage.outputTokens.total}`
          )

          return response.text
        },
        // Apply the ExecutionPlan so failures fall back across providers.
        Effect.withExecutionPlan(draftsModel),
        Effect.mapError(AiWriterError.fromAiError)
      )

      // --- generateObject: schema-validated structured output -------------
      const extractLaunchPlan = Effect.fn("AiWriter.extractLaunchPlan")(
        function*(notes: string) {
          const response = yield* LanguageModel.generateObject({
            // `objectName` names the structure for the provider.
            objectName: "launch_plan",
            prompt: `Convert these notes into a launch plan:\n${notes}`,
            // The model output is decoded through this schema before you see it.
            schema: LaunchPlan
          })

          // `response.value` is a fully decoded, typed `LaunchPlan`.
          return response.value
        },
        Effect.provide(model),
        Effect.mapError(AiWriterError.fromAiError)
      )

      // --- streamText: incremental output ---------------------------------
      const streamReleaseHighlights = (version: string) =>
        LanguageModel.streamText({
          prompt: `Write release highlights for version ${version} as bullets.`
        }).pipe(
          // streamText emits typed parts; keep only the incremental text deltas.
          Stream.filter((part): part is Response.TextDeltaPart => part.type === "text-delta"),
          Stream.map((part) => part.delta),
          Stream.provide(model),
          Stream.mapError(AiWriterError.fromAiError)
        )

      return AiWriter.of({ draftAnnouncement, extractLaunchPlan, streamReleaseHighlights })
    })
  ).pipe(
    // This Layer needs both clients, since the plan spans both providers.
    Layer.provide([OpenAiClientLayer, AnthropicClientLayer])
  )
}

The three operations

generateText

Returns a GenerateTextResponse. Beyond response.text, the response exposes convenience accessors computed from the underlying content parts:

• text — all text parts concatenated.
• finishReason — why generation stopped (e.g. "stop", "length", "tool-calls").
• usage — token counts (usage.inputTokens.total, usage.outputTokens.total, cache reads, reasoning tokens, …).
• reasoning / reasoningText — reasoning parts, when the model emits them.
• toolCalls / toolResults — populated when you pass a toolkit (see Tools).

generateObject

Takes a schema and an objectName, asks the model to produce a matching structure, then decodes and validates the result through the schema. You get back a GenerateObjectResponse whose value is the fully typed, decoded object — plus the same metadata accessors as generateText. Validation failures surface as an AiError, so a successful effect always carries a valid object.

streamText

Returns a Stream of typed Response.StreamParts rather than a single value. Filter for the part types you care about — text-delta for incremental text, tool-call for tool invocations — and map them into your own shape. The example above narrows to TextDeltaPart and projects the delta string.

Calling the service directly

LanguageModel.generateText(...) is sugar for yield* LanguageModel.LanguageModel followed by model.generateText(...). Use the module-level helpers in most code; yield the service tag directly when you need to thread the same model instance through several calls.

Selecting and providing a model

Every generation needs a concrete model in its environment. Provider packages give you a model(...) helper returning a Model.Model Layer:

• Effect.provide(model) / Stream.provide(model) — run a single effect or stream with that model.
• model.captureRequirements — within a Layer.effect, capture the model’s requirements (the client) into the surrounding Layer, as shown above.
• Effect.withExecutionPlan(plan) — apply an ExecutionPlan built from several models for retries and cross-provider fallback.

Model.ProviderName and Model.ModelName are services the model Layer also provides, so any code running under a model can read which provider and model answered.

Handling errors

All operations fail with AiError.AiError, a wrapper carrying module, method, and a reason describing what went wrong (rate limits, auth failures, malformed output, …). Each reason has an isRetryable getter. Because AiErrorReason is a Schema, you can embed it directly in your own tagged error, as AiWriterError does. Map at the boundary with Effect.mapError, or branch on the reason with Effect.catchTag / Effect.catch.

Related

• Tools — let the model call functions you define.
• Chat — keep conversation history across many turns.
• Streaming — operators for working with streamText output.
• Schema — define the structures behind generateObject.