# Tokenizer, Model, and Telemetry

The Effect AI SDK ships several small supporting modules that the higher-level
[`LanguageModel`](https://effect.plants.sh/ai/language-model/) builds on. They are useful on their own
when you need to count tokens, inspect which provider is active, generate
stable identifiers, or attach OpenTelemetry GenAI attributes to your spans.

All of these modules live under `effect/unstable/ai`:

```ts
import {
  IdGenerator,
  Model,
  ResponseIdTracker,
  Telemetry,
  Tokenizer
} from "effect/unstable/ai"
```
**Unstable:** These modules are part of the unstable AI surface. Import paths and APIs may
  change between minor releases.

## Tokenizer

A `Tokenizer` converts a prompt into provider-specific token ids and truncates
a prompt to fit within a token budget. It is a service because tokenization
depends on the target model, provider, and encoding rules — not on Effect
itself. Provider packages usually supply a `Tokenizer`, but you can build your
own with `Tokenizer.make`.

The common cases are counting tokens before choosing a model or request size,
and dropping older conversation messages before a chat completion call.

```ts
import { Effect } from "effect"
import { Tokenizer } from "effect/unstable/ai"

// A simple word-based tokenizer for tests / unsupported providers.
// `prompt.content` is a normalized array of messages.
const wordTokenizer = Tokenizer.make({
  tokenize: (prompt) =>
    Effect.succeed(
      prompt.content
        .flatMap((msg) =>
          typeof msg.content === "string"
            ? msg.content.split(" ")
            : msg.content.flatMap((part) =>
                part.type === "text" ? part.text.split(" ") : []
              )
        )
        .map((_, index) => index)
    )
})

const program = Effect.gen(function* () {
  const tokenizer = yield* Tokenizer.Tokenizer

  // Count tokens in a raw string prompt
  const tokens = yield* tokenizer.tokenize("Count the tokens in this prompt")
  yield* Effect.log(`token count: ${tokens.length}`) // => token count: 6

  // Keep the prompt under a 100-token budget
  const trimmed = yield* tokenizer.truncate(
    "A potentially very long conversation...",
    100
  )
  return trimmed
}).pipe(Effect.provideService(Tokenizer.Tokenizer, wordTokenizer))
```
**Note:** `truncate` keeps **complete** messages from the **end** of the prompt until
  the next message would exceed the budget; it never splits a message to fit
  the remaining space. Token ids are model-specific, so counts from one
  tokenizer will not match another model's tokenizer.

### Reference

#### `Tokenizer` (service)

`Context.Service` tag (id `"effect/ai/Tokenizer"`) used to access or provide a
tokenization service. Programs that read it require a tokenizer to be provided.

```ts
import { Effect } from "effect"
import { Tokenizer } from "effect/unstable/ai"

const useTokenizer = Effect.gen(function* () {
  const tokenizer = yield* Tokenizer.Tokenizer
  const tokens = yield* tokenizer.tokenize("Hello, world!")
  return tokens.length // => number
})
```

#### `Service`

The service interface. Has two methods, both accepting a `Prompt.RawInput`
(a string, a message array, or a `Prompt`):

- `tokenize(input)` → `Effect<Array<number>, AiError>` — token ids for the input.
- `truncate(input, tokens)` → `Effect<Prompt, AiError>` — a prompt trimmed to at
  most `tokens` tokens.

```ts
import { Effect } from "effect"
import { Prompt } from "effect/unstable/ai"
import type { Tokenizer } from "effect/unstable/ai"

const custom: Tokenizer.Service = {
  tokenize: (input) =>
    Effect.succeed(input.toString().split(" ").map((_, i) => i)),
  truncate: (input, maxTokens) =>
    Effect.succeed(Prompt.make(input.toString().slice(0, maxTokens * 5)))
}
```

#### `make`

Builds a `Service` from a single `tokenize` function that operates on a
normalized `Prompt`. The `truncate` implementation is derived for you (it
repeatedly tokenizes trailing messages and stops when the budget is reached).

```ts
import { Effect } from "effect"
import { Tokenizer } from "effect/unstable/ai"

const tokenizer = Tokenizer.make({
  // Pretend each message is one token
  tokenize: (prompt) => Effect.succeed(prompt.content.map((_, i) => i))
})
// tokenizer.tokenize / tokenizer.truncate are now available
```

## Model

A `Model` wraps a provider-specific language model `Layer` together with the
metadata Effect AI needs at runtime. A `Model` **is** a `Layer`, but providing
it also installs two metadata services — `Model.ProviderName` and
`Model.ModelName` — so any program can inspect which provider/model is in use
(useful for telemetry, routing, or provider-agnostic services).

Provider packages (OpenAI, Anthropic, Bedrock, etc.) construct `Model`s for you;
`Model.make` is the building block they use.

```ts
import { Effect, Layer } from "effect"
import { LanguageModel, Model } from "effect/unstable/ai"

declare const languageModelLayer: Layer.Layer<LanguageModel.LanguageModel>

// Combine a provider id, a model id, and the layer that provides AI services.
const model = Model.make("openai", "gpt-5", languageModelLayer)

const program = Effect.gen(function* () {
  // ProviderName and ModelName are provided automatically by the Model
  const provider = yield* Model.ProviderName // => "openai"
  const modelName = yield* Model.ModelName // => "gpt-5"

  const response = yield* LanguageModel.generateText({
    prompt: `Hello from ${provider}/${modelName}!`
  })
  return response.text
}).pipe(Effect.provide(model))
```
**Note:** The provider and model names are metadata only — the actual implementation
  still comes from the layer passed to `Model.make`. If that layer has
  requirements, they must be satisfied before the model can be provided.

### Reference

#### `Model<Provider, Provides, Requires>`

An interface extending `Layer.Layer<Provides | ProviderName | ModelName, never,
Requires>`. Beyond the `Layer` shape it carries:

- `provider` — the provider identifier (e.g. `"openai"`, `"anthropic"`,
  `"amazon-bedrock"`).
- `captureRequirements` — an `Effect` that captures the current context into a
  concrete `Layer` whose requirements are already satisfied. Useful when you
  need a `Model` from inside an Effect service and want to "lift" its
  dependencies into the parent Effect.

```ts
import { Effect, Layer } from "effect"
import { LanguageModel, Model } from "effect/unstable/ai"

declare const layer: Layer.Layer<LanguageModel.LanguageModel>
const model = Model.make("anthropic", "claude-3-5-haiku", layer)

model.provider // => "anthropic"

// Lift the Model's requirements into the surrounding Effect
const lifted = Effect.gen(function* () {
  const ready: Layer.Layer<unknown> = yield* model.captureRequirements
  return ready
})
```

#### `ProviderName` (service)

`Context.Service<ProviderName, string>` (id
`"effect/unstable/ai/Model/ProviderName"`). Automatically provided by a `Model`;
read it to discover the active provider name.

```ts
import { Effect } from "effect"
import { Model } from "effect/unstable/ai"

const readProvider = Effect.gen(function* () {
  const provider = yield* Model.ProviderName
  return provider // => the provider string, e.g. "openai"
})
```

#### `ModelName` (service)

`Context.Service<ModelName, string>` (id
`"effect/unstable/ai/Model/ModelName"`). Automatically provided by a `Model`;
read it to discover the active model name.

```ts
import { Effect } from "effect"
import { Model } from "effect/unstable/ai"

const readModel = Effect.gen(function* () {
  const modelName = yield* Model.ModelName
  return modelName // => the model string, e.g. "gpt-5"
})
```

#### `make`

Creates a `Model` from a provider name, a model name, and a `Layer` of AI
services. The result merges the provided layer with the `ProviderName` /
`ModelName` metadata services.

```ts
import { Layer } from "effect"
import { LanguageModel, Model } from "effect/unstable/ai"

declare const bedrockLayer: Layer.Layer<LanguageModel.LanguageModel>

const model = Model.make("amazon-bedrock", "claude-3-5-haiku", bedrockLayer)
// => Model<"amazon-bedrock", LanguageModel, never>
```

## IdGenerator

`IdGenerator` is a pluggable service for generating unique identifiers for tool
calls, assistant messages, and other generated items. It supports custom
alphabets, prefixes, separators, and sizes — useful when you want deterministic
IDs in tests or provider-specific ID formats in production.

```ts
import { Effect } from "effect"
import { IdGenerator } from "effect/unstable/ai"

// A custom generator for tool-call IDs
const toolCallIdLayer = IdGenerator.layer({
  alphabet: "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ",
  prefix: "tool_call",
  separator: "_",
  size: 12
})

const program = Effect.gen(function* () {
  const idGen = yield* IdGenerator.IdGenerator
  const id = yield* idGen.generateId()
  return id // => "tool_call_A7XK9MP2QR5T"
}).pipe(Effect.provide(toolCallIdLayer))
```

If you do not provide your own, use `IdGenerator.defaultIdGenerator`, which
emits `id_XXXXXXXXXXXXXXXX`-style identifiers.
**Note:** The random portion is drawn from the Effect `Random` service, so seeding it
  (for example with `Effect.withRandom`) makes generated IDs deterministic in
  tests.

### Reference

#### `IdGenerator` (service)

`Context.Service<IdGenerator, Service>` (id `"@effect/ai/IdGenerator"`). Access
or provide ID generation throughout your application.

```ts
import { Effect } from "effect"
import { IdGenerator } from "effect/unstable/ai"

const useIdGenerator = Effect.gen(function* () {
  const idGenerator = yield* IdGenerator.IdGenerator
  return yield* idGenerator.generateId() // => string
})
```

#### `Service`

The service interface: a single `generateId(): Effect<string>` method. Any
implementation that satisfies this can be provided as the service.

```ts
import { Effect } from "effect"
import type { IdGenerator } from "effect/unstable/ai"

let nextId = 0
const customService: IdGenerator.Service = {
  generateId: () => Effect.sync(() => `custom_${++nextId}`)
}
// customService.generateId() => Effect that yields "custom_1", "custom_2", ...
```

#### `MakeOptions`

Configuration for custom generators:

- `alphabet` — character set for the random portion (required).
- `prefix` — optional string prepended to the id.
- `separator` — character between prefix and random portion (must **not** be
  part of the alphabet).
- `size` — length of the random portion.

```ts
import type { IdGenerator } from "effect/unstable/ai"

const options: IdGenerator.MakeOptions = {
  alphabet: "0123456789ABCDEF",
  prefix: "tool",
  separator: "_",
  size: 8
}
// Generates IDs like: "tool_A1B2C3D4"
```

#### `defaultIdGenerator`

A ready-made `Service` using the `"id"` prefix and a 16-character alphanumeric
random portion.

```ts
import { Effect } from "effect"
import { IdGenerator } from "effect/unstable/ai"

const program = Effect.gen(function* () {
  const id = yield* IdGenerator.defaultIdGenerator.generateId()
  return id // => "id_A7xK9mP2qR5tY8uV"
})
```

#### `make`

An `Effect` that builds a custom `Service` from `MakeOptions`. It fails with
`Cause.IllegalArgumentError` if the separator appears in the alphabet (which
would make IDs ambiguous to parse).

```ts
import { Effect } from "effect"
import { IdGenerator } from "effect/unstable/ai"

const program = Effect.gen(function* () {
  const gen = yield* IdGenerator.make({
    alphabet: "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ",
    prefix: "msg",
    separator: "-",
    size: 10
  })
  return yield* gen.generateId() // => "msg-A7X9K2M5P8"
})
```

#### `layer`

The recommended way to provide an `IdGenerator`. Wraps `make` in a `Layer`; the
layer fails during construction if the options are invalid.

```ts
import { Effect } from "effect"
import { IdGenerator } from "effect/unstable/ai"

const idLayer = IdGenerator.layer({
  alphabet: "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ",
  prefix: "tool_call",
  separator: "_",
  size: 12
})

const program = Effect.gen(function* () {
  const idGen = yield* IdGenerator.IdGenerator
  return yield* idGen.generateId() // => "tool_call_A7XK9MP2QR5T"
}).pipe(Effect.provide(idLayer))
```

## ResponseIdTracker

Some providers can continue from a prior response by accepting a
`previousResponseId` plus only the messages added since that response.
`ResponseIdTracker` is a small mutable cache that remembers which exact prompt
message objects were included in each provider response, then prepares a shorter
prompt when a later call extends the same conversation.

It is an optimization cache used by language model implementations — not
conversation storage. You typically just `provide` it and let the provider use
it.

```ts
import { Effect, Option } from "effect"
import { Prompt, ResponseIdTracker } from "effect/unstable/ai"

const program = Effect.gen(function* () {
  const tracker = yield* ResponseIdTracker.make

  // After a provider responds, associate the messages that were sent
  // with the provider's response id.
  const firstPrompt = Prompt.make("Hello")
  tracker.markParts(firstPrompt.content, "resp_123")

  // On a follow-up turn, prepare a shorter prompt. If the prefix is fully
  // tracked, you get back the previous response id plus only the new messages.
  const followUp = Prompt.fromMessages([
    ...firstPrompt.content,
    { role: "assistant", content: "Hi!" },
    { role: "user", content: "How are you?" }
  ] as any)

  const prepared = tracker.prepareUnsafe(followUp)
  if (Option.isSome(prepared)) {
    prepared.value.previousResponseId // => "resp_123"
    prepared.value.prompt // => prompt with only the messages after the last assistant turn
  }
})
```
**Caution:** Tracking is based on **object identity** — equivalent message values are only
  recognized if they are the same objects. The service intentionally exposes
  `Unsafe` methods because callers coordinate it inside provider
  request/response code. A prefix mismatch clears all tracked state so a
  response id is never reused for the wrong prompt.

### Reference

#### `ResponseIdTracker` (service)

`Context.Service<ResponseIdTracker, Service>` (id
`"effect/ai/ResponseIdTracker"`). Provide it so a language model can send only
new prompt messages alongside the provider's prior response id.

```ts
import { Effect } from "effect"
import { ResponseIdTracker } from "effect/unstable/ai"

const useTracker = Effect.gen(function* () {
  const tracker = yield* ResponseIdTracker.ResponseIdTracker
  return tracker // => ResponseIdTracker.Service
})
```

#### `Service`

The mutable service interface:

- `markParts(parts, responseId)` — record the message objects that produced a
  response.
- `prepareUnsafe(prompt)` → `Option<PrepareResult>` — returns the previous
  response id and the untracked suffix when the prefix is fully recognized;
  otherwise clears state and returns `Option.none()`.
- `clearUnsafe()` — drop all tracked state.

```ts
import { Option } from "effect"
import type { ResponseIdTracker } from "effect/unstable/ai"

declare const tracker: ResponseIdTracker.Service
tracker.clearUnsafe() // => void, forgets everything tracked so far
```

#### `PrepareResult`

The shape returned (inside an `Option`) by `prepareUnsafe`:

- `previousResponseId: string` — pass this to the provider as
  `previousResponseId`.
- `prompt: Prompt` — only the new messages after the latest assistant turn.

```ts
import type { ResponseIdTracker } from "effect/unstable/ai"

declare const result: ResponseIdTracker.PrepareResult
result.previousResponseId // => "resp_123"
result.prompt // => Prompt containing only the new messages
```

#### `make`

An `Effect<Service>` that constructs a fresh in-memory tracker.

```ts
import { Effect } from "effect"
import { ResponseIdTracker } from "effect/unstable/ai"

const program = Effect.gen(function* () {
  const tracker = yield* ResponseIdTracker.make // => ResponseIdTracker.Service
  return tracker
})
```

## Telemetry

The `Telemetry` module models the OpenTelemetry
[GenAI semantic-convention](https://opentelemetry.io/docs/specs/semconv/attributes-registry/gen-ai/)
attributes (`gen_ai.*`) and exposes small helpers for writing them onto Effect
tracing spans. Provider implementations use it to annotate spans with request,
response, and token-usage metadata; applications can use it to add consistent
GenAI metadata around their own model calls.

These helpers **annotate spans that already exist** — they do not create or
scope spans. Attribute writers mutate the provided span, emit only non-nullish
values, and convert camelCase field names to snake_case keys.

```ts
import { Effect } from "effect"
import { LanguageModel, Telemetry } from "effect/unstable/ai"

// Wrap a generateText call with GenAI annotations on the current span.
const generateWithTelemetry = Effect.gen(function* () {
  const response = yield* LanguageModel.generateText({
    prompt: "Explain Effect in one sentence"
  })

  const span = yield* Effect.currentSpan
  Telemetry.addGenAIAnnotations(span, {
    system: "openai",
    operation: { name: "chat" },
    request: { model: "gpt-5", temperature: 0.7 },
    response: {
      finishReasons: [response.finishReason]
    },
    usage: {
      inputTokens: response.usage.inputTokens.total,
      outputTokens: response.usage.outputTokens.total
    }
  })

  return response.text
}).pipe(Effect.withSpan("generate-text"))
// Span now has gen_ai.system, gen_ai.operation.name, gen_ai.request.model,
// gen_ai.request.temperature, gen_ai.response.finish_reasons,
// gen_ai.usage.input_tokens, gen_ai.usage.output_tokens
```

### Reference

#### `addGenAIAnnotations`

Applies standard GenAI attributes to a span. Dual-signature: call it with a
span and options directly, or pre-apply options to get `(span) => void`. Only
non-nullish values are written. **Mutates the span in-place.**

```ts
import { Effect } from "effect"
import { Telemetry } from "effect/unstable/ai"

const annotate = Effect.gen(function* () {
  const span = yield* Effect.currentSpan
  Telemetry.addGenAIAnnotations(span, {
    system: "anthropic",
    usage: { inputTokens: 100, outputTokens: 50 }
  })
  // => sets gen_ai.system, gen_ai.usage.input_tokens, gen_ai.usage.output_tokens

  // Curried form
  const apply = Telemetry.addGenAIAnnotations({ request: { model: "claude-3-5" } })
  apply(span) // => sets gen_ai.request.model
})
```

#### `addSpanAttributes`

Creates a reusable attribute writer for an arbitrary key prefix and key
transformer. The returned function writes each non-nullish attribute as
`${prefix}.${transformKey(key)}`.

```ts
import { String } from "effect"
import type { Tracer } from "effect"
import { Telemetry } from "effect/unstable/ai"

const addCustom = Telemetry.addSpanAttributes("custom.ai", String.camelToSnake)

declare const span: Tracer.Span
addCustom(span, { modelName: "gpt-5", maxTokens: 1000 })
// => sets custom.ai.model_name and custom.ai.max_tokens
```

#### `GenAITelemetryAttributeOptions`

The options object accepted by `addGenAIAnnotations`. It is `BaseAttributes`
(so `system` lives at the top level) plus optional grouped namespaces:
`operation`, `request`, `response`, `token`, and `usage`.

```ts
import type { Telemetry } from "effect/unstable/ai"

const options: Telemetry.GenAITelemetryAttributeOptions = {
  system: "openai",
  operation: { name: "chat" },
  request: { model: "gpt-4-turbo", temperature: 0.7, maxTokens: 2000 },
  response: { id: "chatcmpl-123", model: "gpt-4-turbo", finishReasons: ["stop"] },
  usage: { inputTokens: 50, outputTokens: 25 }
}
```

#### Attribute interfaces

The grouped attribute interfaces map directly to `gen_ai.*` namespaces. All
fields are optional and nullable; nullish values are skipped when written.

- `BaseAttributes` — `gen_ai.*`. Field: `system` (a `WellKnownSystem` or custom
  string).
- `OperationAttributes` — `gen_ai.operation.*`. Field: `name` (a
  `WellKnownOperationName` or custom string).
- `TokenAttributes` — `gen_ai.token.*`. Field: `type`.
- `UsageAttributes` — `gen_ai.usage.*`. Fields: `inputTokens`, `outputTokens`.
- `RequestAttributes` — `gen_ai.request.*`. Fields: `model`, `temperature`,
  `topK`, `topP`, `maxTokens`, `encodingFormats`, `stopSequences`,
  `frequencyPenalty`, `presencePenalty`, `seed`.
- `ResponseAttributes` — `gen_ai.response.*`. Fields: `id`, `model`,
  `finishReasons`.

```ts
import type { Telemetry } from "effect/unstable/ai"

const request: Telemetry.RequestAttributes = {
  model: "gpt-5",
  temperature: 0.7,
  maxTokens: 1024,
  stopSequences: ["\n\n"]
}
const usage: Telemetry.UsageAttributes = { inputTokens: 50, outputTokens: 25 }
```

There are also two aggregate types: `AllAttributes` (the intersection of all
six interfaces) and `GenAITelemetryAttributes` (the fully prefixed, snake-cased
shape produced on the span).

#### `WellKnownOperationName`

The well-known values for `gen_ai.operation.name`: `"chat"`, `"embeddings"`,
`"text_completion"`. A custom string may be used if none apply.

```ts
import type { Telemetry } from "effect/unstable/ai"

const op: Telemetry.WellKnownOperationName = "chat" // => "chat"
```

#### `WellKnownSystem`

The well-known values for `gen_ai.system`, including `"anthropic"`,
`"aws.bedrock"`, `"az.ai.inference"`, `"az.ai.openai"`, `"cohere"`,
`"deepseek"`, `"gemini"`, `"groq"`, `"ibm.watsonx.ai"`, `"mistral_ai"`,
`"openai"`, `"perplexity"`, `"vertex_ai"`, `"xai"`. A custom string may be used
if none apply.

```ts
import type { Telemetry } from "effect/unstable/ai"

const system: Telemetry.WellKnownSystem = "openai" // => "openai"
```

#### `SpanTransformer`

A function that receives the provider request options plus the array of
`response` parts and mutates the span. Use it to derive custom attributes from
the actual response content.

```ts
import type { Telemetry } from "effect/unstable/ai"

const transformer: Telemetry.SpanTransformer = ({ response, span }) => {
  const totalTextLength = response.reduce(
    (sum, part) => sum + (part.type === "text" ? part.text.length : 0),
    0
  )
  span.attribute("total_text_length", totalTextLength) // => sets total_text_length
}
```

#### `CurrentSpanTransformer` (service)

`Context.Service<CurrentSpanTransformer, SpanTransformer>` (id
`"effect/ai/Telemetry/CurrentSpanTransformer"`). Provide a `SpanTransformer`
through context so a language model implementation can annotate the span after
it has seen the provider response.

```ts
import { Effect } from "effect"
import { Telemetry } from "effect/unstable/ai"

const transformer: Telemetry.SpanTransformer = ({ span }) => {
  span.attribute("annotated_by", "custom-transformer")
}

const program = Effect.gen(function* () {
  // language model operations within this scope will run the transformer
  return yield* Effect.void
}).pipe(Effect.provideService(Telemetry.CurrentSpanTransformer, transformer))
```

#### Utility types

- `AttributesWithPrefix<Attributes, Prefix>` — prefixes each key with `Prefix.`
  and snake-cases it (e.g. `{ maxTokens: number }` with prefix
  `"gen_ai.request"` → `{ "gen_ai.request.max_tokens": number }`).
- `FormatAttributeName<T>` — converts a camelCase literal to snake_case (e.g.
  `"modelName"` → `"model_name"`).

```ts
import type { Telemetry } from "effect/unstable/ai"

type Formatted = Telemetry.FormatAttributeName<"maxTokens"> // => "max_tokens"
```