Cache & ScopedCache

A Cache<Key, A, E, R> is a mutable, effectful key-value store. You give it a lookup function — (key: Key) => Effect<A, E, R> — and a capacity. Reading a key with Cache.get returns the cached value if present, or runs the lookup and stores the result. The headline feature is concurrency: when many fibers request the same missing key at once, the lookup runs once and every caller awaits the same result.

import { Cache, Effect } from "effect"

const program = Effect.gen(function*() {
  // The lookup is the only place the "real" work lives. Here we pretend
  // it is an expensive call; the cache makes sure it runs at most once
  // per key while the entry is alive.
  let lookups = 0
  const cache = yield* Cache.make({
    capacity: 100,
    lookup: (key: string) =>
      Effect.sync(() => {
        lookups++
        return key.length
      })
  })

  // Three concurrent gets of the same key: the lookup runs once,
  // and all three callers receive the same value.
  const results = yield* Effect.all(
    [Cache.get(cache, "hello"), Cache.get(cache, "hello"), Cache.get(cache, "hello")],
    { concurrency: "unbounded" }
  )

  return { results, lookups } // { results: [5, 5, 5], lookups: 1 }
})

The lookup returns an Effect, so it covers both synchronous computation and asynchronous I/O uniformly. The E and R type parameters flow from the lookup: a cache whose lookup can fail with E produces a Cache<Key, A, E>, and Cache.get fails with that same E.

Note

Both successes and failures are cached as Exit values. A failed lookup is remembered and will fail again on the next get until the entry expires, is invalidated, or is refreshed. If the lookup is interrupted, the entry is removed so the next get retries.

Creating a cache

Use Cache.make when every entry should share one TTL, and Cache.makeWith when the lifetime depends on the result or the key.

Fixed TTL

import { Cache, Effect } from "effect"

interface User {
  readonly id: number
  readonly name: string
}

// A cache of users keyed by id. Entries live for 15 minutes, then the
// next read re-runs the lookup. Capacity caps the number of entries.
const makeUserCache = (fetchUser: (id: number) => Effect.Effect<User, string>) =>
  Cache.make({
    capacity: 500,
    timeToLive: "15 minutes",
    lookup: fetchUser
  })

Dynamic TTL

import { Cache, Effect, Exit } from "effect"

interface User {
  readonly id: number
  readonly active: boolean
}

// makeWith receives the lookup Exit, so the TTL can depend on the
// outcome: cache successes for longer, retry failures sooner.
const makeUserCache = (fetchUser: (id: number) => Effect.Effect<User, string>) =>
  Cache.makeWith(fetchUser, {
    capacity: 500,
    timeToLive: (exit) =>
      Exit.isSuccess(exit)
        ? (exit.value.active ? "1 hour" : "5 minutes")
        : "30 seconds" // don't hammer a failing dependency, but retry soon
  })

TTLs accept a Duration input — a string like "15 minutes", a Duration value, or Duration.infinity (the default) for entries that never expire on their own.

Capacity and eviction

capacity bounds how many entries the cache holds. The cache tracks access order: reading an entry moves it to the back, and when the cache overflows the oldest (least recently used) entries are evicted first. The size may briefly exceed capacity between operations.

import { Cache, Effect } from "effect"

const program = Effect.gen(function*() {
  const cache = yield* Cache.make({
    capacity: 2,
    lookup: (key: string) => Effect.succeed(key.length)
  })

  yield* Cache.get(cache, "a")
  yield* Cache.get(cache, "b")
  yield* Cache.get(cache, "a") // touches "a", making "b" the oldest
  yield* Cache.get(cache, "c") // overflow: evicts "b"

  return {
    a: yield* Cache.has(cache, "a"), // true
    b: yield* Cache.has(cache, "b"), // false (evicted)
    c: yield* Cache.has(cache, "c") // true
  }
})

Reading, seeding, and invalidating

Beyond get, the cache exposes a full set of operations:

| Operation | What it does | | ------------------------------------ | ------------------------------------------------------------------------ | | Cache.get | Return the cached value, or run the lookup on a miss/expiry. | | Cache.getOption | Read an existing entry without running the lookup; None if absent. | | Cache.getSuccess | Like getOption, but only for already-resolved successful entries. | | Cache.set | Insert or overwrite a value directly, skipping the lookup. | | Cache.refresh | Force a fresh lookup, resetting the TTL and overwriting the old value. | | Cache.invalidate / invalidateAll | Evict one entry, or clear the whole cache. | | Cache.invalidateWhen | Evict an entry only if its value matches a predicate. | | Cache.has, size, keys, values, entries | Inspect the cache contents. |

A common pattern is a service that wraps a cache and exposes only the operations callers should use:

import { Cache, Context, Effect, Layer } from "effect"

// A service backed by a cache. Construction builds the cache once and
// captures it; the public method just delegates to Cache.get.
class UserRepo extends Context.Service<UserRepo, {
  readonly byId: (id: number) => Effect.Effect<string, string>
  readonly invalidate: (id: number) => Effect.Effect<void>
}>()("app/UserRepo") {
  static layer = Layer.effect(
    UserRepo,
    Effect.gen(function*() {
      const cache = yield* Cache.make({
        capacity: 10_000,
        timeToLive: "5 minutes",
        // In a real app this would hit a database or HTTP API.
        lookup: (id: number): Effect.Effect<string, string> =>
          id < 0 ? Effect.fail(`invalid id: ${id}`) : Effect.succeed(`user-${id}`)
      })

      return UserRepo.of({
        byId: (id) => Cache.get(cache, id),
        invalidate: (id) => Cache.invalidate(cache, id)
      })
    })
  )
}

refresh differs from invalidate: invalidate removes the entry so the next get recomputes it, while refresh recomputes it now and keeps serving the old value to any concurrent readers until the new one is ready.

Caution

When keys are objects, use a structurally comparable type — a Data class or another Equal/Hash type — so that two keys with the same contents hit the same entry. Plain object literals compare by reference and will miss.

Caching a single effect

When there is no key — you just want to run one effect once and reuse its result — reach for the Effect.cached family instead of building a whole cache. Each returns an effect that, when run, yields a new memoized effect.

import { Effect } from "effect"

const program = Effect.gen(function*() {
  let runs = 0
  const expensive = Effect.sync(() => ++runs)

  // `cached` is an Effect<A, E, R> that reuses its first result.
  const cached = yield* Effect.cached(expensive)

  const a = yield* cached // runs the effect -> 1
  const b = yield* cached // reuses the cached result -> 1
  return { a, b, runs } // { a: 1, b: 1, runs: 1 }
})

Two TTL-aware variants build on this:

• Effect.cachedWithTTL(self, duration) — caches the result for duration, then recomputes on the next evaluation after it expires.
• Effect.cachedInvalidateWithTTL(self, duration) — same, but also returns an invalidate effect so you can clear the cache before it expires.

import { Effect } from "effect"

const program = Effect.gen(function*() {
  const expensive = Effect.sync(() => Math.random())

  // Returns a tuple: the cached effect, plus an effect that clears it.
  const [cached, invalidate] = yield* Effect.cachedInvalidateWithTTL(
    expensive,
    "1 minute"
  )

  const first = yield* cached
  const same = yield* cached // cached: identical to `first`
  yield* invalidate // drop the cached value early
  const fresh = yield* cached // recomputed

  return { first, same, fresh }
})

ScopedCache: caching values that own resources

When a lookup acquires a resource — opens a connection, starts a worker, subscribes to a stream — you cannot simply cache the value and forget about it, because the resource must eventually be released. ScopedCache solves this: each entry owns its own Scope, and that scope is closed when the entry expires, is evicted, is invalidated, or when the cache’s owning scope closes.

import { Effect, ScopedCache } from "effect"

// A pretend connection that logs when opened and closed.
const openConnection = (host: string) =>
  Effect.acquireRelease(
    Effect.as(Effect.log(`open ${host}`), { host }),
    () => Effect.log(`close ${host}`)
  )

const program = Effect.gen(function*() {
  // The cache itself is scoped: it requires a Scope and tears down all
  // remaining entries when that scope closes.
  const cache = yield* ScopedCache.make({
    capacity: 4,
    timeToLive: "1 minute",
    lookup: openConnection
  })

  // First get acquires the connection; the second shares it.
  const a = yield* ScopedCache.get(cache, "db-1")
  const b = yield* ScopedCache.get(cache, "db-1")
  // a and b are the same acquired connection.

  // Invalidating closes that entry's scope, releasing the connection.
  yield* ScopedCache.invalidate(cache, "db-1")
}).pipe(Effect.scoped)

The API mirrors Cache — get, getOption, set, refresh, invalidate, has, keys, and so on — but the lookup’s type is (key: Key) => Effect<A, E, R | Scope>, and the resource lifecycle is managed for you.

Note

Because the cache is created inside a scope, ScopedCache.make requires a Scope in its context. Run the workflow with Effect.scoped (as above) or inside an existing scoped operation. Once the owning scope closes, lookup-style operations on the cache are interrupted.

When not to use Cache

If you need to share a live resource (where identity and cleanup matter more than memoizing a computed value), reach for reference counting instead — see RcMap and RcRef. If you have a single value that should be reloaded periodically, Resource is a better fit.