Agent Replay Lab, a local-first VCR for TypeScript agent tool I/O by Jarvis Osorio

Agent Replay Lab is a local-first TypeScript VCR by Jarvis Osorio for tool I/O in TypeScript agents: record once, mock-replay tools offline, and CI-gate structural drift. The GitHub repository is agent-replay-lab.

View the GitHub repository

The current version is 0.1.0, released under the Apache License 2.0. It is written in TypeScript and requires Node.js 22+ and pnpm. The packages are not on the npm registry yet — install from the GitHub repository.

It is local-first (SQLite and portable JSON). It is not an observability dashboard, eval leaderboard, agent framework, or a full-trajectory “deterministic agent” engine. It is not an MCP server, and there is no MCP adapter in v0.1. Offline mock stubs tools only; it does not replay LLM decisions, agent state machines, clock or other nondeterministic inputs. Live-tools and live-model modes are planned for 0.2, not shipped in v0.1.

Why

Observability products answer what happened, and usually re-run live. Agent Replay Lab answers a narrower question: does this agent still take the same tool path (name, order, args and responses) against the same recorded tool world, offline, in CI? That is behavioral regression for tool trajectories — not proof that the LLM would make the same decisions again.

Modes in v0.1

You can optionally record model.requested / model.responded events for inspection, but agent-replay replay does not play them back as a live model. Mock replay is not deterministic LLM verification.

Install

Clone the repository. After pnpm build, the 5-minute proof is the basic-agent demo:

git clone https://github.com/JarvisOsorio/agent-replay-lab.git
cd agent-replay-lab
pnpm install
pnpm build
pnpm test

# 5-minute proof
pnpm --filter @agent-replay-lab/example-basic-agent demo

Prefer the monorepo clone until the packages are published. An advanced path is to pnpm pack in packages/core and file-install into another project; the README describes the extra rewrite step for the CLI workspace dependency.

Capture seam (wrapTool)

import { Replay, ReplayStore, wrapTool } from "@agent-replay-lab/core";

const store = new ReplayStore(".agent-replay/store.sqlite");
store.ensureProject("demo", "demo");

const replay = new Replay({ store, projectId: "demo" });
const getWeather = wrapTool(replay, "getWeather", async (args: { city: string }) => {
  return { tempC: 21, city: args.city };
});

await getWeather.execute({ city: "Berlin" });
replay.complete();

Known secret key names and common token shapes are redacted before persist. This is a best-effort first layer, not a PII or governance product. The README suggests pairing it with agentscrub for after-the-fact store remediation.

CLI

The command is agent-replay, from @agent-replay-lab/cli. Offline by default. Human and --json output are supported. import remaps the cassette into the local project id so list, replay and test work immediately (the integrity hash is still verified against the file).

pnpm --filter @agent-replay-lab/cli build
pnpm --filter @agent-replay-lab/cli exec agent-replay init my-agent
pnpm --filter @agent-replay-lab/cli exec agent-replay list
pnpm --filter @agent-replay-lab/cli exec agent-replay inspect <runId>
pnpm --filter @agent-replay-lab/cli exec agent-replay replay <runId>      # offline mock
pnpm --filter @agent-replay-lab/cli exec agent-replay diff <runA> <runB>
pnpm --filter @agent-replay-lab/cli exec agent-replay fixture create <runId> golden
pnpm --filter @agent-replay-lab/cli exec agent-replay test golden --json
pnpm --filter @agent-replay-lab/cli exec agent-replay test golden --run <candidateRunId>
pnpm --filter @agent-replay-lab/cli exec agent-replay export ./trace.json
pnpm --filter @agent-replay-lab/cli exec agent-replay import ./trace.json
pnpm --filter @agent-replay-lab/cli exec agent-replay doctor

fixture create pins tool sequence, args and responses. test golden --run <otherRun> fails (exit 1) on structural drift — the same bar as diff.

CI exit codes (test)

Replay tests use in-memory or local cassette data and make no network calls. The offline guarantee is enforced by the mock engine (fail closed, no HTTP client).

Packages

Not in v0.1: MCP adapter, OpenTelemetry adapter, flexible matching, semantic compare, dashboard, live auto-replay, telemetry, full LLM/state/clock cassette replay, and behavioral policy or “agent assurance” enforcement.

Normalization is strict: object keys are sorted for canonical JSON; volatile fields such as requestId, timestamps, traceId and spanId are dropped; misses and argument mismatches are errors, never silent skips.

Source and license

• Repository: github.com/JarvisOsorio/agent-replay-lab
• Release: v0.1.0
• License: Apache-2.0
• Author: Jarvis Osorio, technology entrepreneur and AI innovator, based in Miami Beach, Florida
• Project site: jarvisosorio.com/agent-replay-lab/
• All open-source tools: Open-source AI tools by Jarvis Osorio
• Also by Jarvis Osorio: mcp-pordego, an open-source MCP security scanner; agentscrub, a local-first CLI for agent session secrets and PII; and Ambient Memory, a local-first MCP memory layer

Identity: Jarvis Osorio · Jarvis Nelson Osorio · AI work · Official profiles

Updated: November 15, 2026 (America/Santiago)