Abrham Bunaro

AgentOps Lite

PrototypeSoloPublic code2026

Open-source tracing and evaluation for LLM agents. A Python SDK sends traces, a dashboard shows runs, latency, tokens and cost, and eval runs compare models side by side.

AgentOps Lite overview: KPI strip, runs per day with a release marker, latest failures, and per-agent and per-model tables.
The overview, running on the built-in demo data.

The problem

When an agent gets slower, more expensive or wrong, a team needs to see which run, which step and which model, without wiring up a paid platform first.

How it works

Two services on one PostgreSQL database. A FastAPI service ingests nested spans in idempotent batches and serves analytics; a Next.js dashboard reads the same API the SDK writes to, so authorization lives in one place.

Prisma owns the schema. A drift test applies the migrations to a fresh database and checks every column the API maps, and the web app's types are generated from the API's OpenAPI file.

What I worked on

  • The Python SDK, the FastAPI service, the Next.js dashboard and the schema.
  • Server-side scorers and the eval comparison view.
  • A demo mode with deterministic synthetic data, so it runs without a paid API key.
A trace detail view: span waterfall with a failing tool call selected in the inspector.
One run as a span waterfall, with the failing tool call open.
Three models compared on five scorers, with a table of exact values.
Three eval runs compared scorer by scorer.

Security notes

  • Passwords are hashed with argon2id. Session tokens and API keys are random 256-bit values stored only as SHA-256 hashes.
  • Every query is scoped to a project, and a dedicated test suite checks that one project cannot read another's data.

Where it stands

MIT-licensed and runnable with one Docker Compose command.

Next project

Aurora

Back to all work