AgentOps Lite
Open-source tracing and evaluation for LLM agents. A Python SDK sends traces, a dashboard shows runs, latency, tokens and cost, and eval runs compare models side by side.

The problem
When an agent gets slower, more expensive or wrong, a team needs to see which run, which step and which model, without wiring up a paid platform first.
How it works
Two services on one PostgreSQL database. A FastAPI service ingests nested spans in idempotent batches and serves analytics; a Next.js dashboard reads the same API the SDK writes to, so authorization lives in one place.
Prisma owns the schema. A drift test applies the migrations to a fresh database and checks every column the API maps, and the web app's types are generated from the API's OpenAPI file.
What I worked on
- The Python SDK, the FastAPI service, the Next.js dashboard and the schema.
- Server-side scorers and the eval comparison view.
- A demo mode with deterministic synthetic data, so it runs without a paid API key.


Security notes
- Passwords are hashed with argon2id. Session tokens and API keys are random 256-bit values stored only as SHA-256 hashes.
- Every query is scoped to a project, and a dedicated test suite checks that one project cannot read another's data.
Where it stands
MIT-licensed and runnable with one Docker Compose command.