A LangGraph multi-agent system that turns plain-English data requests into validated Trino SQL, with four human-in-the-loop interrupts and resumable state.
Role
Architect and lead engineer
Where
Zinnov (Draup)
When
Sep 2025 to present
Access
Work · private repo
Problem
Analysts spent hours turning business questions into SQL and Excel reports. Each request meant finding the right tables in a large warehouse, resolving company, location, and vertical names across millions of records, writing the query, and checking it by hand. A single LLM prompt could not do this reliably: it guessed table names, merged different companies, and gave no way for a person to step in when it was unsure.
SystemStep 01 / 09
LLM model call
◇ routing gate
HITL human review interrupt
× n parallel fan-out
▭ store / memory
- - - conditional path
↺ retry loop
01
A request in plain English
Requests arrive from a CLI, a FastAPI service, or a batch CSV. Attachments (CSV, Excel, PDF) are parsed and personal data is masked before any model sees the text.
02
Intent, with a confidence gate
The intent agent extracts the entities and metrics being asked for. When its confidence is below the threshold, the graph interrupts and asks a person to clarify; the checkpointer saves state so the run resumes exactly where it stopped.
03
A manager agent plans the work
The manager agent reads a corpus of agent and tool specs and writes a typed task plan: which subgraphs to run and in what order. Plans are cached by request type, so repeat requests skip the planning call.
04
Entities fan out by type
The normalize subgraph splits the request into companies, locations, and verticals and routes each to its own normalizer. Companies are batched and fanned out per chunk, each resolved with hybrid FAISS + BM25 retrieval over 2.6M company records.
05
Ambiguity goes to a person
Results are merged and checked for ambiguous matches. A name that could be two different companies interrupts the graph for a human choice instead of silently picking one.
06
The schema comes from a knowledge graph
A schema knowledge graph of tables, fields, join paths, and semantic tags is loaded once per process. An LLM selects the relevant schema, the table selector resolves tables and join paths, and a DDL snippet is built for the SQL step. No match means a human interrupt, not a guess.
07
SQL is generated, then distrusted
The SQL generator writes Trino SQL with the canonical names injected, using a query cache keyed by SQL hash. A validator allows SELECT only and rejects blocked keywords, and a dry run confirms every table exists before anything executes.
08
Invalid SQL is reviewed or retried
If validation fails, the graph interrupts for review and can loop back to the generator with the error, instead of returning a broken query.
09
A typed result, in under a minute
The assemble step returns a typed result with status (success, partial, or failed), the deliverable queries, and which steps completed, failed, or were skipped. Analyst turnaround went from hours to under a minute.
Measured
Turnaround, down from hours
< 1 min
Company records in hybrid retrieval
2.6M
Human-in-the-loop interrupt points
4
Subgraphs under one manager graph
3
Tests across 31 test files
281
FastAPI endpoints
4
How it's verified
Harness
LangGraph manager graph over three subgraphs, typed state with merge reducers, a SQLite checkpointer for interrupt and resume, and per-stage model routing.
Evals
An evaluation framework is one of the ten core components; per-agent call statistics are saved for every session.
Guardrails
Personal data is masked before any model call; SQL is SELECT-only with a keyword blocklist and a dry run; four human-in-the-loop interrupts cover the expensive failure modes.
Testing
281 tests across 31 test files.
In depth
State you can stop and resume
Every stage reads and writes one typed graph state, and every stage result is a Pydantic v2 model, so a malformed output fails at the boundary instead of three steps later.
·Typed state with merge reducers, so parallel branches combine cleanly
·A SQLite checkpointer persists state across interrupt and resume cycles
·Each run records completed, failed, and skipped steps in the final result
Four places a person can step in
Human review sits where a wrong answer is expensive, not everywhere.
·Intent: confidence below threshold
·Normalize: an entity matches more than one record
·Schema: no schema matches the request
·SQL: the generated query fails validation
Batch mode for bulk requests
The same graph runs over a CSV of requests. Batch mode turns off every interrupt; an error is recorded, the row is marked failed, and processing continues with the next row.
·One failing request never stops the batch
·A batch abort skips the remaining steps for that row
Memory, caching, and routing
Repeated work is cached at three levels, and each stage can use a different model.
·Plan memory keyed by request type
·Query memory keyed by SQL hash
·The schema knowledge graph loads once per process
·Per-stage model routing for intent, schema, and SQL generation
·Per-agent call statistics saved per session
Read-only by construction
The SQL path is built so a bad query cannot change data: SELECT-only validation, a keyword blocklist, and a dry run that checks table existence. The deliverable is the validated query set, marked as dry-run.
Decisions
01A manager agent over three subgraphs instead of one prompt, so normalization, schema selection, and SQL generation are each tested and evaluated on their own.
02Hybrid FAISS + BM25 for entity names: semantic search handles paraphrase, BM25 keeps exact names and acronyms from drifting.
03Interrupts plus a checkpointer instead of full autonomy: a person resolves the four expensive failure modes and the run resumes without starting over.
04Batch mode trades interactivity for throughput: no interrupts, failures recorded per row.
Stack
LangGraph · Claude API · FastAPI · Trino · FAISS · BM25 · Pydantic v2 · SQLite checkpointer · pytest