A single-user personal AI: a two-stage router hands each request to one of nine specialist agents, and every consequential action has to match what the owner actually said.
Role
Designed and built
Where
Personal project
When
2026
Access
Private repo
Problem
A personal assistant that can read notes, send messages, and change data is only useful if it can act, and only safe if nothing it reads can make it act. Web pages, documents, and another agent's output can all carry instructions, so the harness cannot rely on the model to tell data from orders. It also holds the owner's private data, so which model sees a turn matters as much as what the model does with it.
SystemStep 01 / 09
LLM model call
◇ routing gate
HITL human review interrupt
× n parallel fan-out
▭ store / memory
- - - conditional path
01
One owner, one gate
Every page and API route sits behind an owner session; Telegram and WhatsApp answer only the owner's own chat or number, behind the platform's secret header or signature. Input is stripped of control and invisible characters, capped at 4,000 characters, and checked for injection attempts.
02
Two-stage routing
Everyday phrases are answered at once from templates in English, Tamil, or Tanglish. Pattern rules catch obvious intents and follow-ups next, which is fast and free. Anything else goes to a small LLM dispatcher that must return a schema-validated agent name, falling back to the knowledge agent.
03
Nine specialists, each with its own tools
Each agent sees only the tools for its domain. A function-calling loop runs at most six steps with a per-run tool budget, and the answer passes an output guard that redacts keys and blocks prompt leaks before it streams with numbered sources.
04
Private turns stay private
Agent turns hold the owner's data, so the gateway only sends them to providers marked private-safe: Groq with an eight-second timeout, then a local Ollama model. Gemini, NVIDIA, and OpenRouter serve only turns that are allowed to leave that chain.
05
Providers that fail get skipped
Every provider has a circuit breaker: three failures in a row open it, one test call runs after a cool-down that doubles up to ten minutes, and success closes it again. Breaker state is shared across server instances, every change lands in a healing log, and if every provider is down the owner still gets one calm reply in their language.
06
Delegation without escalation
An agent can ask one colleague for help, one level deep. Delegated runs, routine steps, and Autopilot can never call destructive tools, because a colleague's instructions may contain text from the web.
07
The owner's words, not the model's
A consequential tool (delete, message, call, update, web task) only runs when the owner's own words in this turn ask for that action. Each of the 106 tools has a risk level set on the server (47 read, 39 write, 8 send, 12 delete), not chosen by the model.
08
Confirm cards for sends and deletes
A send or delete parks as a confirm card with a one-time ID that expires in ten minutes; the model only learns that approval is waiting. Approving a delete also needs a fresh second factor on the device, so Telegram and WhatsApp hand deletes back to the app. Every approve, decline, or refused token writes an audit row with tool and argument names only.
09
Retrieval and fetching that fail safely
Answers cite numbered sources from a pgvector store, and fall back to multilingual word search, Tamil included, when embeddings are unavailable. Every outbound fetch blocks private addresses, pins DNS, and rechecks each redirect.
Measured
Specialist agents with scoped tools
9
Tools, each with a server-set risk level
106
Maximum steps per tool loop
6
Tests (1,140 unit, 4 end-to-end)
1,144
How it's verified
Harness
Rule router plus an LLM dispatcher into nine scoped specialists; a six-step function-calling loop with one-level delegation and a tool budget, run through an AI gateway that keeps agent turns on private-safe providers and puts a circuit breaker on each one.
Guardrails
Owner-session gate on every route, input and output guards, an owner-intent check before any consequential tool, no destructive tools in delegated runs, server-set risk levels, audited confirm cards for sends and deletes with step-up for deletes, a read-only guest mode, and SSRF-safe fetching.
Testing
1,140 unit tests and 4 Playwright tests, plus an opt-in live suite of 13 tests against real data; CI runs lint, type checks, unit tests, and the build.
In depth
Injection defense that does not trust the model
Prompt rules alone can be talked around. The deciding check is deterministic: a consequential action needs a match in the owner's own message for this turn, so instructions hidden in a web page or a document cannot trigger it. Even then, a send or delete only parks a confirm card for the owner to approve.
Delegation without privilege escalation
One agent can consult another, but only one level deep and never with destructive tools, so a chain of agents cannot launder an action the owner never asked for.
Privacy as a routing rule
Each provider in the gateway catalog is marked private-safe or not, and private turns are filtered to the safe ones before any fallback runs. A faster or free model that might keep prompts is never in the chain for the owner's data.
A cloned voice, with consent
Voice replication is off by default. Enabling it requires recording a sample together with the provider's fixed consent sentence, and the voice ID is stored server-side.
Decisions
01Rules before the LLM router, for latency and free-tier cost.
02Deterministic intent gating and server-set risk levels instead of trusting the model to resist injection.
03Private data only goes to private-safe providers, even when a faster model is available.
04Every model and data dependency has a fallback or a circuit breaker, so the assistant degrades instead of failing.