An AI agent that rewrites your documentation — and can't corrupt it

Role: design & implementation, full stack · Surface: in-editor agent, API & streaming layer · Stack: TypeScript · Node.js · LLM tool use

The brief: plain-language requests, workspace-wide edits

The client — a SaaS documentation platform — wanted an agent that lives inside the editor. A user describes a change in plain language: "split this page into a quick-start and a reference," "add a warning callout about the v2 migration," "delete every draft except the API docs." The agent then executes it across the workspace: reading, searching, creating, renaming, restructuring, and deleting documents in a multi-turn conversation.

The catch: the output lands in a production knowledge base that the client's customers read. A hallucinated block or a botched bulk delete isn't a bad demo — it's data loss in someone's product docs. The whole design question was how to give a language model real write access without ever trusting it.

Why this is harder than a chatbot

Content model — the docs aren't markdown

The platform runs a block-based rich-text editor: 22 generable block types — tabbed panes, callouts, column splits, interactive API references, diagrams, embeds — each with strict rules about what may nest inside what. Tabs can't contain tabs. API blocks can't live inside callouts.

Blast radius — destructive by design

The useful version of this agent deletes, renames, and rewrites in bulk. "Confirm every step" would kill the product; so would trusting the model. Safety had to come from the architecture, not from friction.

Reliability — models lie about syntax

In early testing the model reliably produced plausible-but-wrong block markup. The parser would accept it, build a corrupted document tree, and crash the editor on open. Prompting alone never got this to zero.

Environment — live, multi-tenant SaaS

Team-scoped permissions, AI token quotas, co-editors watching the same documents over realtime sync, and users who close the tab mid-run. Every long-running agent loop had to be cancellable, resumable, and tenant-safe.

The shape of the system

Editor client            chat panel in the docs editor
     │ prompt ↓          ↑ typed events, streamed line by line
Session API              run · follow-up · cancel · reset · restore
     │                   auth, schema validation, quota & branch checks
Orchestrator loop        model ⇄ tool dispatcher · ≤ 50 iterations
     │                   cancellation checked between every step
     │                   history persisted as it runs
     │                     [ Postgres — sessions & transcripts ]
     │                     [ Redis — cross-process cancel flags ]
     │                     [ pgvector — semantic search ]
     │ tool calls
11 tools                 read:  list · read · search
     │                   write: create · update · rename · delete · restore
     │                   ux:    progress · navigate  ·  generate_block
     │ every write passes through
Guardrail stack          three validation layers — invalid structure is
     │                   rejected, never repaired silently
     ├─▶ Branch workspace   copy-on-write staging — the agent's only
     │                      writable surface
     └─▶ Main docs          untouched until a human reviews the diff
                            and merges

Full request path: one streaming HTTP response per turn; the model never touches production content directly.

Decision 01 — Own the loop

Agent frameworks bundle the tool-use loop into a black box. That box traded away exactly what production needed: mid-loop cancellation, typed streaming events, transcript persistence, and hard iteration caps. So the loop is explicit application code — the LLM library is used only as a thin model-and-message adapter, and every tool is a hand-written JSON schema rather than a framework abstraction.

Each iteration: check for cancellation, invoke the model, check again (the user may have closed the tab during a 20-second model call), execute the returned tool calls, feed results back, and periodically checkpoint the conversation to the database. A hard cap of 50 iterations bounds the worst case; a cooperative cancel flag in Redis lets any process — the cancel endpoint, or the HTTP connection closing — stop a run cleanly between steps.

Roughly 150 lines of explicit loop bought full control over the three things frameworks make hard: stopping, streaming, and remembering.

Decision 02 — Copy-on-write staging: the agent can't reach production

The agent operates exclusively on a branch — a git-like fork of the documentation workspace. This isn't a convention, it's enforced at the API boundary: a run only starts if the target branch exists, belongs to the caller's team, and is in an editable state.

The first time the agent touches a document that hasn't been branched yet, the write tool transparently creates a branch copy from the point-in-time snapshot the branch was cut from, rewires the document tree to point at the copy, and applies the edit there. Mainline content is never modified — by any code path the agent can invoke.

This moves human review to the workflow level instead of per-action nagging. The user watches changes land live, then reviews the accumulated diff through the platform's existing branch-review flow and merges — or discards the whole branch. The worst case of a fully misbehaving agent is a deleted branch.

Decision 03 — Make invalid content impossible to save

The fix for hallucinated markup was to stop asking the model to write it. For anything beyond simple prose, the model calls a generate_block tool — "a warning callout," "three tabs titled…" — and the server returns the canonical, validated syntax to embed. The model composes documents; the server owns the grammar.

Around that sits a three-layer defense, each layer catching what the previous one can't see:

Layer 1 — validate the request

Before generating anything, the requested nesting is checked against a container→allowed-children ruleset mirrored from the editor's own configuration — including self-nesting bans (no tabs inside tabs). Illegal requests are refused with the list of what is allowed.

Layer 2 — scan the content

Markdown destined for inside a container is scanned for embedded block directives that would violate that container's rules — catching violations the model smuggles in as free text rather than as tool parameters.

Layer 3 — walk the parsed tree

Before any save, the full document is parsed and its node tree walked for structural violations. If any are found, the save is rejected outright — no silent auto-repair — and the agent gets an error explaining how to restructure. A corrupted parse must never be persisted.

Alongside the layers, a round-trip hardening pass absorbs the model's recurring syntax tics — wrong directive punctuation, comma-separated attributes, misnamed parameters — and preserves details like intentional empty paragraphs through the markdown↔document conversion, so cosmetic mistakes get repaired while structural ones get rejected.

Decision 04 — Error messages are prompts

In an agent loop, every error message is read by a language model deciding what to do next. So each rejection is written as instruction, not diagnosis: it names what went wrong, states what's allowed, and — where the mistake has a recognizable intent — redirects toward it.

// tool result returned to the model, verbatim:

Cannot place "callout" inside "callout".
You probably want to REPLACE the existing callout, not nest
one inside it. To modify an existing block, update the
document markdown directly. Do NOT set parentBlockType —
omit it entirely when the block is not being placed inside
a different container.

A validation failure, written for its actual audience.

Tool failures never crash the run — they return to the model as tool results, and the loop self-corrects. In testing, the agent routinely recovered from its own invalid nesting in a single retry, because the error told it exactly where the block should go instead.

Decision 05 — Streaming, sessions, and surviving interruption

Each turn is a single long-lived HTTP response streaming newline-delimited JSON. The first line carries the session ID; after that, typed events drive the editor UI in real time — thinking states, per-document completion, automatic navigation to whatever the agent just changed, and a final summary.

{"sessionId": "k7Jm2…"}
{"type": "agent_status",       "message": "Agent started…"}
{"type": "agent_thinking",     "thought": "Reading document…"}
{"type": "agent_doc_complete", "docName": "Quick Start", …}
{"type": "agent_done",         "summary": "Split the page into…"}

The wire protocol: one JSON event per line, nine event types.

Sessions are durable. Transcripts checkpoint to Postgres during the run and at completion, so a follow-up message rehydrates the full conversation — with two deliberate twists. The stored system prompt is always discarded in favor of the current one, so prompt improvements apply retroactively to every old session. And interrupted histories are repaired on load: a cancelled run can leave tool calls without recorded results, which model APIs reject as malformed — so placeholder results are synthesized before resuming. Users can cancel mid-run, close the tab, come back, and continue the same conversation.

The unglamorous 40%

• Tenant safety everywhere. Every endpoint re-verifies team ownership; write tools re-check the individual user's document permissions — the agent never has more authority than the person driving it.
• Cost control up front. Per-team AI token quotas are enforced before a run starts, and usage is metered per session.
• Forgiving argument parsing. Models put parameters at the wrong nesting level constantly; the dispatcher merges misplaced arguments instead of failing the call.
• Semantic search as a tool. The agent finds relevant pages through vector search over the workspace's embeddings, not by reading everything.
• Bulk-operation brakes. System-generated folders and structural categories are protected from bulk deletes; a restore tool recovers deleted documents, and the branch itself is the undo of last resort.
• Co-editor sync. Every agent write emits the same realtime events as a human edit, so collaborators watching the document see changes appear live.

By the numbers

11	agent tools
22	generable block types
3	validation layers per write
0	writes that can reach production directly
9	typed streaming events
50	hard iteration cap
7	container nesting rulesets
~4.4k	lines of TypeScript

The result is an agent trusted with genuinely destructive operations — bulk restructures, workspace-wide rewrites, mass deletions — because the architecture, not the model, is what guarantees safety: structurally invalid content cannot be saved, production content cannot be touched, and every run is bounded, cancellable, and reviewable before it ships.

---

TypeScript · Node.js / Express · LLM tool use (function calling) · PostgreSQL + pgvector · Redis · NDJSON streaming · block-based rich-text pipeline. Client and product details anonymized. Architecture and figures are drawn from the shipped implementation.