Architecture Tour

Industrial AI needs infrastructure

The model is the least reliable component in the system. The job of the architecture is to make its output grounded, durable, and safe.

A simplified topology

We’ll keep one diagram and reuse it. Each flow just highlights different edges.

Chat turn: request → run → stream

• UI posts a message and attaches to SSE for run_id.
• front starts the run (or resumes), enforces a single in-flight turn per session, and streams events.
• chat-agent executes a durable workflow; the model output is streamed, not buffered.

Pulse → SSE: the UI tails a log

• Stream identity is out-of-band: the SSE URL uses /chat/runs/{run_id}/events.
• Resume is a first-class feature: from_event_id lets the UI reconnect without guessing.
• Typed events: thinking, tool_call, tool_result, artifacts — debug is a trace, not a vibe check.

Tool calls: one interface, many providers

• Tool IDs are canonical: atlas.read.get_time_series, todos.list, gdrive.search.
• tool-registry routes registry-backed tools to the right service (schema + endpoint at runtime).
• goa-ai gives us typed schemas/codecs and a consistent execution contract across providers.

Bounded results: don’t hand the model a firehose

Tool results have to fit inside a context window. We treat “too much data” as an interface bug. The fix is structural: bounded results + refinement hints + UI-only artifacts.

Session: a ledger, not chat history

• session_id is the logical container; conversation_id is a thread; run_id is a single execution.
• Turn locking: if a run is active, front rejects a second turn (turn_in_progress).
• Auditability: tool calls, results, and artifacts are stored so we can answer: “what did the model see?”

Agent-as-tool: specialists without spaghetti

• When it’s an agent: it needs its own prompt + policies + tool access + a multi-step reasoning loop.
• Invocation model: chat-agent calls specialists as tools, creating child runs with linked streams.
• Learning moment: Ada used to take an extra model turn after every tool completion. Now: return tool results directly unless synthesis is required.

Attachments: vision vs RAG

• Vision path (pixels): photos/screenshots/nameplates. We inline images into the model input (subject to limits) and keep evidence linkable.
• RAG path (chunks + citations): manuals/SOPs. We retrieve scoped excerpts through documents and return citations.
• Task runs: images can be inlined at run start; documents are retrieved on-demand via scoped tools.

Storyboard: draft transformation

• User edits the draft in the UI. The model proposes changes; it does not “own” the task definition.
• tasks-designer-agent returns structured edits (think patch/diff), so we can apply, review, and keep intent explicit.
• tasks persists versions and run history — authoring and execution are separate concerns. Smart rule: if the user can’t see what changed, we shouldn’t do it.

Task run: a new thread with its own run_id

• front starts a run via POST /tasks/{task_id}/run and returns {conversation_id, run_id}.
• chat-agent loads/compiles the task definition at run time and executes as a standard tool-using run.
• Progress is streamed: step status updates are events — the UI is not polling.

Write tools: confirmation is an architectural feature

• Explicit approval: the model proposes; the human commits.
• Separation of concerns: atlas-commands owns write semantics and auth boundaries.
• Ledger trail: before/after is recorded alongside the conversation.
• Learning moment: the “building shutdown freakout” was missing operational context. Fix: capture site facts + treat ambiguous states as questions, not emergencies.

Long-term memory: extract facts from runs

• Knowledge Agent runs background workflows to extract stable “Site Facts” from sessions.
• Facts are treated as data: they’re stored, cited, and re-used — not re-invented each turn.
• Compounding effect: future runs start with better context and fewer clarifying turns.

Where we draw boundaries

Demos that land with engineers