Capstone Phase 2: Orchestration and State Routing

Graph Orchestration and State Management

In the previous phase, we established tool schemas and executed a single-loop agent with access to all tools. A common question arises when designing these systems: if the goal of using an agent is to let the language model autonomously decide which tools to call, why do we need to restrict its choices with a state machine?

The answer lies in the distinction between global routing and local autonomy. Letting an agent autonomously choose from a massive pool of global tools leads to selection degradation, high latency, and security issues. In a production system, we do not eliminate the agent's autonomy; instead, we scope it. Under a routing graph (or state machine), the system manages the high-level transitions deterministically, while leaving the model fully autonomous within each specialized node.

For example, when routing a customer query to a database specialist node, the model still autonomously decides how to write the SQL query, what filters to apply, and how to retry if a query fails. However, because it is restricted to this node, it cannot accidentally execute a policy lookup or call a write-restricted refund tool. This hybrid architecture combines the flexibility of natural language reasoning with the strict safety and performance boundaries required by software engineering.

Our orchestrated support agent decomposes the execution path into a structured sequence of states:

1. State: Triage - A classifier analyzes the user's intent to route the conversation to the correct specialists.
2. State: Specialist Node - Execution is routed to a specialized assistant with a focused prompt and access only to the tools relevant to its domain.
3. State: Response Compilation - A final node synthesizes the retrieved findings into a clean response, resetting the system to idle.

State Routing Graph Architecture

The state diagram for our orchestrated support agent is illustrated below:

• Triage Node: Evaluates user intent and returns a category classification.
• SQL Tool Node: Triggered if customer/order lookup is required.
• RAG Tool Node: Triggered if policy documentation lookup is required.
• Convergence: All paths merge into the final output compilation step.

Interactive Playground: State Routing Loop

The playground's multi-turn classification and state execution trace is shown below:

The playground below implements this state routing graph. We define separate specialist agents and route the incoming customer query based on the triage classification result.

By decoupling triage from execution, we ensure that:

• The SQL specialist prompt remains focused purely on relational schema translation.
• The Policy specialist prompt is focused solely on vector document retrieval.
• No excess tools are exposed to nodes that do not require them, increasing classification safety and reducing context overhead.

The next phase covers Self-Correction and Safety Gates, where we add write validation rules and LLM critique loops.