HITL System — Architecture¶

This content was migrated from Documentation/SWISPER_HUMAN_IN_THE_LOOP_ARCHITECTURE.md and restructured into audience sections. Review for accuracy against the current codebase.

Context and Purpose¶

The HITL System exists because AI agents sometimes need human input to proceed safely and correctly. Without it, agents would either guess (risking errors) or fail silently (losing user trust).

The driving requirements are:

• Agents never talk directly to users — All user-facing questions go through a centralized handler for consistent UX
• Indefinite pause/resume — The system must survive server restarts and arbitrarily long user pauses via LangGraph checkpoint persistence
• Multi-turn support — Complex tasks may require multiple clarification rounds before completion
• Blocking vs non-blocking — Entity disambiguation must distinguish between cases that block the answer and cases that are incidental

Architecture Overview¶

flowchart TB
    subgraph Triggers["HITL Triggers"]
        DA[Domain Agent] -->|WAITING_FOR_INPUT| AE[Agent Execution Node]
        ER[Entity Resolution] -->|ambiguity detected| DB[Disambiguation Blocking]
    end

    subgraph Orchestration["HITL Orchestration"]
        AE -->|user_in_the_loop.is_waiting| UIR[UI Router]
        DB -->|streams question| UIR
        UIR --> HT[HITL Text Node]
        HT -->|streams question to frontend| HANDLER[HITL Handler]
        HANDLER -->|interrupt\(\)| PAUSE([Graph Pauses])
        PAUSE -->|checkpoint to Redis| WAIT[Wait for User]
    end

    subgraph Resume["Resume Flow"]
        WAIT -->|user responds| CMD["Command(resume=answer)"]
        CMD --> HANDLER2[HITL Handler]
        HANDLER2 -->|escaping?| NEW[Process New Intent]
        HANDLER2 -->|answer received| ROUTE{Resume Route}
        ROUTE -->|agent HITL| AE2[Agent Execution]
        ROUTE -->|disambiguation| DR[Disambiguation Resolution]
    end

    subgraph Resolution["Disambiguation Resolution"]
        DR -->|resolve entity| FACTS[Persist Pending Facts]
        DR -->|create new entity| CNE[Create New Entity Node]
        FACTS --> CONTINUE[Continue Graph]
        CNE --> CONTINUE
    end

Flow summary: A trigger (agent or disambiguation) creates a UserInTheLoop payload with is_waiting=True. The HITL Handler calls interrupt() to pause the graph and checkpoint state. When the user responds, Command(resume=answer) restores execution. The handler routes to either the original agent or disambiguation resolution based on the interrupt source.

Component Responsibilities¶

Data Model¶

UserInTheLoop (Pydantic Model)¶

State Persistence¶

Key Design Decisions¶

1. LangGraph interrupt() Over Custom Polling¶

• Chosen: Native LangGraph interrupt() with Command(resume=...) for pause/resume
• Rejected: Custom polling loop, WebSocket-based waiting, database flag checking
• Rationale: interrupt() integrates directly with LangGraph's checkpoint system, providing automatic state persistence, deterministic resume, and multi-turn support without custom infrastructure

2. Centralized Handler Over Per-Agent Questions¶

• Chosen: All HITL requests flow through a single handler node that formats and delivers questions
• Rejected: Each agent formats its own user-facing questions
• Rationale: Consistent UX across all agents and channels. Single point for audit logging, A/B testing of question formats, and channel-specific adaptation (text vs voice)

3. Blocking vs Non-Blocking Disambiguation¶

• Chosen: Two separate flows — blocking (ask first) and non-blocking (answer first, ask "by the way")
• Rejected: Always blocking; always non-blocking; let the LLM decide
• Rationale: Blocking every disambiguation adds unnecessary latency for incidental mentions. Non-blocking every time risks incorrect answers when entity identity matters. The entity resolution node sets a relevance field ("blocking" vs "non_blocking") based on whether the entity affects the answer

Interfaces and Contracts¶

Known Trade-offs and Debt¶