Skip to content

Per-Node Provider Configuration

This content was migrated from Documentation/PER_NODE_PROVIDER_CONFIGURATION.md and verified against the current codebase.

Purpose

Each node in the Global Supervisor graph can use a different LLM provider and model. This allows cost optimization (cheaper models for classification), performance tuning (faster providers for latency-sensitive nodes), and gradual migration (test new providers on individual nodes before full rollout).

How It Works

Provider Resolution

When an LLM call is made with an agent_type (node name):

  1. Look up the provider column in llm_node_configuration for that node
  2. If set (e.g., "vertex", "azure", "kvant"): use that provider
  3. If NULL: fall back to the global LLM_ROUTING configuration value

Database Schema

The llm_node_configuration table stores per-node settings:

Column Type Description
node_name VARCHAR(100) Primary key — the agent type / node name
provider VARCHAR(50), nullable Per-node provider override. NULL = use global
model VARCHAR(255) Model name for this node
temperature FLOAT Default 0.2
max_output_tokens INT Default 10000
reasoning_enabled BOOL Default false — enables extended thinking for this node
max_reasoning_tokens INT, nullable Default 200 — token budget for thinking when reasoning is enabled
description VARCHAR, nullable Human-readable description of the node's purpose

Valid Providers

  • kvant — Kvant inference (default global provider)
  • vertex — Google Vertex AI (legacy LangChain-based Gemini integration)
  • gemini / vertex-gemini — Google Gemini (native SDK via llm-adapter)
  • claude / vertex-claude / anthropic — Anthropic Claude (native SDK via llm-adapter)
  • azure — Azure OpenAI

Current Production Configuration

Node Provider Model Use Case
intent_classification vertex gemini-2.5-flash-lite Fast, cheap classification
fact_extractor vertex gemini-2.5-flash-lite Fast extraction
preference_extractor vertex gemini-2.5-flash-lite Fast extraction
conversation_summarization kvant inference-deepseek-v32 Summarization quality
title_generation kvant inference-llama4-maverick Creative title generation
All other nodes kvant (global) various Default provider

Configuration Changes

LLM node config is read-only via the Admin API. To change a node's configuration:

-- Switch intent_classification to a different model
UPDATE llm_node_configuration
SET provider = 'vertex',
    model = 'gemini-2.5-flash',
    temperature = 0.1
WHERE node_name = 'intent_classification';

After updating, call the config refresh endpoint to clear the cache:

POST /api/v1/admin/config/refresh

Provider Architecture

The LLM Adapter (SwisperLLMAdapter) supports two provider tiers:

  • Native providers (via llm-adapter package): GeminiProvider, ClaudeProvider, KvantProvider — these support thinking configuration and are the preferred path.
  • Legacy providers (via LegacyProviderBridge): LangChainAzure, LangChainVertex, LangChainKvant — these wrap LangChain integrations for backward compatibility.

Key Code Locations

Path Purpose
swisper/services/llm_node_configuration.py LLMNodeConfigurationService — config lookup with hierarchical resolution
swisper/persistence/models/user.py LLMNodeConfiguration SQLModel
swisper/gateways/llm/adapter.py SwisperLLMAdapter — unified LLM interface with per-node provider selection
swisper/alembic/versions/20260203_add_per_node_provider.py Migration adding provider column

All paths relative to apps/backend/.

See Also