​
Semantic Router: Technical Architecture

​
System Overview

Semantic Router is built around three core components that work together to enable intelligent routing of inputs based on semantic meaning.

​
Core Components

​
1. Encoders

Encoders transform inputs into vector representations in semantic space.

Types of Encoders:

  • Dense encoders: Generate continuous vectors (OpenAI, HuggingFace, etc.)
  • Sparse encoders: Generate sparse vectors (BM25, TFIDF, AurelioSparse, etc.)
  • Multimodal encoders: Handle images and text (CLIP, ViT)

​
2. Routes

Routes define patterns to match against, with examples of inputs that should trigger them.

Key properties:

  • name: Identifier for the route
  • utterances: Example inputs that should match this route
  • function_schemas: Optional specifications for function calling
  • score_threshold: Minimum similarity score required to match

​
3. Indexing Systems

Indexes store and retrieve route vectors efficiently.

Index types:

  • LocalIndex: In-memory vector storage for dense embeddings
  • HybridLocalIndex: In-memory storage supporting both dense and sparse vectors
  • PineconeIndex/QdrantIndex: Cloud-based vector DBs
  • PostgresIndex: SQL-based vector storage

​
Data Flow

  1. Input Reception: The system receives an input (text, image)
  2. Encoding: The input is transformed into a vector representation
  3. Retrieval: The vector is compared against stored route vectors
  4. Matching: The best matching route is selected based on similarity
  5. Response: The system returns the matched route, enabling appropriate handling

​
Router Types

  • SemanticRouter: Uses dense vector embeddings for semantic matching
  • HybridRouter: Combines both dense and sparse vectors for enhanced accuracy

​
Integration Example

from semantic_router import Route, SemanticRouter
from semantic_router.encoders import OpenAIEncoder

# 1. Define routes
weather_route = Route(name="weather", utterances=["What's the weather like?"])
greeting_route = Route(name="greeting", utterances=["Hello there!", "Hi!"])

# 2. Initialize encoder
encoder = OpenAIEncoder()

# 3. Create router with routes
router = SemanticRouter(encoder=encoder, routes=[weather_route, greeting_route])

# 4. Route an incoming query
result = router("What's the forecast for tomorrow?")
print(result.name)  # "weather"

​
Performance Considerations

  • In-memory vs. Vector DB: Choose based on scale and latency requirements
  • Encoder selection: Balance accuracy vs. speed based on use case
  • Batch processing: Use batch methods for higher throughput
  • Async support: Available for high-concurrency environments and applications relying on heavy network use