Understanding Encoders
In Semantic Router, an encoder serves two primary purposes:- Convert utterances from routes into embeddings during initialization
- Convert incoming user queries into embeddings during routing
Dense vs. Sparse Encoders
Semantic Router supports two main types of encoders:Dense Encoders
Dense encoders generate embeddings where every dimension has a value, resulting in a “dense” vector. These encoders typically:- Produce fixed-size vectors (e.g., 1536 dimensions for OpenAI’s text-embedding-3-small)
- Capture complex semantic relationships in the text
- Perform well on tasks requiring understanding of context and meaning
Sparse Encoders
Sparse encoders generate embeddings where most dimensions are zero, with only a few dimensions having non-zero values. These encoders typically:- Focus on specific words or tokens in the text
- Excel at keyword matching and term frequency
- Can be more interpretable than dense encoders (non-zero dimensions often correspond to specific words)
Hybrid Approaches
Semantic Router also allows combining both dense and sparse encoders in a hybrid approach through theHybridRouter
. This can leverage the strengths of both encoding methods:
Supported Encoders
Dense Encoders
Encoder | Description | Installation |
---|---|---|
OpenAIEncoder | Uses OpenAI’s text embedding models | pip install -qU semantic-router |
AzureOpenAIEncoder | Uses Azure OpenAI’s text embedding models | pip install -qU semantic-router |
CohereEncoder | Uses Cohere’s text embedding models | pip install -qU semantic-router |
HuggingFaceEncoder | Uses local Hugging Face models | pip install -qU "semantic-router[local]" |
HFEndpointEncoder | Uses Hugging Face Inference API | pip install -qU semantic-router |
FastEmbedEncoder | Uses FastEmbed for local embeddings | pip install -qU "semantic-router[local]" |
MistralEncoder | Uses Mistral’s text embedding models | pip install -qU semantic-router |
GoogleEncoder | Uses Google’s text embedding models | pip install -qU semantic-router |
BedrockEncoder | Uses AWS Bedrock embedding models | pip install -qU semantic-router |
VitEncoder | Vision Transformer for image embeddings | pip install -qU semantic-router |
CLIPEncoder | Uses CLIP for image embeddings | pip install -qU semantic-router |
Sparse Encoders
Encoder | Description | Installation |
---|---|---|
BM25Encoder | Implements BM25 algorithm for sparse embeddings | pip install -qU semantic-router |
TfidfEncoder | Implements TF-IDF for sparse embeddings | pip install -qU semantic-router |
AurelioSparseEncoder | Uses Aurelio’s API for BM25 sparse embeddings | pip install -qU semantic-router |
LocalSparseEncoder | Uses local sentence-transformers SPLADE/CSR models for neural sparse embeddings | pip install -qU "semantic-router[local]" |
- This encoder uses sentence-transformers >=v5’s SparseEncoder API to generate high-dimensional sparse vectors (e.g., SPLADE, CSR).
- No API key required; all computation is local (CPU, CUDA, or MPS).
- You can use any compatible sparse model from the Hugging Face Hub (e.g., naver/splade-v3, mixedbread-ai/mxbai-embed-large-v1, etc.).
Using AutoEncoder
Semantic Router provides anAutoEncoder
class that automatically selects the appropriate encoder based on the specified type:
Considerations for Choosing an Encoder
When selecting an encoder for your application, consider:- Accuracy: Dense encoders typically provide better semantic understanding but may miss exact keyword matches
- Speed: Local encoders are faster but may be less accurate than cloud-based ones
- Cost: Cloud-based encoders (OpenAI, Cohere, Aurelio AI) incur API costs
- Privacy: Local encoders keep data within your environment
- Use case: Hybrid approaches may work best for balanced retrieval