Beginner LevelAI, LLMs, Embeddings & MCP
What is a Semantic Router and how does it route user prompts to specialized pipelines?
❓ Interview Prompt: Explain vector-based prompt classification, fast intent routing, and avoiding expensive LLM classifier calls.
A Semantic Router classifies and routes incoming user prompts to specific execution handlers based on embedding similarity rather than calling an expensive, slow LLM classifier.
How It Works:
1
Pre-embed example queries for specific route destinations (e.g. "billing", "technical support", "chitchat").
2
Embed incoming user query.
3
Compute cosine similarity against pre-embedded route centroids.
4
If similarity exceeds a threshold (e.g. $> 0.82$), route instantly in sub-10ms without an LLM call!
Code Implementation Example
typescript
// Fast Semantic Router
async function routeQuery(userQuery: string): Promise<'billing' | 'tech_support' | 'general'> {
const queryVector = await embedText(userQuery);
const billingScore = cosineSimilarity(queryVector, billingRouteCentroid);
const techScore = cosineSimilarity(queryVector, techRouteCentroid);
if (billingScore > 0.80) return 'billing';
if (techScore > 0.80) return 'tech_support';
return 'general';
}💡Sub-10ms query classification using semantic vector distance routing.
🎯 Interview Key Takeaway
Semantic routers classify intent via fast vector similarity search, bypassing slow LLM prompt classification.
#Semantic Router#Classification#Routing#Latency#Beginner
Back to all AI, LLMs, Embeddings & MCP Questions→