Intermediate LevelAI, LLMs, Embeddings & MCP
How does Context-Free Grammar (CFG) decoding enforce valid JSON and TypeScript output from LLMs?
❓ Interview Prompt: Explain Context-Free Grammars (GBNF), logit masking during token generation, and syntax guarantees.
Prompting LLMs to output JSON often results in syntax errors (missing commas, unclosed brackets). Grammar-Guided Decoding (CFG Decoding) enforces structural rules at the token sampling level.
How Logit Masking Works:
1
The engine parses a target syntax rule (e.g. GBNF grammar or JSON Schema) into a Finite State Machine (FSM).
2
Before sampling the next token, the engine identifies which vocabulary tokens maintain valid grammar state.
3
Sets logits of invalid tokens to $-\infty$ (masking them completely).
4
Guarantees: Model can physically only sample tokens that maintain 100% valid syntax!
Code Implementation Example
typescript
// Conceptual Logit Masking for JSON Grammar
function applyGrammarMask(logits: number[], allowedTokenIds: Set<number>): number[] {
return logits.map((score, tokenId) =>
allowedTokenIds.has(tokenId) ? score : -Infinity
);
}💡Logit masking suppressing invalid syntax tokens during decoding.
🎯 Interview Key Takeaway
Grammar decoding masks invalid vocabulary tokens at each step, guaranteeing 100% syntactically valid JSON/code generation.
#Grammar Decoding#CFG#Logit Masking#JSON#Intermediate
Back to all AI, LLMs, Embeddings & MCP Questions→