Beam Search: The AI Decoding Strategy That Balances Quality and Speed
Beam search is a heuristic decoding algorithm that maintains the top k candidate sequences at each step, trading compute for output quality compared to greedy decoding.
Beam search is a heuristic decoding algorithm used in sequence-generation AI — including machine translation, text generation, and speech recognition. At each step, it maintains the top *k* candidate sequences (where *k* is the "beam width") ranked by cumulative probability, expanding each candidate by one token and pruning back to the top *k*. With beam width 1, beam search degenerates to greedy decoding (always picking the highest-probability next token). Wider beams explore more of the output space and find higher-probability complete sequences, but at linear compute cost. Typical beam widths range from 4–10 for translation tasks. A known limitation: beam search can produce repetitive or generic outputs because it maximizes total sequence probability rather than diversity. Modern large language model inference often combines beam search with temperature sampling, nucleus sampling (top-p), or top-k sampling to trade some optimality for more natural, varied text. **See also:** Context Management Patterns for Long-Running AI Agents