Cheat codes for LLM performance: An introduction to speculative decoding

Sometimes two models really are faster than one

Hands on  When it comes to AI inferencing, the faster you can generate a response, the better – and over the past few weeks, we've seen a number of announcements from chip upstarts claiming mind-bogglingly high numbers.…