Detailed notes for this lesson are being written. They appear here together with the video.
Video coming soonThis lesson’s video is being made
Start with the notes below; the video appears here as soon as it is on YouTube.
Generative AI Engineering · Lesson 66 of 90
Latency optimisation
end-to-end latencytime to first tokentime per output token (TPOT)prefill vs decodeprompt length and latencyparallel callsstreaming for perceived speedspeculative decoding (introduction)region choice