Get the latest tech news
Character Prefix Conditioning
A clever algorithm for more accurate code completion sampling.
← Back to the main blog January 6, 2025Character Prefix Conditioning We want to sample a sequence of tokens s=t1,t2,…,tns = t_1, t_2, \ldots, t_n from a distribution specified by an autoregressive model p(s)p(s) given by Can you construct an efficient algorithm for sampling from q(tk∣t1,…,tk−1)q(t_k | t_1, \ldots, t_{k - 1}), that minimizes calls to the original language model?
Or read this on Hacker News