Get the latest tech news
Demystifying NPUs: Questions and Answers
Let's answer some 'what', 'why' and 'hows' for the Neural Processing Unit.
Most well-known and popular Large Language Models (LLMs) like ChatGPT and its competitors use hardware in the cloud to perform inference and don’t (yet) use NPUs on your PC or smartphone. It has done this through the development of both a high-quality software ecosystem (centered on CUDA) and through the refinement of its hardware, including adding specialized tensor matrix multiplication units to its GPUs. However, in some ways, Qualcomm has ‘stolen a march’ on the x86 based PC incumbents, Intel and AMD, with its Snapdragon X Elite SoCs, which are used in the latest Surface laptops from Microsoft itself.
Or read this on Hacker News
