What is an NPU?

If you look at any major tech company in the last few years you may have noticed a new buzz word, AI. It’s quickly picking up a lot of steam, being integrated into nearly every product, and is already a part of our daily lives. Especially with AI features being added to smartphones, we’re seeing a rise in NPUs, or Neural Processing Units, to help speed up these complicated tasks around photo manipulation and AI assistants. Now with Intel and AMD adding dedicated NPUs into their mobile chips, we’re seeing their benefits come to laptops and maybe even desktops in the future. To understand the impact of this trend, let’s break it down even further.

A Neural Processing Unit (NPU) is an umbrella term that refers to any processor that specializes in the types of computation used in neural networks, the most common type of algorithm used in AI and machine learning programs. At the end of the day, all processers are made to do math, the difference is in how they’re designed to handle different functions or workloads. Similar to how a GPU’s functions can be run on a CPU (but isn’t recommended), an NPU’s core functions are rather simple as well. But by designing the chip for a specific task, whether that be for graphics processing or neural network functions, it can run at an efficiency several magnitudes faster. This specialization helps drastically cut down power consumption while increasing speed, both important factors on mobile devices.

As the use of AI becomes more and more common in our smartphones and other devices, the benefits of NPUs will become more and more obvious. Even though we’re only in the early days of AI, we’re already seeing it used all the time. Any sort of audio or video input likely goes through some sort of AI processing. Videos are stabilized, colors are improved, and vocals are boosted. AI assistants like Siri now process voice commands locally on your phone before deciding if it needs the internet.

With the advent the AMD’s Ryzen 8040 series and Intel’s Core Ultra series we’re starting to see how AI can be used as an everyday tool. The same benefits to power and speed that we see on smartphones are going to apply to laptops as well. This time the most common use will likely be in video calls where AI can be used for a background blur, a higher resolution output, or a clear voice. And with Microsoft introducing copilot for Windows, we’ll see quicker responses with less power consumption as we transition to AI as regular part of the PC experience.