神经处理单元：详解

AI and Games is made possible thanks to our premium subscribers who support our work. Subscribe now to have your name in video credits, contribute to future episode topics, watch content in early access, and exclusive supporters-only content.

Subscribe now

If you’d like to work with us on your own games projects, please check out our consulting services provided at AI and Games. To sponsor our content, please visit our dedicated sponsorship page.

In the last few years when buying new PC hardware, both developers and players now contend with a new acronym thrown into the mix.

• We have the CPU, the central processing unit, for computing instructions of game logic

• The graphical processing unit or GPU for handling the graphics rendering - and a little bit of physics,

• And of course RAM for holding the game in dynamic (randomly-)accessible memory.

But now the new thing is NPU, the neural processing unit that is designed to support AI compute workloads. We’ve seen GPUs start to carry dedicated AI compute nodes courtesy of tensor cores which can execute some AI tasks but now NPUs are emerging as standalone blocks on hardware for energy-efficient AI execution. Not just in gaming PCs and laptops, but also in mobile phones, and even the next generation of games consoles from Microsoft and Sony will carry NPUs as part of their hardware makeup.

But what’s the point of all of this? What value is there in having NPUs inside your device? Is it just so a game can ship some Generative AI feature that audiences might not even use? What value does this have for game developers in building their games, and what benefits will players gain from this?

Follow AI and Games on: BlueSky | YouTube | LinkedIn | TikTok

What is an NPU?

Now let’s back things up a little bit and talk about what an NPU actually is. Neural processor units are a specific type of hardware accelerator that speed up performance of AI applications. They’re called Neural processor units given the main use case is running artificial neural networks.

NPUs are built to address a fundamental need for how machine learning AI is computed, and how traditional CPUs are not a good fit for these workloads. CPUs are built to handle sequential computational tasks that run on a single-thread, and while nowadays your typical processor will have a number of individual ‘cores’ with high clock speeds, they’re still built predominantly for handling sequential computational workloads: complex step-by-step logic where we process instructions one after the other. This is perfect of course for the core logic of your game or any other regular software, but things get complicated when you try to do any intensive mathematics such as graphics which is built on linear algebra.

The reason for this is down to how CPUs are built, and how to compute linear algebra at scale. 3D graphics relies on running large numbers of calculations across vectors and matrices, and whenever you do something even as simple as rotate the camera in a game, you’re causing millions of operations to be executed, and given CPUs are not built for large-scale parallel compute, this is where GPUs come into play.

Graphics processors became increasingly popular in PCs in the 1990s thanks to the boom in 3D gaming. The big difference between a GPU and CPU is that they’re built to have thousands of smaller cores to achieve parallel compute, where they run lots of calculations simultaneously, rather than sequentially like on the CPU. This means rendering games at higher resolutions at consistent frame rates is far more attainable. Hence for modern gaming PCs its essential to have a GPU alongside the CPU.

And this brings us to AI and machine learning, notably in that the mathematics behind machine learning, or more specifically training neural networks, shares many similarities with graphics rendering given it’s all about parallel processing of matrices. So in theory, a GPU is a great platform for running AI code. In fact when I started grad school 20 years ago, there was already research in writing shader code to handle neural network processing on GPU. In the decade that followed GPU providers were building software APIs such as Nvidia’s CUDA and AMD’s ROCm to allow for general purpose use of GPUs, making it easier to run AI tasks on the hardware. Plus we then saw the likes of Nvidia’s RTX-20 series in 2018 begin integrating dedicated AI cores directly into the GPU. All of this led to GPU supply being affected in the late 2010s as machine learning engineers and cryptominers started buying them in bulk, and pushing these processes into the hardware. All of this growing interest in using this type of hardware for AI acceleration was ultimately what led to the NPU.

The individual calculation units within an NPU are designed specifically for the execution of AI models, what we refer to as inferencing. While they’re not great for training AI models - given they’re not only built for lower-precision math, but lack the broader ecosystem you would have for training in the cloud - they’re optimised to ensure they run quickly and using less energy when compared to running the same task on a the AI cores of a GPU. This means that while you might still train an AI on a GPU you can free up execution of the AI from the GPU cores when playing the game.

While NPU adoption in consumer hardware is becoming a big talking point now, it’s the end result of over a decade’s worth of developments. The first NPUs that hit the market date back to 2016 when Google’s Tensor Processing Units or TPUs were deployed to cloud compute datacentres. Meanwhile 2017 saw the move into mobile with both the A11 Bionic chip shipped in Apple’s iPhone 8 and Kirin 970 used in the Huawei Mate 10 shipping with integrated NPUs alongside the GPU. Plus AMD was the first company to ship an NPU built into their CPUs with the Ryzen 7040 series back in 2023.

Plus it’s worth mentioning that NPUs appearing in consumer hardware is part of an ongoing trend in edge-based computing, where rather than taking data and sending it to the cloud for processing, the device is better suited to handling that information at the source. NPUs allow for AI compute to be handled more readily on your local device without the issues of latency, security of personal data, and reduces the overall energy consumption.

So hey, now that we have all this AI compute in place? What do we do with it? That’s a bigger question that’s still to be fully explored in games and we’ll dig into that next.

Thanks for reading AI and Games! This post is public so feel free to share it.

Share

What Value Are NPUs for Games?

Okay, so you go ahead and buy a new desktop or laptop that has NPUs built-in, what’s the benefit? Why do gamers - or game developers - even care?

Right now we’re not seeing a lot of NPU usage in games because the space isn’t uniform. With PC, mobile, and console all having different NPU makeups at this time, studios can’t build games designed to ship across multiple platforms in ways that exploit these capabilities. By the time the next generation of Xbox and PlayStation consoles ship, games will be built knowing not only that those devices have NPUs built in, but also that any PCs sold in the years building up to also have this capacity.

The big selling point of NPUs is less about how they will enable new features, but rather like a GPU it offloads workload to another part of the device, thus protecting the CPU and ultimately the game’s frame rate. A lot of existing workloads are becoming AI-driven as new techniques emerge to tackle them, and as such when they’re executed they’ll shift to the NPU where possible.

We’ve already seeing this happen in graphics, with more and more rendering jobs being ran on GPU courtesy of AI techniques. The likes of Nvidia’s DLSS and AMD’s FSR and now the likes of neural texture compression all run on the GPU with courtesy of the AI cores built into the graphics card, and that’s not really going to change: graphics tasks will remain on the GPU, given it makes sense to keep those operations on the same part of the device. While there could be an opportunity for developers to exploit built-in NPUs to help offload some of these tasks, particularly on laptops with limited integrated GPUs, it seems rather unlikely.

Already we’re seeing NPUs being used to run existing features as they capitalise on ML techniques. Communication tools such as voice isolation on your microphone to cut background noise or real-time transcription and captions for accessibility in online chat can run on NPUs. Meanwhile background blurring or image keying for cameras while livestreaming on Discord or StreamLabs can be pumped through the NPU.

At the time of writing, there’s not really any games that are utilising NPUs as effectively as they could in gameplay. But considering we are seeing machine learning being used more frequently in non-graphics tasks in shipped games, there’s an opportunity to start moving that workload. We’re seeing machine learning slowly be adopted in specific aspects of design in a variety of genres, be it autobattle systems for real-time strategy, controlling vehicles in racing games, or handling aspects of decision making for combat NPCs. As games explore running these via neural networks, they could in-turn benefit from performance gains using NPUs. It wouldn’t be too surprising to see these tasks getting pushed to NPUs on future hardware.

Meanwhile we’ve previously talked about motion matching animation which is often trained using AI techniques. Animation processes such as bone and skeleton calculations are traditionally CPU-based tasks, but the more advanced models of motion matching - such as this one published by Ubisoft’s La Forge in 2020 - use neural networks to optimise parts of the animation process as well as lower overall memory overhead. These could run even more efficiently in an NPU. Of course developers need the tools to then maximise this utility and deploy to the NPU effectively.

Of course releasing this article in 2026 we can’t avoid the obvious fact that one area for NPUs to prove useful is for running Generative AI models as we see more and more AI native games hit Steam. AI Native games is the (rather annoying) term used to describe games that adopt Generative AI in gameplay, and there’s a big push for them to use NPUs as the reality is sinking in that using LLMs will seldom work in a lot of games as it currently stands. We’re seeing a trend now as AI native games adopt smaller large language models (dubbed SLMs) that are small enough to deploy locally but often by sacrificing some fidelity. The reason being that it’s simply impractical to run LLMs in the cloud. The token costs - which is the base unit of AI compute - is too expensive and often inconsistent from one playthrough to the next, and the latency kills any opportunity for fluid engagement. By migrating to local models it reduces latency, removes token costs, and perhaps more importantly the corresponding energy costs and carbon footprint as devs don’t need to run AI in some horrid datacentre. The future is small models kids, don’t buy into the big model hype!

Training ML models designed to operate locally on device is the most sensible approach for any realistic future where machine learning is deployed as part of regular video game experiences - and everything else for that matter. That’s where NPUs will see their value come through in the long term as more and more features used to build, play, and enjoy games move towards trained AI models.

AI and Games is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.