Recently, processor manufacturers have announced new processors for the consumer market called NPUs (Natural Processing Units). These processors are specialized in natural language AI training processing directly on the user's computer.

This move is similar to what happened with GPUs, which were initially incorporated into personal computers for graphics processing in games and are now used for various purposes, including AI training and cryptocurrency mining.

The difference between NPUs and GPUs is that GPUs, despite being specialized processors, are specialized in a type of generic mathematical processing, while NPUs are specialized in an even more specific type of processing. In practice, this means that GPUs can be used for various purposes, whereas NPUs are designed for a specific function. NPUs end up being cheaper to produce and consume less power, which is important in a world where sales of laptops and notebooks to end-users are already higher than those of desktop computers.

NPU processors will form the foundation of AI co-pilots integrated into the operating system, such as the recently launched Windows Co-pilot. The existence of an NPU on the user's computer allows AI models to be trained, fine-tuned, and accessed on the user's computer itself, eliminating the need for the entire AI model to run in the cloud and be accessed via an API.

This is important for scaling AI processing. Today, widely used AI models generate a very high computational cost. An estimate from April 2023 shows that OpenAI's system and its ChatGPT cost $700,000 per day! With AI processing happening partially or even mostly on the user's computer, it will be possible to create much more scalable and powerful AI solutions.

The emergence of NPUs is already a revolution within the AI revolution. But there is another revolution on the horizon: the emergence of WebNN (Web Neural Network API), recently launched by the W3C, still in Candidate Recommendation Draft status.

Just as WebGPU allowed the creation of web applications that accessed GPU resources and enabled more complex graphics games to run in the browser, WebNN will allow web applications to access hardware resources specialized in AI (NPUs), enabling the creation of web apps with real AI applied.

Conclusion

We are still at the beginning of the entire AI revolution. NPUs still need to carve out a market share to become relevant and widely available, and WebNN still needs to evolve into a final adopted standard. But be sure to follow this small but significant revolution within the larger AI revolution.