Neural Processing Unit

Context (BS): Apple introduced the M4 chip with the iPad Pro recently. The most notable change in the M4 is the addition of a 16-core Neural Engine, also known as a Neural Processing Unit (NPU).

What is NPU (Neural Processing Unit)?

NPU is a specialised processor explicitly designed for executing machine learning algorithms.

CPUs use sequential computing, processing one instruction at a time, while NPUs employ parallel computing to execute multiple calculations simultaneously.
NPUs are faster and more efficient compared to CPUs due to their parallel computing approach.

GPUs also have parallel computing capabilities, but they are primarily used for graphic rendering and resolution upscaling.
NPUs consume less power and are more efficient than GPUs for AI-related tasks.

Large language models (LLMs) are typically too large to run on a device, so big tech companies have started releasing small language models that can run entirely on-device.
- For example, Google’s Gemma, Microsoft’s Phi-3, and Apple’s OpenELM.
The NPUs play a crucial role in deploying on-device AI applications.
- On-device AI models reduce the need for cloud processing and enable AI applications to run directly on hardware.

Neural networks are a series of algorithms that mimic the operations of a human brain to recognise relationships between vast amounts of data.