Neural Processing Unit

• Context (BS): Apple introduced the M4 chip with the iPad Pro recently. The most notable change in the M4 is the addition of a 16-core Neural Engine, also known as a Neural Processing Unit (NPU).

What is NPU (Neural Processing Unit)?

• NPU is a specialised processor explicitly designed for executing machine learning algorithms.

How is NPU different from a Central processing unit (CPU)?

• CPUs use sequential computing, processing one instruction at a time, while NPUs employ parallel computing to execute multiple calculations simultaneously.
• NPUs are faster and more efficient compared to CPUs due to their parallel computing approach.

How is NPU different from a Graphics processing unit (GPU)?

• GPUs also have parallel computing capabilities, but they are primarily used for graphic rendering and resolution upscaling.
• NPUs consume less power and are more efficient than GPUs for AI-related tasks.

Significance: Shift towards smaller AI models

• Large language models (LLMs) are typically too large to run on a device, so big tech companies have started releasing small language models that can run entirely on-device.
  • For example, Google’s Gemma, Microsoft’s Phi-3, and Apple’s OpenELM.
• The NPUs play a crucial role in deploying on-device AI applications.
  • On-device AI models reduce the need for cloud processing and enable AI applications to run directly on hardware.