FAQ: AI Hardware for Notebooks and PCs
AMD, Apple, Intel, Nvidia, and Qualcomm promote special AI features in their chips. Apple and Microsoft also highlight AI features in their operating systems, like Apple Intelligence and Copilot+. This article aims to clarify the often confusing world of AI technology.
What Do AI Accelerators Do?
Why are AI accelerators better than normal CPU cores? Many AI algorithms require high computational power but mainly rely on matrix multiplication and addition, known as Matrix Multiply Accumulate (MMA). Specialized units for these tasks process data faster and more efficiently than general-purpose CPU cores. However, this efficiency only occurs under certain conditions.
Compatibility of AI Apps and Accelerators
Can any AI software use any AI accelerator? No, AI software must be specifically programmed for a particular AI accelerator. Chips from AMD, Apple, Intel, Nvidia, and Qualcomm are not binary compatible with each other. Some chips even contain multiple AI units, complicating compatibility. While drivers and APIs exist to facilitate integration, not all AI apps work with every AI unit.
AI Programming Interfaces
How can I find out which AI software best utilizes my hardware? This is challenging, as many software companies do not disclose which API their AI app uses. Even with this information, performance can vary greatly depending on the combination of AI frameworks, APIs, drivers, and hardware.
What Are Tops?
The term “Tops” stands for Tera-Operations per Second. It’s a measure of how many operations an AI unit can perform in a second. Many AI apps use quantized data, simplifying values to improve performance. For example, they use Int8 values instead of FP32, allowing more data to fit in RAM and caches. Tops measure the processing of Int8 values, often reaching into trillions.
Types of AI Accelerators
What types of AI accelerators are in current Windows and macOS computers? Most x86 and ARM processors now have optimized CPU cores for AI algorithms. They process AI data formats like BF16, FP16, and Int8 faster than older processors. Integrated GPUs in these processors also support AI data formats. Additionally, separate AI units called Neural Processing Units (NPUs) are included for efficient AI processing.
How Much RAM Does an AI PC Need?
Does an AI PC require a lot of RAM? It depends. Locally executed AI models can use more RAM than typical applications. Microsoft requires at least 16 GB RAM for Windows 11 PCs with the Copilot+ logo. Apple has also increased its minimum RAM requirement to 16 GB for devices supporting Apple Intelligence.
Upgrading AI Accelerators
Can I upgrade my notebook or PC with an AI accelerator? Desktop PCs with a free PCI-Express x16 slot can install modern graphics cards, enhancing AI performance. The choice of card depends on budget and power supply capacity. Nvidia RTX cards are recommended for their AI performance and support. Few notebooks and mini-PCs have slots for graphics cards, but M.2 form factor AI accelerators like the Hailo 8L are available for under 100 euros.