Lesson 9: Apple Silicon & Unified Memory Architecture

Opening: when the CPU and GPU stop competing and start sharing

Lesson 7 and Lesson 8 described the memory hierarchy of a traditional PC: the CPU has its own RAM, the GPU has its own VRAM, and the two are joined by a PCIe bus. Every time the CPU needs the GPU to process data (rendering graphics, training an AI model), that data has to be COPIED from RAM into VRAM over PCIe — a step that costs time while computing nothing, merely "moving data from one place to another". Apple Silicon asked the reverse question: if the CPU and GPU sit on ONE piece of silicon, why not have them SHARE a single pool of RAM?

1. System on a Chip (SoC) vs a traditional motherboard

A traditional PC mounts the CPU, GPU, RAM and drives as SEPARATE COMPONENTS on a motherboard, connected by buses (PCIe, SATA and so on). Apple Silicon (the M series) integrates the CPU, GPU, Neural Engine (NPU) and memory controller onto the SAME piece of silicon — a SoC (System on a Chip). The physical distance between blocks becomes extremely short, cutting latency and energy consumption substantially compared with signals travelling long board traces between separate chips.

A traditional PC (separate components on a motherboard):
  [CPU chip] ---bus--- [discrete GPU chip] ---bus--- [RAM DIMM]
  Signals travel LONG board traces between separate chips

An Apple Silicon SoC (a single piece of silicon):
  +---------------------------------------------------+
  | [CPU: Firestorm x N] [CPU: Icestorm x M] [GPU]    |
  | [Neural Engine (NPU)] [UMA memory controller]     |
  +---------------------------------------------------+
  Physical distance between blocks is VERY SHORT
  -> lower latency and lower energy per transfer

2. Big and little cores (big.LITTLE) & UMA

The Apple M1 has 2 kinds of CPU core: Firestorm (high performance, high power draw) and Icestorm (power efficient, lower performance) — the operating system AUTOMATICALLY pushes background work (syncing email, virus scanning) onto Icestorm to save battery, and heavy work (compiling code, rendering video) onto Firestorm. More importantly for this lesson: the CPU (both core types) and the GPU all access ONE shared pool of LPDDR5 RAM — that is UMA (Unified Memory Architecture).

Traditional PC:
  [CPU] --own RAM--     [GPU] --own VRAM--
     |______________PCIe Gen4 x16 (~32 GB/s)_____________|
     Whenever the GPU needs data the CPU holds -> COPY it over PCIe

Apple Silicon (UMA):
  [CPU] ---+
           +--- ONE shared LPDDR5 pool (e.g. 400 GB/s on M1 Max) ---+
  [GPU] ---+                                                        |
     Both read and write it DIRECTLY - no copy step at all.
     But see Section 4: that pool is SHARED, not private to the GPU.

3. Working the bandwidth numbers: PCIe vs UMA

Rendering ONE 4K frame (3840×2160 pixels at 32-bit colour — 4 bytes per pixel for R/G/B/Alpha) needs exactly:

$$\text{Frame Bytes} = 3840 \times 2160 \times 4 = 33{,}177{,}600 \text{ bytes}$$

Verified for real: a 4K frame weighs exactly 33,177,600 bytes (31.64 MiB). Compare the time to move that data over 2 channels — PCIe Gen 4 x16 (about 32 GB/s in practice, requiring a COPY from RAM into VRAM) against UMA on an Apple M1 Max (400 GB/s of RAM bandwidth, with the GPU accessing it DIRECTLY, no copy):

function frameBytes(width, height, bytesPerPixel) {
  return width * height * bytesPerPixel;
}
function transferTimeSeconds(bytes, bandwidthGBps) {
  return bytes / (bandwidthGBps * 1e9);
}
// Verified: frameBytes(3840, 2160, 4) = 33.177.600 byte
// Verified: transferTimeSeconds(33177600, 32) * 1000  = 1,0368 ms  (PCIe Gen 4 x16)
// Verified: transferTimeSeconds(33177600, 400) * 1000 = 0,0829 ms  (UMA M1 Max)
// UMA beats PCIe by EXACTLY the bandwidth ratio: 400/32 = 12.5x

For this 4K frame, PCIe takes 1.0368 ms and UMA only 0.0829 ms — exactly 12.5 times faster, matching the bandwidth ratio PRECISELY ($400/32=12.5$), rather than being a vague estimate. At 60fps (a per-frame budget of $1000/60 \approx 16.67$ ms), PCIe consumes 6.22% of the budget JUST TO COPY data (before the GPU does any actual work), while UMA takes under 0.5%.

const frameBudgetMs = 1000 / 60; // ~16.6667 ms per frame at 60fps
const pcieShare = (cmp.pcieTimeMs / frameBudgetMs) * 100; // ~6.22%
const umaShare = (cmp.umaTimeMs / frameBudgetMs) * 100;   // ~0.50%
// PCIe: nearly 1/16 of the frame budget spent JUST copying data
// UMA: negligible - the GPU keeps almost the whole budget for real COMPUTE

4. The architectural price of UMA

The three sections above cover only the upside, and the one pitfall raised so far — the RAM not being upgradeable — is a PACKAGING trade-off, not an architectural one. But UMA has two costs that belong to its design itself, and both of them live inside the very 400 GB/s figure Section 3 just used.

4.1 Bandwidth is SHARED, not additive

That 400 GB/s in Section 3 is not "the GPU's bandwidth". It is the bandwidth of the WHOLE pool — the thing the CPU, the GPU and the Neural Engine all draw from, at the same time. When the CPU is working hard, what remains for the GPU shrinks by exactly what the CPU takes:

const bytes = frameBytes(3840, 2160, 4);            // 33,177,600 bytes
const ms = (bw) => transferTimeSeconds(bytes, bw) * 1000;

// UMA: one pool, so whatever the CPU consumes is taken OFF the GPU's share.
// Verified by running it:
//   CPU idle       -> GPU has 400 GB/s -> 0.0829 ms  (the figure from Section 3)
//   CPU 100 GB/s   -> GPU has 300 GB/s -> 0.1106 ms  (1.33x slower)
//   CPU 200 GB/s   -> GPU has 200 GB/s -> 0.1659 ms  (2.00x slower)

// A DISCRETE PC has the opposite property: the two pools are separate, so their
// bandwidths ADD instead of competing. A GPU reading from its own 320 GB/s VRAM
// gets 0.1037 ms no matter how hard the CPU hammers its own RAM.

That is why Section 3's "32 GB/s vs 400 GB/s" comparison is correct but incomplete. It is correct in that PCIe must COPY while UMA need not — and that copy step genuinely disappears, so the benefit is real. It is incomplete in that 400 GB/s is a SHARED figure: under heavy simultaneous load the GPU's actual share is considerably lower, and the 2.00x slowdown on the last row above is a measurement, not a guess. A discrete architecture trades away having to copy in exchange for two independent bandwidth pools; UMA trades the other way round.

4.2 Sharing memory means keeping caches coherent

Section 2 said the CPU and GPU "read and write it DIRECTLY, with no copy step at all". That sentence skips a question: both of them have their own caches (Lesson 7). If the CPU writes a value and that value is still sitting in the CPU's cache rather than in RAM, the GPU reading the same address would see STALE data.

What prevents that is cache coherency: hardware tracks who currently holds a copy of which cache line, and when one side writes, the copies on the other side are invalidated. This is not an optional extra — it is the condition that makes "no copy needed" SAFE rather than merely fast. And it has a price: every write into a shared region can generate extra tracking traffic on chip, and at each CPU↔GPU handover point the software still has to place synchronisation markers to be certain the other side has seen the new data.

5. Hands-on: comparing PCIe against UMA bandwidth

Change the frame dimensions, the bytes per pixel, and the bandwidth of both channels to watch transfer time respond directly — the coloured bars below visualise the ratio:

Summary

• ✅ UMA trades away the PCIe copy step in exchange for one shared RAM pool — but that pool's bandwidth is SHARED: verified, if the CPU is consuming 200 GB/s the same 4K frame takes 0.1659 ms instead of 0.0829 ms, exactly 2.00 times slower.
• ✅ "Zero-copy" removes the COPY step, not the SYNCHRONISATION step — cache coherency is what makes the sharing safe, and the program must still place synchronisation markers between CPU and GPU.
• ✅ An SoC integrates CPU/GPU/RAM onto one piece of silicon, cutting latency and energy — at the cost of losing upgradeability and replaceable parts.
• ✅ big.LITTLE (Firestorm/Icestorm) balances performance against battery through dynamic scheduling.
• ✅ UMA removes the CPU→GPU copy step of the traditional PC model entirely — CPU and GPU access the same RAM pool DIRECTLY.
• ✅ Verified: a 4K frame is 33,177,600 bytes; PCIe Gen 4 (32GB/s) takes 1.0368 ms; UMA (400GB/s) takes 0.0829 ms — exactly 12.5 times faster, matching the bandwidth ratio.
• ✅ Pitfall: the performance gap comes from memory BANDWIDTH, not CPU/GPU clock speed — comparing raw clocks ignores the factor that actually decides it.

Review quiz