DMA journey
Follow one write across the machine
A network card writes 16 bytes to address 0x201000, as the device sees it. The write is posted: the card sends it and waits for no reply. Scroll to follow it into RAM.
The machine
The IOMMU checks every request
A device could write anywhere in RAM. The request crosses the PCIe link to the root port, where the IOMMU gives the device only the memory it may touch and translates its addresses. The IOMMU reads its own tables from RAM. The table at the bottom right shows the packet's header.
Inside the IOMMU
The DeviceID picks one entry
The device address bus:device.function 03:00.0 packs into the 16-bit DeviceID 0x0300. Its Device Table Entry says: Mode 3, three table levels, root pointer 0x7000.
Chapter 4 · first request
Three tables, then the page
The device uses its own address, a DVA (device virtual address). A 4 KiB table holds 512 eight-byte entries, so each table takes 9 bits of the DVA to pick one entry. The last entry names the RAM page, and all 16 bytes arrive. The card's own cache of translations is the ATC (Address Translation Cache); only ATS (Address Translation Services) devices use it, and this write is sent untranslated, so the IOMMU still checks it.
Chapter 5 · permission
The page exists. Is every operation allowed?
A second write, to DVA 0x202000, uses the next entry of the same last table. The page is mapped, but that entry has IW=0 (IW = write allowed, IR = read allowed), so the IOMMU blocks the write and records an IO_PAGE_FAULT event. A read (IR=1) would succeed.
Chapter 6 · recovery
Fixing the table is not enough.
A third write, to DVA 0x203000, finds its entry not present (PR=0), so it is refused too. An IOMMU with NpCache=1 (typically virtual) also caches “not present”. Fix the table, send INVALIDATE_IOMMU_PAGES and INVALIDATE_IOTLB_PAGES, wait for COMPLETION_WAIT, retry. The second command only matters to an ATS card, which may hold an older translation in its ATC. Real AMD hardware typically reports NpCache=0.
Chapter 7 · GVA to SPA
From a program's address to real RAM.
This is a new request. Chapters 4 to 6 had one stage, DVA to SPA. Inside a VM there are two. A program on a PASID-capable device uses a guest virtual address (GVA). The guest page table gives a guest physical address (GPA), and the host table gives the system physical address (SPA). DTE[GV]=1 turns this on.
Chapter 8 · the GCR3 table
The PASID finds the guest's root.
The PASID is a tag saying which program sent the request. A GCR3 table (guest CR3) has 512 entries, enough for PASIDs 0 to 511. PASID 513 needs 10 bits, so the DTE sets GLX=01b: 2 levels, each picking an entry with 9 bits of the PASID. The last entry is the guest page table root.
Chapter 9 · guest walk
Now walk the guest page table.
4 pieces of the GVA pick one entry per level, starting from the GPA the GCR3 table gave us.
Chapter 10 · host walk
Each GPA becomes an SPA.
The same Device Table entry holds the host page table root and Mode. We walk only the final GPA; the same walk runs for every GPA in chapters 8 and 9, which is why nested translation costs many reads. Host table pages are addressed by SPA, so the chain stops here.
Chapter 11 · DRAM
And finally, memory.
With the SPA known, the request leaves the IOMMU for the memory controller, which turns it into DRAM commands. The 16 bytes are written at page 0xC8000, offset 0x000. The whole path: GVA 0x7F001000 (PASID 513) → GPA 0x203000 → SPA 0xC8000 → DRAM.
Chapter 12 · inside DRAM
How the SPA picks a cell.
DRAM is a stack of grids. The controller cuts the SPA into fields: channel, rank, bank group and bank choose one grid; the row is opened into a row buffer (ACTIVATE); the column picks one 64-byte burst of that row (WRITE). This DRAM picture is general background, not from the AMD IOMMU spec: it is one example mapping, and real controllers choose their own and often hash bits to spread the load.