Start here: what is actually shared?
Share the guest virtual → guest physical page-table memory. Give the CPU and device translation context compatible roots and attributes. Each consumer still has its own identity, permissions, translation caches and maintenance protocol.
This executable lesson uses AMD nested translation, a four-level AMD64 guest table, 4 KiB leaves, two PASIDs and one virtual address. The CPU side is a selected present/write-permission model with a separate semantic nested mapping. It is not a CPU emulator. Intel VT-d and Arm below are comparisons, not alternate backends for this lesson.
Follow one address
| Consumer | Select the guest root | Walk and reach memory |
|---|---|---|
| x86 guest CPU | CR3 root GPA; PCID identifies CPU translation context | GVA → shared guest tables → GPA → CPU nested translation → SPA |
| AMD device | DeviceID → DTE → PASID → GCR3 leaf with the same root GPA | GVA → shared guest tables → GPA → IOMMU host translation → SPA |
Every guest-table fetch also needs its own GPA → SPA translation. Matching the two root GPAs alone is insufficient: the CPU nested mapping and IOMMU host mapping must resolve those table pages to the same backing memory. They must also allow the intended payload access. A working CPU load does not prove that device table-fetch mappings exist.
The lesson starts with CPU root 0x400000, PCID 17, PASID 42, and DeviceID 0x0200. A separate context at root 0x410000 is bound to PASID 43. Both map the same GVA, 0x123456789abc, to different pages. A valid but wrong PASID can therefore succeed at the wrong destination without generating an IOMMU error.
Read the outcome and inspect history
An allowed access is not necessarily the intended access. The workload contract starts with PASID 42, root A and page A. Publishing the lesson's remap changes the intended destination to page B. The named CPU root-B switch changes the intended shared root and destination to B; it does not change the device's actual PASID or binding. Selecting PASID 43 changes only the request, so even an allowed request reaching page B can report Allowed, but different from the expected binding. This contract is stored in the model and each historical result; captions do not infer it from current form controls.
Use Inspect a moment to select a recorded CPU or device request. Roots, caches and bytes come from that snapshot, without issuing another request. Mutation controls pause until Return to live state. Export includes the live state and full history. Invalid completion or duplicate-submission controls show validation feedback while preserving the last applied result, pending operation and history.
Set up a shared x86 root
- Choose the profile. Check guest translation support, address widths, guest paging mode, GLX/PASID limits, request privileges and stage permissions. This example has PCIDE=1, CR0.WP=1, GLX=0, GCR3TRPMode=0 and a 52-bit address limit; it excludes encryption/address-decoration bits.
- Build and map the table pages. Allocate 4 KiB-aligned PML4, PDPT, PD and PT pages. Populate aligned 64-bit entries using the required publication ordering. Map their GPAs into both CPU nested translation and the IOMMU host domain. Map the payload too.
- Select the CPU root. In this profile, stored
CR3=0x400011means root GPA0x400000, PCID17. PCID is a CPU identifier; it is not PASID. CR3 load/invalidation effects depend on CPU mode and instruction options. The lesson's CR3 switch flushes its selected nonglobal CPU context. - Bind the device context. DTE selects the guest CR3 table. Its PASID 42 leaf encodes root
0x400000plus V=1, giving0x400001. Do not copy raw CR3: its low PCID bits are not the GCR3 entry's fields. In AMD Table 26, bits 4:3 are ignored, 11:5 and 2:1 are reserved when valid; upper reserved bits also require checking. Derive the pointer according to the selected CPU/platform profile, then encode the IOMMU structure independently. - Publish and maintain. Use the architecture's ordering, configuration and page-cache maintenance before making a new binding visible to traffic. Submit the relevant commands and observe completion. Updating memory alone does not update a retained context.
- Verify both paths. Compare root, effective request identity, GPA, SPA, permission and bytes. Run a deliberate denial and check that zero bytes are transferred. Verify topology/routing separately; a successful DMA proves neither isolation nor that the expected IOMMU handled it.
For AMD GCR3TRPMode=0, the DTE's guest-CR3-table root is an SPA, while the guest PML4 pointer in the leaf is a GPA. Supported GCR3TRPMode=1 changes how the table root is interpreted. Neither case means that DTE's host page-table root is CPU CR3. AMD §2.2.4 host-table sharing is a separate feature with host-format NextLevel and address-width requirements.
Change a mapping safely
These steps describe responsibilities, not a drop-in driver sequence. Stop new work or use an OS-managed synchronization protocol; keep old pages alive until all required consumers and in-flight requests have completed.
| Change | CPU responsibility | AMD device responsibility | Endpoint responsibility |
|---|---|---|---|
| Shared guest leaf target / permission | Publish entry; invalidate affected CPU translation scope as required | Invalidate affected guest translations, including directory entries if needed | Invalidate matching ATC entries if ATS can retain this translation |
| PASID guest-root binding | CPU CR3 is independent; change it only if intended | Invalidate affected cached guest-root/guest translation information; use the required DTE maintenance for DTE changes | Maintain the corresponding PASID ATC scope |
| IOMMU host mapping, including guest-table metadata | Maintain CPU nested mappings only if they changed too | Use host/nested scope; guest-only GN=1 is not a host invalidation | Account for affected translated endpoint entries |
| IRTE / guest interrupt routing | Follow interrupt synchronization requirements | Interrupt-cache maintenance is independent of guest page translation | A page/ATC invalidation is not an interrupt-cache flush |
The lesson's guest maintenance button represents a completed full guest scope for one DomainID/PASID, including cached guest-root information: INVALIDATE_IOMMU_PAGES, GN=1, PDE=1, S=1, all guest addresses. It does not model a host-cache invalidation. The separate endpoint button represents INVALIDATE_IOTLB_PAGES and its completion. Submission alone does not clear the lesson's cache. “Observe completion” bundles command processing, invalidation and successful completion for this single-request model; use the cache stories for the separate queue stages.
Successful cache entries are retained deterministically to demonstrate a permitted stale outcome, not a guaranteed silicon result. Real implementations can evict or over-invalidate. A cached AMD guest permission denial requires a fresh guest walk; the lesson implements that exception. CPU cached denial and endpoint denial have distinct handling. No PRI request, page allocation or DMA retry happens automatically.
AMD, Intel VT-d and Arm: keep the names separate
| Question | AMD 48882 Rev. 3.11 | Intel VT-d Rev. 5.0 | Arm IHI 0070 H.a |
|---|---|---|---|
| Device / process identity | DeviceID + PASID | Source identity + PASID in scalable mode | StreamID + SubstreamID; PCIe integration maps request identity |
| First-stage root selection | DTE + GCR3 hierarchy | Scalable context + PASID structures; FSPTPTR | STE + Context Descriptor; TTBR fields |
| CPU root counterpart | x86 CR3 | x86 CR3 | Arm TTBR; there is no Arm CR3 |
| Second stage | IOMMU host translation | Second-stage translation | Stage 2, selected through stream configuration |
| Configuration versus page maintenance | DTE / guest-root / page scopes are distinct | Context, PASID and IOTLB scopes are distinct | CFGI invalidates configuration; TLBI maintains translations |
| CPU participation | CPU TLB work is not an AMD IOMMU command | CPU maintenance and VT-d maintenance must be coordinated | Supported/configured PE broadcast TLBI can reach SMMU; otherwise use applicable CMD_TLBI operations |
Intel VT-d supports CPU-compatible first-stage paging structures, but uses its own scalable-mode PASID entry and capabilities. Do not copy an AMD DTE/GCR3 encoding or invalidation command into VT-d. The source for this comparison is VT-d 5.0 §§3.2, 6 and 9; this page runs no Intel model. The glossary has more AMD, Arm and Intel names side by side.
Arm shared stage-1 tables
A PE and SMMUv3 can refer to compatible stage-1 table memory, with independently configured TTBR, address-size, granule, memory-attribute and permission controls. StreamID selects an STE; SubstreamID selects a CD when enabled. Under nested translation, stage-1 descriptor fetches may themselves require stage 2. ASID and VMID scopes must be consistent with the selected configuration.
H.a §3.21 distinguishes explicit command maintenance from PE broadcast maintenance. Broadcast participation depends on support, configuration and shareability; CD.ASET affects matching to shared PE ASIDs. A completed CMD_SYNC after applicable commands is not interchangeable with simply observing that the command queue head moved. Broadcast completion uses the corresponding system synchronization path.
For a valid-to-valid remap, follow the applicable break-before-make rules and advertised capabilities. The AMD demonstration's direct leaf replacement is not an Arm update recipe. Do not assume CFGI_STE/CD alone removes every stale page translation. EVTQ fault records, stalled-transaction resume/termination and PRIQ responses are different paths; a CPU fault is not an EVTQ entry.
PCIe request identity and negative tests
The transaction's actual PASID presence matters. “PASID enabled” in software does not prove a request carries PASID. An untagged request can select a different/default translation path according to configuration; it must not silently inherit the UI's selected process identity. Capture or model Requester ID, PASID-present/value, AT type, requested access and address together.
ATS obtains a translation/permission result; a translated Memory Request uses that grant. PRI is a separate page-request protocol. Neither “ATS enabled” nor a completed PRI response means that a particular DMA was translated successfully. Memory reads are non-posted; writes are posted, so their completion/error evidence differs. Request lengths, byte enables, MPS/MRRS and 4 KiB request boundaries are separate from 4 KiB/2 MiB/1 GiB mapping-base alignment: ordinary byte addresses do not need page alignment.
| Experiment | Expected distinction / evidence | Where to run |
|---|---|---|
| CPU flush only after shared leaf remap | CPU sees B; retained IOMMU translation can still select A | SHARED-01 |
| Revoke write after warming both consumers | Separate CPU fault and device translation failure, each after its own maintenance; no denied payload write | SHARED-02 |
| Switch CR3 but keep old PASID root | CPU and device disagree without necessarily generating an error | SHARED-03 |
| Correct IOMMU, stale endpoint ATC | Translated request can still use A until endpoint maintenance completes | SHARED-04 |
| CPU can fetch tables; IOMMU host mapping missing | Device metadata walk fails before payload transfer | SHARED-05 (cold host path) |
| Read-only guest cache, then promote to R/W | AMD guest denial triggers a rewalk; a CPU holding the old read-only entry takes one #PF, which drops that entry (Intel SDM §4.10.4.1), so the retry succeeds | Manual actions: revoke, read both, restore, write both |
| Wrong valid PASID / missing PASID | Wrong valid context can reach another page; untagged behaviour depends on profile | PASID 43 manual action / identity recipes in Request identity |
| Wrong maintenance PASID | Completion for context 43 cannot be taken as evidence that context 42 was maintained | Submit for 43, switch to 42, observe completion, retry |
| Stale IRTE while page translation is fresh | Page maintenance does not repair interrupt routing | IRQ cache experiments in cache stories |
The PCIe discussion is bounded by the public primary references linked in the negative-test catalogue. This is not an audit of every PCIe revision, TLP format or optional capability.
Read the evidence
The diagram shows published roots, not an assertion that a particular request walked them. The result names the consumer and cache source. Each cache card shows its retained root and mapping version. Memory panels show the actual synthetic bytes at A and B. Exported JSON contains immutable snapshots of completed actions; reading an old snapshot does not perform a new DMA.
Logs here are semantic teaching outcomes, not packed Event/EVTQ/TLP records. CPU errors are not inserted into the IOMMU Event Log. The endpoint path reports denied translation without inventing a PRI request. Raw event and command inspection remains in the engineering workbench.
Limitations: no full CPU exception/error-code model, U/S, NX, SMEP/SMAP, protection keys, global mappings, encryption, five-level walk, hardware A/D updates, CPU nested-table codec, host translation cache, speculative fill, in-flight DMA races or wire-level ATS completion protocol. The working set is one 16-byte transfer at one GVA, two contexts and 4 KiB pages. Other chapters cover variable DMA patterns/sizes, larger mapping pages, interrupt scenarios and detailed queues.
Sources and revision pinning
- AMD IOMMU 48882 Rev. 3.11, April 2026 PDF: §§2.2.1, 2.2.4, 2.2.6.3 (Table 26), 2.2.6.5, 2.2.7.3, 2.4.2–2.4.4 and 2.4.11.
- Arm IHI 0070 H.a, 6 March 2026 PDF: shared address-space/ASID rules, §3.21 update and completion rules, STE/CD definitions and IDR0.BTM. The “latest” URL may later change edition.
- Intel VT-d 5.0, August 2024, D51397-017: §§3.2, 6, 9; comparison only, not claimed to be the newest edition.
- Intel SDM Volume 3A, 253668-060US, September 2016: §§4.5 and 4.10 for the bounded four-level CR3/PCID model. This older pinned baseline does not establish newer CPU feature behaviour.
- Linux x86 shared virtual addressing: OS-level binding, PASID, ATS/PRI and coordinated invalidation context.
- PCIe primary references and request recipes: public PCI-SIG and device-vendor material; see the catalogue's explicit scope labels.