IOMMU / INTERRUPTS & NEGATIVE TESTING

Follow the interrupt. Inspect the evidence.

Inject a fault, watch the relevant buffer, and inspect the guest’s pending interrupt in memory.

What claims does this page make? 3 points

Spec rule: the IRTE routing order, IRR byte arithmetic, 8-byte GA_GUEST_NR record and GA log restart steps come from AMD 48882 rev 3.11 (§2.2.5.2, §2.7.1 to §2.7.4), and the stall and PRI rules from Arm IHI 0070 H.a.
Platform assumption: whether a cached entry is still present before invalidation is up to the hardware, and the AVIC/EOI behaviour is defined by the AMD64 APM, not by this page.
Teaching model: the eight-slot rings, the always-retained caches and the two-NIC shared region are synthetic simplifications.

From device completion to guest notification

A device often writes data with DMA, then sends an MSI or MSI-X to tell software that work is ready. Although MSI uses a memory-write transaction, the interrupt-remapping path checks the interrupt request and its device identity; it is not an ordinary payload DMA page-table walk. An interrupt is also not proof that the intended payload reached protected memory. Validate the destination, bytes and ordering separately.

MSI-X, x2APIC and a 128-bit IRTE describe different things. MSI-X is a device interrupt mechanism. x2APIC changes APIC addressing and delivery facilities. A 128-bit IRTE is an IOMMU table format selected with guest-APIC enablement; it can represent either host or guest delivery. GuestMode chooses which interpretation applies. The Raw-records workbench includes the packed formats and x2APIC capability gates.

The idea: think of a mail room. The IOMMU first checks who sent the note. Then it checks whether the recipient is the host or a guest. For a guest it drops the note in their box (the backing page). If the guest is in, it rings the doorbell; if the guest is out, it leaves a message at the front office (the GA log).

Dashed arrows are paths this page explains but the lab does not run. Try each branch in the Guest APIC lab.

1 · Identify & validate

Requester DeviceID → DTE interrupt policy → table/index bounds → IRTE format and RemapEn.

2 · Select destination

GuestMode=0: host APIC delivery. GuestMode=1: atomically set the vector’s virtual IRR bit in the backing page.

3 · Notify if appropriate

IsRun=1: guest APIC doorbell. IsRun=0 and GALogIntr=1: GA_GUEST_NR, if logging can accept it.

A rejected remapping request belongs in the Event log when that event is not suppressed. A successfully recorded interrupt for a non-running guest belongs in the GA log. A peripheral page request belongs in the PPR log. These records have different purposes and are not interchangeable evidence. See all three logs side by side.

Guest APIC backing page, virtual IRR, ISR and EOI

The idea: think of the backing page as a message board at a hotel front desk, one board per guest. The IOMMU is the caller who pins a sticky note (the IRR bit) on it. The guest’s CPU takes the note off the board and starts the call (ISR). When the call ends, the guest tells the desk “done” (EOI). Two calls with the same vector while the note is still pinned leave one note, not two.

Try the buttons in any order, including sending vector 81 twice.

Solid is modelled by the simulator. Dashed is what the processor does; the page explains it but does not execute it. Try it in the Guest APIC lab, or follow the experiments below.

The IRTE points to a 4 KiB-aligned virtual APIC backing page in system physical memory. In this fixture its base is 0x100000. The virtual interrupt request register (IRR) holds pending vectors: eight 32-bit words at offsets 0x200, 0x210, …, 0x270 (48882 §2.2.5.2 gives the address; the layout is the APIC’s, from AMD64 APM vol. 2). The gaps reflect the APIC register layout; this is not a contiguous 32-byte bitmap.

For vector v, the containing byte is base + 0x200 + floor(v/32) × 16 + floor((v mod 32)/8); set bit v mod 8. Vector 81 therefore sets bit 1 of byte 0x100222. The full backing page below is real mutable simulator memory. Only IRR is modelled; zero-filled bytes elsewhere do not claim complete APIC register behaviour.

Repeated arrival of the same vector leaves one pending IRR bit; the bit is not an interrupt counter. CPU acceptance, priority arbitration, in-service state (ISR) and guest EOI belong to the processor/AVIC lifecycle. They are explained here but are not executed. Neither consuming a GA entry nor acknowledging GAInt clears IRR or performs EOI. Resetting the fixture clears its memory only as a test reset.

The IRTE’s vector and backing root can remain cached. Publish a new value, send an interrupt, and inspect the old page before invalidating. In this model, IsRun, Destination, GATag, GALogIntr and GAPPIDis are reread live as specified in 48882 §2.2.5.2. Updating the backing IRTE is distinct from completing interrupt-table invalidation.

GA log format, notification, overflow and restart

GA_GUEST_NR means the target guest was not running. Its 8-byte record contains GATag in bits 31:0, DeviceID in 47:32, reserved zero in 59:48 and opcode 1 in 63:60. GATag is assigned by software to correlate the target; it is not the vector, a PASID or a backing-page pointer. For DeviceID 0x0200 and GATag 42, the raw value is 0x100002000000002a, with little-endian bytes 2a 00 00 00 00 02 00 10.

The IOMMU writes at Tail, advances Tail and maintains a shadow Tail in memory. Software reads at Head, processes the record and advances Head. Equal pointers mean empty; one unused slot distinguishes full from empty. Numeric Head > Tail can be valid after wraparound. Do not move Head past unread records or write the hardware-owned Tail.

GAEn, the logger’s enable/run state and GALogIntr gate logging. A written record sets GAInt; GAIntEn gates the host notification interrupt. The guest APIC doorbell and this host log notification are different signals. This fixture exposes a semantic notification flag, not physical MSI timing, coalescing, shadow-Tail coherency or interrupt-controller behaviour.

Fill seven entries, then send an eighth interrupt: the teaching ring overflows without overwriting unread records. The virtual IRR remains pending even when its GA record cannot be retained. Preserve/drain evidence, observe Run=0, disable logging, clear overflow and restart (§2.7.4). Consuming records alone does not restart a stopped engine; lost records are not replayed. Real GA buffers are 4 KiB-aligned, in power-of-two sizes from 4 to 64 KiB (512–8192 entries); the eight slots here are deliberately reduced for experimentation.

The model follows the rev. 3.11 Status[GALOverflow] rule and does not manufacture an additional IRQ from overflow alone.

Experiments worth trying

Send an MSI with IsRun clear. Inspect IRR and the GA entry. Consume GA, then acknowledge notification: IRR remains set. Run it in the Guest APIC lab · Walk through it
Publish IsRun=1 without invalidating. Send again: the pending bit remains and the semantic doorbell replaces GA logging. Run it in the Guest APIC lab · Walk through it
Change vector 81 to 82 and publish. Send before invalidation, then invalidate and send again. Compare both pending bits and cache/backing values. Run it in the Guest APIC lab · Walk through it
Clear GALogIntr with IsRun=0. Publish and send: IRR is set with no new GA record. Absence of a record is not proof of no interrupt. Run it in the Guest APIC lab · Walk through it
Reset, clear RemapEn, publish and send: expect an Event-log fault, with no new IRR bit or GA record. A PASID-prefixed MSI is another explicit rejection. Run it in the Guest APIC lab · Walk through it
Fill GA, Event A/B or PPR A/B without consuming. Observe overflow and retained records. Recovery of a log is separate from recovery of its underlying work. Run it on the buffers · Walk through it

Sources: AMD 48882-PUB Rev. 3.11 §§2.2.5.1–2.2.5.3, 2.7.1–2.7.4, Tables 23/76 and Status register; Arm IHI 0070 H.a §§3.5, 3.12–3.13, 7.3. Complete AVIC/EOI behaviour requires AMD64 APM Volume 2, Chapter 15 and is outside this simulator. Source versions and coverage.

Three logs: where the IOMMU writes down what happened

The idea: picture an office with three in-trays. The Event log holds incident reports: something was blocked or broken. The PPR log holds requests: a device asks for a page to be made available. The GA log holds missed-call slips: an interrupt arrived for a guest that was out. Most traffic goes in no tray at all.

Each outcome runs on the same model as the labs below, so the lit path and the record name come from the model.

Swipe or scroll sideways to see every column.

	Event log	PPR log	GA log
In one line	Something was blocked or broken.	A device asks for a page.	A guest’s interrupt arrived while it was not running.
Written when	A DMA or interrupt request is rejected, a table entry is malformed, or a command fails.	A PRI-capable device sends a page request message. Not for every DMA fault.	IsRun=0 and GALogIntr=1 in the interrupt’s remapping entry, with GA logging enabled.
Example records	`IO_PAGE_FAULT`, `ILLEGAL_DEV_TABLE_ENTRY`, `ILLEGAL_COMMAND_ERROR`	`PAGE_SERVICE_REQUEST`	`GA_GUEST_NR`
Entry size	16 bytes	16 bytes	8 bytes
Software then	Diagnoses, fixes the cause, and decides whether to retry.	Services the request (or refuses it), then sends `COMPLETE_PPR_REQUEST`.	Uses the GATag to find and schedule the guest. The vector is already pending in IRR.
Host notification	EventLogInt, if EventIntEn	PprInt, if PprIntEn	GAInt, if GAIntEn

Four rules that hold for all three

No record is not proof of success. A suppressed fault (SA=1) and an interrupt with GALogIntr=0 both leave no log record.
Each log is a ring. The IOMMU writes at Tail; software reads at Head and moves it on. Watch the rings.
A full log stops, it does not overwrite. Unread records are kept, new ones are lost, and software must clear the overflow bit and re-enable logging. (Optional Event/PPR autoswap and PPR auto-response change this.) Fill a log and watch it stop.
Reading a record fixes nothing. Consuming an Event record does not repair the mapping; consuming a GA record does not clear IRR. Try it in the Guest APIC lab.

Record names and sizes: AMD reference: commands, events, PPR and GA.

Guest APIC: make pending state visible

Inputs describe the initial applied IRTE.

Try the valid → invalid → valid IRTE sequence

For an IRTE cache test: send a valid MSI first, set reserved bit 3 in backing, then send without invalidation. The retained valid entry is used in this fixture. Complete IRTE invalidation and send again: the reserved-field check fails into Event A/B. Publish the valid form and send once more to recover. Invalid entries are not installed in this model’s interrupt cache; do not assume hardware must negatively cache a failed lookup. RemapEn and vector changes provide additional permutations.

Inspect applied IRTE, cached entry and all backing pages

Inspect the selected 4 KiB backing page

Allocated backing page

Warm the DMA cache, then corrupt backing

A valid DMA first establishes retained DTE and page-translation state. Editing a memory-resident DTE/PTE does not itself invalidate the IOMMU’s cached copy. This experiment retains successful entries deterministically so the stale-state path is reproducible. Real cache residency is not guaranteed: hardware may evict/refetch, so a pre-invalidation pass requires evidence of a hit, and an early fault does not by itself prove incorrect hardware.

Send a valid write to warm both caches.
Publish one bad backing field: DTE reserved bit, guest PTE R/W=0, host PTE IW=0 or reserved bit.
Send again before maintenance: the retained valid state still permits the request.
Submit the appropriate invalidation. Observe that submission alone is not completion.
Complete maintenance and retry: inspect the refetch, denied bytes and Event record. For R/W or IW=0, a read remains a positive control.
Repair backing and follow the required synchronization protocol before reuse. A cached guest permission denial triggers the required fresh guest walk under AMD §2.2.7.3; permission promotion is not the mirror of revocation. This model does not install failed lookups as negative entries.

Backing editMaintenance scope

Inspect DMA cache, backing and attempts

Wrong-scope controls matter. DTE-only maintenance leaves the page cache retained; page-only maintenance leaves a cached DTE retained. AMD INVALIDATE_DEVTAB_ENTRY (§2.4.2) does not invalidate translation-cache entries. Page maintenance must select the correct DomainID/PASID/address and guest/host scope. For an ATS-enabled endpoint, its ATC is another cache requiring the appropriate device invalidation and completion; this particular experiment is untranslated DMA and has no ATC. The controls represent completed semantic scopes, not packed command execution.

Two PCIe NICs, one shared memory region

NIC A (DeviceID 0x0200) may read the region. NIC B (0x0208) may write it. Both map the same system physical bytes using separate synthetic protection contexts. Write a pattern through B, then read through A and compare the actual returned bytes. Try the reversed permissions to prove the test checks the requester as well as the address.

NIC B write pattern (1–32 hex bytes)

Current shared RAM bytes:

Inspect both NIC identities and transferred bytes

The descriptor here is 16 bytes; the supplied pattern repeats or is truncated to that length. Both NICs use the same mutable RAM object, so this is a data-handoff test, not two separately initialized screenshots. Each access is serialized explicitly. It does not emulate concurrent races, PCIe ordering/relaxed ordering, CPU caches or DMA API ownership fences. On hardware, establish writer completion and visibility/ownership before permitting the reader; an interrupt alone is not that proof. A shared page intentionally permits communication and does not demonstrate isolation between these two clients.

DMA faults and command failures

DMA fixtures execute a 16-byte synthetic transfer or ATS assessment. Event suppression and ATS responses can produce no Event record. Use the Raw-records workbench for variable descriptor sizes/patterns and the request classifier for observed PASID identity.

Publish is not fetch; fetch is not completion. Illegal command/fetch failures halt this serial fixture and route diagnostics to Event A/B. A missing device invalidation response leaves completion outstanding. The command buffer has no overflow status bit: publishing over unread commands is rejected as software misuse.

Inspect records and recover the logger

Ready. Choose an experiment above.

Inspect the latest action result

Buffer to inspect

Current retained entries; empty slots do not represent consumed history
Slot	Record / command

AMD Event/PPR use one engine with two autoswap banks: seven usable slots per bank here. Clearing/consuming bank A is distinct from bank B. Arm queues use producer/consumer wrap bits and all eight slots. Use the Arm controls below for their different recovery rules. Records are semantic except the explicitly decoded 8-byte GA payload.

Decode retained GA records to raw little-endian bytes

Inspect all engine state and the action history

Arm SMMUv3: separate queues and interrupt architecture

Arm CMDQ accepts maintenance and response commands; EVTQ reports SMMU events; PRIQ carries peripheral PRI requests. A command error uses CMDQ_CONS.ERR (a CERROR_* code) and the GERROR/GERRORN handshake—not an AMD ILLEGAL_COMMAND_ERROR record. A stalled transaction uses StreamID/STAG and CMD_RESUME; a PRI response uses its request group and still requires an endpoint retry.

AMD’s Guest APIC backing page and GA_GUEST_NR are not Arm SMMUv3 structures. On Arm systems, the GIC/ITS and virtual interrupt facilities handle interrupt delivery. SMMU translation, device MSI routing and GIC virtualization must be analysed together, but this SMMU model does not emulate a GIC/ITS or claim a one-to-one GA equivalent.

This Arm profile is Non-secure, 48-bit output, 4 KiB granule and one CD per stream. Full PRIQ admission is a software guard here, not complete architectural overflow/auto-response emulation. EVTQ full behaviour distinguishes retained stalled requests from lost terminated diagnostics. No physical DMA, MSI or CPU scheduling is performed.

IOMMU star map · 112 negative-test recipes · All AMD command operations