ASAravind Sivakumaran
Theme

Theme and appearance

Appearance
Palette

Pick any colour. The displayed accent adjusts for readable contrast in light or dark mode.

← IOMMU star map

LINUX-STYLE DMA / EMULATED IOMMU

Follow the buffer. Explain the fault.

Use a Linux bounce buffer (SWIOTLB: a copy in memory the device can reach), then inspect the bytes and fault records produced by the emulated IOMMU. No hardware is required.

What claims does this page make? 3 points

Linux DMA experiment

Try your own sequence

Changing a path, size or direction starts a new fixture. The Translation lab supports larger transfers.

DMA handle

    Request result

    Original and bounce memory

    The preview shows up to 32 bytes. Observer panels do not perform CPU or device accesses.

    Original A · PA 0x100002040

    
          
    
          
    
            
    Original B · PA 0x100003040
    
            
    
          
    
          
    
            
    Bounce page 0 · PA 0x2000040
    
            
    
          
    
          
    
            
    Bounce page 1 · PA 0x2001040
    
            
    
          
    
        
    
      
    
      
    
        
    SWIOTLB slots
    
        
    
          The bounce pool is the dmakit library's allocator, the same code the
          C++ headers mirror. Each slot is 2 KiB. The number is the kernel's
          free-run count: how many free slots start here, up to the next used
          slot or segment boundary.
        
    
        
    
        
      
      
      
      
      
        
      Emulated AMD Event log
      
        
      
          These records come from the emulated IOMMU. SWIOTLB alone generates no
          IOMMU events. Four teaching slots are not a hardware allocation. How
          Head and Tail work: next chapter.
        
      
        
      
        
      
        
        
      
        
      
        
        
          Translation cache & evidence 
        
        
        
        
        
        
      
        
      
        
        
        
          
        Reuse it: the dmakit library
        
          
        
            The SWIOTLB slots and this code lab run on dmakit 4.3.0
            (the walker and payload above use the site’s smaller teaching model),
            a small library with the same API in JavaScript and header-only
            C++20. It maps memory, sends DMA, configures MSI and emulates an
            AMD IOMMU that intercepts unmapped DMA and logs the fault, as
            hardware does. It also runs bare, as with
            amd_iommu=off: your code programs the registers and
            tables, faulting DMA writes are posted, so the device is told nothing, and PRI page requests fill a PPR log
            you can overflow on purpose (ppr_overflow.cpp). Devices
            can use ATS with their own translation cache, so a page-out, PRI,
            page-in and retry cycle runs step by step
            (ats_pri_pagein.cpp), also with a PASID through guest
            page tables (pasid_pagein.cpp). Dual PPR logs with
            auto-swap and auto response (ppr_autoswap.cpp), guest
            virtual APIC with the GA log, and performance counters
            (vapic_counters.cpp) are there too, as is host dirty
            tracking with the A and D bits (dirty_tracking.cpp)
            and a VM whose own IOMMU the host runs nested
            (viommu_nested.cpp, explained in the
            virtual IOMMU lab), the hardware-assisted
            vIOMMU that runs each guest's own command buffer
            (viommu_hw.cpp, with its edge cases in
            viommu_hw_edges.cpp and a guest reset from a callback
            in viommu_hw_callbacks.cpp; callbacks that re-enter
            the IOMMU in callbacks_reentry.cpp), table reads that memory
            does not answer and a DeviceID beyond the device table
            (table_errors.cpp), and SEV-SNP with
            Secure ATS (snp_secure_ats.cpp, explained in
            secure DMA).
            Both languages print identical output, checked by the library's tests,
            including 1,580 random parity scenarios.
          
        
        
        
        
        
          Pick an example and edit any file. Run in this browser uses the
          example's JavaScript twin and sends nothing. Run project externally
          compiles your edited C++ on
          Compiler Explorer, an external service. Use only public, non-confidential code.
        
        
        
        
        
        
        
        
        
        
          Download single-file .cpp
        
        
        
        
        
          Ready. No code has been sent.
        
        
        
        Program output
        
        
        Run the example to see actual output here.
        
        
        
          Download dmakit 4.3.0 (.tar)
          Original single-file example
          README and API
        
        
        
        
          Locally, inside the unpacked folder:
          g++ -std=c++20 -I cpp/include cpp/examples/iommu_fault.cpp,
          or node examples/iommu_fault.mjs for the JavaScript twin.
        
        
      
        
      
        
        
        Start with one buffer
        

        Choose Bounce buffers + emulated IOMMU faults and press Take the next step. Watch three separate things: where the device writes, when the CPU sees those bytes, and whether a fault record fits in the Event log. This runs entirely in your browser; it needs no IOMMU hardware and does not execute a Linux kernel.
        

        The model combines a Linux-style buffer lifecycle with a synthetic AMD translation and Event-log machinery. It is an independent implementation of selected behaviour, not Linux code compiled to JavaScript.
        

        What SWIOTLB contributes
        

        SWIOTLB gives a device an accessible temporary buffer and uses CPU copies to move data between it and the original. Drivers still use the DMA API. In this fixture, the original is above 4 GiB and the bounce region is below 4 GiB. Mapping initializes the bounce data; receive/bidirectional data is copied back at the applicable CPU sync or unmap. Transmit/bidirectional CPU edits need the corresponding sync for the device.
        

        SWIOTLB alone does not block malicious DMA or produce IOMMU Event/PPR/GA records. Its name does not mean that it is a hardware translation-cache emulator. A SWIOTLB-only wrong-direction transaction can still touch accessible bounce memory. The lesson reports the API violation separately from the hardware outcome.
        

        Where the synthetic faults come from
        

        The combined profile maps the bounce memory through the emulated AMD IOMMU. Linux's DMA-IOMMU implementation can also use SWIOTLB for selected buffers; this site’s combination makes that layering visible without requiring hardware. It does not reproduce every condition that selects bouncing in a real kernel.
        

        


        
Observation
Producer in this lesson
What to do next

        
        

        
Mapping allocation or DMA-mask failure
DMA API layer; no descriptor submitted
Check dma_mapping_error(); recover or fail the operation

        

        
Wrong-direction access through the emulated IOMMU
Selected page permission check; synthetic IO_PAGE_FAULT
Inspect identity, address and requested access; correct the mapping or request

        

        
DMA after completed strict unmap
Emulated not-present translation failure
Fix descriptor lifetime; do not reuse an unmapped handle

        

        
Event log full
Event-buffer state machine; incoming record lost
Preserve/drain records, observe Run=0, disable, clear overflow and restart logging

        

        
Wrong unmap parameters or premature unmap
Teaching API guard
Fix API lifetime/ownership; this guard is not an IOMMU fault

        

        

        The Event record uses the 16-byte AMD encoder and buffer model: four raw 32-bit words, DeviceID, DomainID, fault address, present and permission evidence. No PASID is attached to these ordinary host-domain DMA requests. Record production, notification acknowledgment, consumption and repair are separate operations. A lost record never turns a denied DMA into an allowed transfer.
        

        PPR requires its own page-request protocol; GA requires guest interrupt-remapping activity. Neither is invented for an ordinary bad DMA. Continue to Command & event buffers, the next chapter, then Interrupts & GA log; command halt and restart cases are in Faults & recovery.
        

        Direction is from the device's perspective
        

        


        
DMA API direction
Intended device operation
Selected IOMMU permission

        
        

        
DMA_TO_DEVICE
Device reads memory, such as NIC transmit
IOMMU_READ

        

        
DMA_FROM_DEVICE
Device writes memory, such as NIC receive
IOMMU_WRITE

        

        
DMA_BIDIRECTIONAL
Both
Read + Write

        

        

        Use the returned dma_addr_t in the descriptor. A CPU virtual pointer, a physical address and a DMA address are different concepts even when two happen to be numerically equal. The API also takes the device's DMA mask into account. This fixture uses an identity bus-address mapping only in its direct and SWIOTLB-only profiles.
        

        The short lifecycle
        

          

        1. Prepare the original CPU buffer and choose the direction.
          2. 

        2. Map it and check the result. The mapping publishes device access before a descriptor is submitted.
          3. 

        3. Submit the returned DMA address and let the device access it.
          4. 

        4. Observe device completion. This is a descriptor/driver event, not a fabricated PCIe Completion TLP for every posted write.
          5. 

        5. If retaining a streaming mapping, sync for CPU before CPU access and sync for device before returning ownership. In the bounce profile, these steps expose the actual copies.
          6. 

        6. Unmap with the original device, handle, size and direction after the device has finished using the buffer.
          7. 

        

        A teaching guard catches API misuse here so the visitor can recover. Real Linux may diagnose misuse through DMA-API debugging, or the bug may cause corruption; the guard is not a claim that all such calls fail safely in the kernel. Memory inspection panels are observers and do not represent unsynchronized CPU accesses.
        

        Optional: strict versus deferred invalidation
        

        The IOMMU profiles reuse this site’s host walker and payload-copy engine. Strict unmap performs page removal and modelled invalidation completion before releasing the IOVA. The deferred profile retains the IOVA allocation until its flush completes. Its software mapping can be gone while a retained hardware translation still exists. The deliberate Inject read after unmap action demonstrates that misuse; normal drivers must stop using a mapping before unmap.
        

        The two-slot allocator makes delayed reuse visible: a second mapping uses another slot while the first is retired. Flushing permits reuse. Once an IOVA is legitimately reused for another buffer, an old descriptor using that same address can reach the new mapping: invalidation does not replace descriptor-lifetime discipline. This is not the Linux allocator's exact placement, queue size, timeout or scheduling policy. Hardware cache retention and over-invalidation are also implementation-dependent.
        

        Unmapping DMA is separate from interrupt-cache maintenance and guest CR3/TTBR maintenance. Those remain in the cache experiments and shared guest-table lesson.
        

        Keep the scope small
        

        The default experience needs only Map, device access, completion, sync and unmap. Advanced actions expose the negative cases. Each experiment resets to a deterministic fixture. JSON export preserves the source profile and action snapshots for comparison.
        

        The data buffer starts at offset 64 inside a 4 KiB page. Choose 1–4032 transfer bytes and a repeating pattern of 1–32 hex bytes. Allocation and protection are page-granular, not a promise of sub-page isolation. The memory preview shows up to 32 bytes; exported transfer evidence is bounded by the engine's 64-byte preview. The model moves the requested number of bytes.
        

        Omitted: a running kernel, a kernel driver, full SWIOTLB pool/slot/area allocation, scatter-gather merging, noncoherent cache maintenance, DMA attributes, confidential-computing encryption, endpoint ATC, PASID/SVA binding, topology/group attachment and concurrent DMA. The combined profile uses strict invalidation. No server configuration, boot parameter or kernel module needs changing.
        

        Sources
        

        The implementation is pinned to Linux commit 80a4e3ad8daba66915a9bdf0fcae5831cc8dbd5f; its Makefile identifies 7.3.0-rc2. This is a reviewed reference snapshot, not a requirement to install that kernel. The source links below point to the exact commit; the general documentation links can change over time.
        


        Copy accounting and controlled contrasts
        

        The visible counters separate device requests and payload bytes from CPU copies: initialization, sync-to-device, sync copy-back and unmap cleanup. Every copy records its source, destination, actor and size. A 16-byte receive followed by sync-for-CPU and ordinary unmap performs 16 initialization bytes, 16 sync copy-back bytes and another 16 cleanup-copy bytes. Equal bytes do not mean no copy occurred. IOMMU-only and direct examples report zero bounce-copy bytes; that does not mean the CPU or OS did no work.
        

        LINUX-09 is the positive 64-bit direct-address contrast to the 32-bit mapping failure. LINUX-10 shows that invalidating a translation does not copy bounce bytes into the original buffer.
        

        Fixture addresses: original PA 0x100002040, first bounce PA 0x2000040, first IOVA 0x800040. One active mapping and two teaching allocation slots. Injected allocation failure and deferred-IOVA exhaustion are executable; a concurrent real SWIOTLB pool allocator is outside scope.
        

        
        Exact Linux source references