Arm SMMUv3: what changes

Start with the correct architecture

Read in the H.a spec§3.3.1 Stream table lookup p. 46§5.2 STE, Stream Table Entry p. 250§5.4 CD, Context Descriptor p. 299

SMMU is Arm’s IOMMU architecture. “IOMMU” names the general device address-translation and protection function. This page compares AMD IOMMU with Arm SMMUv3, rather than treating SMMU and IOMMU as different categories.

This comparison targets Arm SMMUv3. SMMUv2 uses a different register and context-bank interface; its programming model must not be mixed into an SMMUv3 queue-based implementation. The Translation lab is AMD-style. The independent Arm workbench executes a bounded semantic STE/CD, translation and queue model. This reference describes both that subset and the wider architecture.

Swipe or scroll horizontally to see every column.

PCIe integration commonly derives StreamID from Requester ID and SubstreamID from PASID. A platform can place multiple PCIe hierarchies in a larger StreamID namespace, so do not hard-code “every StreamID is a 16-bit BDF”. Firmware/platform mappings and implemented ID widths matter. A process ID also does not replace the device identity.

An STE selects the stream’s configuration and stage-2 parameters and locates stage-1 context descriptors where used. A CD supplies stage-1 translation parameters, roots and ASID. Rejecting a bad STE, rejecting a bad CD and failing a page walk are different diagnoses.

STE and CD, bit by bit

Read in the H.a spec§5.2 STE, Stream Table Entry p. 250§5.4 CD, Context Descriptor p. 299

Both are 64 bytes: eight 64-bit little-endian words. The tables give each field’s word and bits as the Linux driver defines them (arm-smmu-v3.h, v6.18), and the site’s codec (architecture/arm/records.mjs) packs and unpacks exactly these. H.a widens the address fields (S1ContextPtr, TTB0, S2TTB, L2Ptr) to bit 55; Linux v6.18 and these tables stop at bit 51. Fields Linux does not define are left out rather than guessed: STE MemAttr, ALLOCCFG, NSCFG, PRIVCFG, INSTCFG, S2AFFD and S2HD/S2HA; CD T1SZ to TTB1, WXN, UWXN, PAN, AFFD, TBI1 and AMAIR. The codec returns their bits as unknown. Read the specification for those, and for which fields each Config uses.

The idea: an entry is a paper form with fixed boxes. Every field owns the same run of bits in every entry, so hardware finds it without searching. Select a box to see its field; hatched bits are ones Linux does not define.

Stream Table Entry: words 0–3 (Linux defines nothing in 4–7)

Select a box to see its field.

Swipe or scroll horizontally to see every column.

Context Descriptor: words 0, 1 and 3

Select a box to see its field.

Two-level stream and CD tables

A linear stream table is one array of STEs indexed by StreamID. With STRTAB_BASE_CFG.FMT = 1 (two-level), the StreamID bits above STRTAB_BASE_CFG.SPLIT (bits 10:6) index an array of 8-byte L1STD descriptors (Span in bits 4:0, L2Ptr in bits 51:6 in Linux; 55:6 in H.a), and the low Split bits index the STE in that L2 table. Linux uses Split 8, so each L2 table holds 256 STEs, one PCI bus, allocated when a device appears: StreamID 0x1234 is STE 0x34 of the table L1STD 0x12 points to.

CD tables split the same way when STE.S1Fmt = 2: the high SubstreamID bits index 8-byte L1CD descriptors (V in bit 0, L2Ptr in bits 51:12 in Linux; 55:12 in H.a) and Linux puts 1024 CDs in each 64 KiB L2 table, so SubstreamID 1500 is CD 476 of the second table. Either way the extra level costs one more memory read when the configuration cache is cold, and saves allocating a table for every possible ID up front.

The workbench’s translation model still holds STE and CD as named values (mode, ASID, VMID), not packed words: the codec is for reading and writing real layouts, and its tests check it against words built the way Linux’s arm_smmu_make_s1_cd(), arm_smmu_make_cdtable_ste() and arm_smmu_make_s2_domain_ste() build them.

Granules, blocks and address alignment

Arm supports capability-dependent translation granules of 4 KiB, 16 KiB and 64 KiB. A 2 MiB block in a 4 KiB-granule translation is not a “2 MiB granule”. Stage 1 and stage 2 have their own configured translation parameters; check the actual supported combinations before creating tables.

Swipe or scroll horizontally to see every column.

This table describes the familiar 64-bit descriptor, 4 KiB-granule case. Larger address spaces, other granules and newer descriptor extensions change the available starting levels and encodings. AMD’s lab labels its bottom leaf L1; the ordinary Arm page leaf above is L3. Sharing an integer “level” between the two walkers would be an error.

A table descriptor points to another translation table; a block/page descriptor points to a mapped frame. Starting-table alignment depends on the configured translation regime and table size, not just a blanket “all Arm roots are 4 KiB aligned” rule. STE/CD allocations and queue bases have their own rules too. Validate a structure according to its own format.

As in the AMD lesson, an ordinary byte address need not be aligned to the page boundary. Split a transfer at the remaining mapped extent, preserve the offset through both stages, and do not issue a segment whose translation or permission failed. CPU instruction alignment and device-specific descriptor alignment are additional, separate questions.

A reusable test vector

For VA offset 0xABC, map the stage-1 4 KiB page to IPA 0x801000. Map the enclosing 2 MiB stage-2 block at IPA 0x800000 to PA 0x12000000. The expected result is 0x12001ABC. That arithmetic can be shared with the AMD test; the descriptors, root selection and fault encodings cannot.

Nested walks and fault evidence

Read in the H.a spec§7.2 Event queue recorded faults and events p. 941

With both stages active, stage-1 translation-table fetches themselves require stage-2 translation. Stage 2 therefore protects translation metadata as well as the eventual data access. A missing stage-2 mapping for a stage-1 table is not the same failure as an absent stage-1 leaf. Some configuration fetches also depend on the selected translation configuration; use the specification’s address-space rules for each structure.

Record the stream, applicable substream, access direction, translation stage, faulting address and whether the access was a table fetch. Keep a descriptor-invalid condition separate from inaccessible table memory and a denied permission. Useful SMMUv3 diagnoses include C_BAD_STE, C_BAD_CD, F_TRANSLATION, F_PERMISSION, F_ACCESS and table/configuration fetch faults.

Do not translate all Arm failures into AMD IO_PAGE_FAULT or copy AMD event words into an Arm queue. An Arm event has its own layout and stage/transaction attributes. The presence of an event record does not prove the original operation is still outstanding: termination and stall are distinct outcomes.

Memory attributes, shareability, access-flag/dirty handling, input/output address sizes and privileged access semantics belong in an actual Arm implementation. A minimal translation demo should explicitly expose which of these it models, and reject unsupported configurations rather than pretending that an AMD field covers them.

Queues, ownership and completion

Read in the H.a spec§3.5 Command and Event queues p. 64§7.4 Event queue overflow p. 969

Swipe or scroll horizontally to see every column.

Sizes: IHI 0070 H.a states 16 bytes for Command queue entries (§4.1.1) and 32 bytes for Event records (§7.3). It gives no “16 bytes” sentence for the PRI queue; its record layout in Chapter 8 spans bits 127:0, which is 16 bytes. Linux's arm-smmu-v3.h uses the same three sizes (CMDQ_ENT_SZ_SHIFT 4, EVTQ_ENT_SZ_SHIFT 5, PRIQ_ENT_SZ_SHIFT 4).

SMMUv3 PROD and CONS contain an entry index plus a wrap bit. Equal indices and equal wrap bits mean empty; equal indices and different wrap bits mean full. The AMD illustration uses byte offsets and reserves one slot. Reusing that ring implementation unchanged would produce the wrong Arm full/empty behaviour.

For example, an eight-entry Arm queue can represent eight pending entries: producer index 0 with wrap 1, consumer index 0 with wrap 0. Multiplying the index by entry size gives a byte offset; the wrap bit is not part of that offset. Check hardware queue-size capability and base alignment independently.

Publish descriptor/command contents before advancing PROD. After observing a new hardware producer position, use the required ordering before reading the record. A JavaScript event loop cannot validate hardware cache coherency or memory barriers. Real code needs the platform’s DMA mapping, cache maintenance and ordering primitives.

Configuration includes supported IDR features, queue base/size registers and enable/acknowledgment handling through CR0/CR0ACK. GERROR reports global failures such as queue-access problems; EVTQ carries individual event records. Treating every global error as an ordinary page fault loses the recovery information.

Sharing CPU and device translation tables

A PE and SMMU can share compatible stage-1 table memory. The CPU uses TTBR registers; a stream's STE and CD independently select the SMMU roots and attributes. There is no Arm CR3. Align granule, address-size, permissions and memory-attribute settings, and map descriptor memory through stage 2 where required.

H.a §3.21 permits applicable PE broadcast TLBI participation when supported and configured in the relevant shareability domain. CD.ASET affects shared PE ASID matching. Explicit CMD_TLBI followed by CMD_SYNC and broadcast synchronization are distinct completion paths. Respect capability-dependent break-before-make rules; an AMD direct leaf-remap lesson is not an Arm update recipe.

Read the shared-table setup and maintenance guide, including the AMD, Intel VT-d and Arm comparison. Its executable CPU CR3 lesson is AMD-specific.

Invalidate the right cached state

Read in the H.a spec§7.1 Command queue errors p. 938

Swipe or scroll horizontally to see every column.

These are conceptual correspondences, not compatible binary opcodes. Invalidating an STE/CD cache does not automatically replace the required translation invalidation. Similarly, an IOMMU TLB invalidation does not evict a peripheral’s ATC. Respect the ordering and completion requirements for the particular change, including break-before-make where required by the translation architecture.

A safe validation scenario holds DMA while changing the mapping, verifies that stale translations cannot access a reused page, then waits for the necessary completion before retrying. Include failed ATC acknowledgments, queue errors and timeouts: none should be counted as successful completion. CMDQ consumption alone is insufficient evidence that a device invalidation has finished.

Every command, and what the workbench does with it

Opcodes are Linux v6.18’s CMDQ_OP_*; the four marked * are not defined by Linux and come from QEMU v10.1.0’s SMMUv3 model. EL3 commands and SMMUv3.2+ additions are left out. A command the workbench does not model is consumed with no effect and says “not modelled”: it is a real command, so it is never reported as CERROR_ILL. Only an opcode outside this list is.

Swipe or scroll horizontally to see every column.

CMD_TLBI_S2_IPA removes stage-2 entries only. A nested stream’s entries combine both stages, so after changing a stage-2 mapping under nested streams, Linux also issues CMD_TLBI_NH_ALL for the VMID (arm_smmu_tlb_inv_range_domain()); the workbench keeps nested entries until that command or CMD_TLBI_S12_VMALL.

Range invalidation

On an SMMU with range invalidation (IDR3.RIL, SMMUv3.2), one CMD_TLBI_NH_VA, CMD_TLBI_NH_VAA or CMD_TLBI_S2_IPA covers (NUM + 1) × 2^SCALE pages from its address. TG gives the page size (1 = 4 KiB, 2 = 16 KiB, 3 = 64 KiB; 0 means one address, the only form without RIL), TTL the level of the leaf being removed (3 for a 4 KiB page, 2 for a 2 MiB block, 1 for 1 GiB; 0 for any) and Leaf whether only leaf entries go. NUM is 5 bits, so Linux’s __arm_smmu_tlb_inv_range() splits a range by its set bits, lowest first: 33 pages from 0x10000 become one page at 0x10000 (NUM 0, SCALE 0), then 32 pages at 0x11000 (NUM 0, SCALE 5). The workbench caches only final 4 KiB results, so Leaf and TTL narrow nothing there; a range drops every cached result whose page or block it overlaps.

Shared ASIDs share TLB entries

TLB entries are tagged by VMID and ASID (stage 1) or VMID alone (stage 2 only), not by StreamID or SubstreamID. Two streams whose CDs use the same ASID in the same VMID therefore share entries, which is right when they share one address space (Linux gives one ASID to a stage-1 domain and attaches every device of that domain to it). Two different address spaces under one ASID would see each other’s translations. The workbench tags entries the same way: a second stream with the same ASID and VMID hits the first stream’s entry.

Stall recovery is different from PCIe PRI

Stalled transaction

If the stream and SMMU support and enable the relevant stall behaviour, a recoverable translation fault can leave a transaction stalled. Software receives identifying event information, repairs the cause where appropriate, invalidates stale state and uses CMD_RESUME with the applicable action. Aborting and retrying are separate choices. Keep StreamID and the stall tag associated with the exact request.

PCIe page request

A PRI-capable peripheral sends a page request carrying its request context and group information. The SMMU places a corresponding record in PRIQ. Software services the group and responds with CMD_PRI_RESP. The endpoint subsequently follows the ATS/PRI protocol to retry; draining PRIQ alone neither grants access nor copies payload bytes.

Do not use CMD_RESUME as a substitute for CMD_PRI_RESP, or assume every event implies PRI is supported. Support, enablement, response identity and request lifetime all need separate tests. Log acknowledgment, request response, mapping repair, invalidation completion and DMA completion are five distinct milestones.

Interrupts: SMMU versus GIC/ITS

The SMMU can notify software about events, PRI work and global errors using supported interrupt mechanisms. That notification path is distinct from a device’s functional interrupt and from the Arm Generic Interrupt Controller (GIC) and Interrupt Translation Service (ITS) used in relevant systems.

AMD’s guest-capable IRTE, virtual APIC backing state and GA_GUEST_NR notification should not be renamed into an “Arm GA entry”. SMMUv3 defines no equivalent GA log. Arm interrupt virtualization belongs to the applicable GIC/ITS architecture and platform integration. A future interrupt lesson would therefore need its own model and reference, with device completion, SMMU notification and guest interrupt delivery shown separately.

Fault injection and recovery: do not share the register protocol

The same test intent—deny a write, invalidate a descriptor, fill a queue—needs architecture-specific expected evidence. This is especially important when reusing test intent across architectures.

Swipe or scroll horizontally to see every column.

For queue-full tests, compare logical occupancy and ownership, not numeric pointer order. For combined-fault tests, record the permitted outcome set and cache state. For recovery tests, verify that the corrected command is re-fetched, that later commands had no effect before recovery, and that the correct completion signal is observed.

Run the AMD fault-injection and buffer workbench. The Arm column is a checked architectural comparison. Run the separate Arm backend for the implemented subset.

Arm implementation and remaining boundaries

Reuse request/response records, page-offset arithmetic, test fixtures and the UI’s presentation components. Keep STE/CD codecs, Arm descriptor walking, queue pointer arithmetic, fault records and commands in Arm-specific modules. Avoid an expanding set of architecture conditionals inside the current AMD encoder.

Swipe or scroll horizontally to see every column.

The Arm workbench implements a semantic, non-secure 4 KiB-granule subset with separate configuration and translation caches, CMDQ/EVTQ/PRIQ, stall and PRI recovery, and the commands listed under invalidation. STE, CD, L1STD, L1CD and command words have an exact codec for the fields Linux defines, but the translation model runs on named STE/CD values. Memory attributes, HTTU, the EL2 regime, ATS/ATC, RME and advanced v3.4/v3.5 facilities remain reference-only. Inspect capability and revision coverage.

The Arm browser lab boots QEMU's virt machine with an SMMUv3 in your browser; its dmakit run edu programs the SMMU by hand with what this model produced.

Public references

Verified sections§3.3.1 Stream table lookup p. 46§3.5 Command and Event queues p. 64§5.2 STE, Stream Table Entry p. 250§5.4 CD, Context Descriptor p. 299§7.1 Command queue errors p. 938§7.2 Event queue recorded faults and events p. 941§7.4 Event queue overflow p. 969

Architecture baseline: Arm SMMUv3, IHI 0070 H.a (6 March 2026). Relevant sections: 3.2–3.5 (identity, translation and queues), Chapter 4 (commands), Chapter 5 (configuration structures), Chapter 6 (registers), Chapter 7 (events), and the PCIe PRI definitions. Optional features must be checked against the actual implementation.

Cross-checks: Linux SMMUv3 definitions, SMMUv3 driver, the STE, CD and command layouts and range invalidation in Linux v6.18 arm-smmu-v3.h and arm-smmu-v3.c, four opcodes Linux lacks in QEMU v10.1.0, and AMD 48882 comparison baseline. This independently written guide does not reproduce the architecture specification or claim hardware conformance.