ASAravind Sivakumaran
Theme

Theme and appearance

Appearance
Palette

Pick any colour. The displayed accent adjusts for readable contrast in light or dark mode.

← PCIe guide

PCIe / ENUMERATION

Nobody gives the computer a list. How does it find every device?

At power-on the processor knows one thing about PCIe: where the root complex is. It does not know that a network card sits in the second slot or that an NVMe drive hides behind a switch. Firmware, and then the operating system, find out the way a new postman learns a town: walk every street, knock on every door, and write down who answers. Junctions get signposts ("streets 02 to 05 this way"), and every house gets a postbox address it can be reached at.

This chapter walks a real machine: QEMU's q35 PC with SeaBIOS, three root ports, a switch, a network card and an NVMe drive, booted with Linux 6.8. The bus numbers, BAR sizes and windows below are what that boot produced. Start with the simple guide; the second half is for people who write firmware, drivers or tests.

What claims does this page make? 5 points
  • Spec rule: configuration headers, bus-number routing, BARs and Request Retry Status come from the PCI Express Base Specification (members only, cited by section as Linux cites it, not checked here).
  • Emulated observation: bus numbers, BAR values, windows and the boot log come from QEMU 8.2's q35 with SeaBIOS, booted with Ubuntu's Linux 6.8; the raw captures, versions and commands are in the site's repository under docs/evidence/pcie-qemu. QEMU emulates the devices, so link speeds and timings are not a real board's.
  • Simulation: the bus walk and the experiments are models that replay that boot; the walk is condensed, and UEFI firmware and Linux walk in a slightly different order and can reach the same numbers.
  • Real hardware: none on this page; the N100's own tree is in Reading lspci.
  • Linux behaviour is quoted from its source, read on 2 October 2026.

The simple guide, in five ideas

A tree: the root complex on bus 00 at the top, three root ports below it, a switch under the middle one and an NVMe drive under the switch. A pulse travels from the root down to the drive, the way a configuration read does. Root complexbus 00 Root port 1c.0bus 01 below Root port 1c.1buses 02-05 below Root port 1c.2bus 06: empty NIC 01:00.0a house Switch 02:00.0a junction NVMe 04:00.0two junctions on
  1. Every function has an address: bus, device, function. 04:00.0 is bus 4, device 0, function 0, written in hex. A card can hold up to eight functions (a network card with two ports often has two); each looks like a separate device to software.
  2. Every function has 4 KiB of configuration space (256 bytes on older, conventional PCI devices). Its first bytes say who it is: vendor ID, device ID, and a class code (network, storage, bridge). lspci prints them.
  3. Finding devices is knocking on doors. Software reads the vendor ID at every address. A device answers with its ID; an empty address answers ffff, a value no vendor may use.
  4. Bridges are junctions, and software numbers the streets. A root port or a switch port is a bridge. Each holds three numbers: the bus it sits on (primary), the bus just below it (secondary) and the highest bus anywhere below it (subordinate). A request for a bus in that range goes through.
  5. Devices ask for address space; software hands it out. A device's BARs (base address registers) say how much memory its registers need. Firmware or the OS picks an address for each, and every bridge above gets a window that covers them.

That is all of enumeration: number the buses top-down, add up the BAR sizes bottom-up, then hand out addresses top-down. The rest of this page shows each step on the real machine.

The bus walk, step by step

This is the walk SeaBIOS ran on the QEMU machine: depth first, so when it meets a bridge it gives it the next free bus number, walks everything below it, and only then writes the bridge's subordinate bus. Addresses show ?? until the walk reaches them, because a device's bus number does not exist until a bridge above it is numbered.

A condensed model of the depth-first numbering, not a trace: one step stands for one or more configuration accesses, and SeaBIOS's first pass over each bus, which closes every bridge on it (secondary ff, subordinate 00) so stale numbers cannot route a request, is left out (pci_bios_init_bus_rec).

    Watch the subordinate numbers: each bridge opens with ff ("send me everything") so reads for buses not yet numbered still reach the walk, then closes to the real highest bus on its way back up. The result, [01], [02-05], [03-05], [04], [05] and [06], is exactly what Linux 6.8 printed for this machine.

    The CPU reaches the host bridge on bus 00. Below it, root port 1c.0 leads to the network card on bus 01; root port 1c.1 leads to a switch whose two downstream ports lead to the NVMe drive on bus 04 and an empty slot on bus 05; root port 1c.2 leads to an empty slot on bus 06. CPUECAM loads Host bridgebus 00 00:1c.0root port [01] 00:1c.1root port [02-05] 00:1c.2root port [06] 01:00.0network card 02:00.0switch up [03-05] bus 06empty slot 03:00.0switch down [04] 03:01.0switch down [05] 04:00.0NVMe drive bus 05empty slot

    Experiments

    The QEMU machine: q35, SeaBIOS, Linux 6.8. Bus numbers from the walk above; addresses from that boot.

    Experiment

      Bridge and device registers

        Configuration accesses

          Notes

            Configuration addresses: ECAM and port cf8

            Software reaches configuration space in one of two ways. The original PC way writes an address to I/O port cf8 and reads the value from port cfc: two accesses, so the kernel holds a lock between them, and it reaches only the first 256 bytes. PCIe's way, ECAM (Enhanced Configuration Access Mechanism), gives every function's 4 KiB its own memory address: 1 MiB a bus, 32 KiB a device, 4 KiB a function, so 256 buses take 256 MiB. On this machine the window starts at b0000000 (PCI: ECAM [mem 0xb0000000-0xbfffffff] in the boot log). Type an address:

              Try register 100, the first extended capability: ECAM reaches it, the legacy ports do not (AMD processors have an extension that puts register bits 11-8 into bits 27-24 of the cf8 value; Linux uses it only where ECAM is missing). Arm and RISC-V boards have no I/O ports and use ECAM, or a controller-specific window, only.

              Type 0 and Type 1 headers, byte by byte

              The first 64 bytes of configuration space are the header. Byte 0e says which kind: type 0 for an endpoint (six BARs), type 1 for a bridge (two BARs, then bus numbers and windows). Bit 7 of the same byte says the device has more functions. Both dumps are from the QEMU machine (lspci -x, bytes in little-endian order). Press any highlighted bytes:

              04:00.0, the NVMe drive: type 0

              00: 10 00 02 00 00 00 10: 00 00 00 00 00 00 00 00 20: 00 00 00 00 00 00 00 00 00 00 00 00 f4 1a 00 11 30: 00 00 00 00 00 00 00 00 00 00 00 0a 01 00 00

              03:00.0, the switch's downstream port above it: type 1

              00: 4c 10 33 82 03 01 10 00 01 00 00 00 00 10: 00 00 00 00 00 00 00 00 00 00 00 20: 00 00 00 00 00 00 00 00 30: 00 00 00 00 90 00 00 00 00 00 00 00 00 00 02 00

              Nothing selected yet.

              OffsetType 0 (endpoint)Type 1 (bridge)
              00-0fVendor, device, command, status, class, header type: the same in both
              10-17BAR0, BAR1BAR0, BAR1 (often unused)
              18-1bBAR2primary, secondary, subordinate bus; secondary latency
              1c-1fBAR3I/O base and limit; secondary status
              20-27BAR4, BAR5memory base and limit; prefetchable base and limit
              28-2fCardBus CIS; subsystem vendor and IDprefetchable base and limit, upper 32 bits
              30-33expansion ROMI/O base and limit, upper 16 bits
              34capabilities pointer
              38-3breservedexpansion ROM
              3c-3finterrupt line and pin; min grant, max latencyinterrupt line and pin; bridge control

              BARs and windows: sizes up, addresses down

              Bus numbers are given out top-down. Address space works the other way: software first learns the size of every BAR at the bottom, adds them up into each bridge's window on the way up, and only then hands out addresses from the top. The NVMe drive's numbers, read from Linux in that QEMU guest:

              BAR0 before:                     fe400004 00000000
              command, decoding off:           0107 → 0000 (Linux writes 0104)
              after writing ffffffff ffffffff: ffffc004 ffffffff
              restored:                        fe400004 00000000, command 0107

              The existing BAR decoder on the lspci page turns any BAR value into lspci's Region line. Two features change these rules on big devices: Resizable BAR (extended capability 0x15) lets a GPU offer several sizes, so the OS can map all of its memory at once, and SR-IOV adds a second set of BARs for the virtual functions, sized once and repeated for each VF (see SR-IOV).

              For professionals: Linux's scan, step by step

              Linux has its own walk, even on a PC where the firmware already did one. It runs once per host bridge the platform described (ACPI on PCs, the device tree on many Arm boards, see the boot discovery chapter), from drivers/pci/probe.c:

              1. pci_create_root_bus(), from acpi_pci_root_create() on PCs or pci_scan_root_bus_bridge() on device-tree hosts: create the root bus with the bus range and windows the platform gave (the root bus resource lines in the log).
              2. pci_scan_child_bus_extend(): for devices 0 to 31, pci_scan_slot(). Below a root or downstream port only device 0 is tried (only_one_child(): a PCIe link carries one device; pci=pcie_scan_all overrides it).
              3. pci_scan_single_device() → pci_bus_read_dev_vendor_id(): ffffffff, 00000000, 0000ffff and ffff0000 all mean empty; vendor 0001 means "retry", and Linux waits about a minute for it (pci_bus_wait_rrs(), a 60 s limit).
              4. pci_setup_device(): read the header type and class, print the [vendor:device] type 01 class 0x060400 line, then pci_read_bases(): six BARs for type 0, two plus the windows for type 1. Functions 1 to 7 are read only if function 0's header has bit 7 set, or through ARI.
              5. Bridges, in two passes over the whole bus: pass 0 keeps bridges the firmware configured and walks below them; pass 1 numbers the rest (an unconfigured or invalid bridge, "bridge configuration invalid ([bus …]), reconfiguring"), from the highest bus used so far, plus spare buses for hot-plug ports (pci=hpbussize, default 1).
              6. Resources: on x86, pcibios_resource_survey() claims what the firmware assigned and Linux places only what is left (the I/O windows in this boot). Generic Arm and RISC-V hosts (pci_host_probe()) reassign everything unless the platform asks to preserve its setup.
              7. Drivers bind once all of this is done: pci_bus_add_devices().
              SeaBIOS (this boot)EDK2 (UEFI)Linux
              Orderdescends into a bridge as soon as it meets itdescends as soon as it meets itscans the whole bus, then each bridge
              Bus numbersassigns allassigns all, with hot-plug paddingx86: keeps the firmware's; generic DT hosts: assigns all; pci=assign-busses forces it
              Windowsat least 2 MiB eachfrom the platform's resource mapat least 1 MiB; pci=realloc may move firmware's choices
              Below a downstream porttries all 32 devicestries all 32device 0 only

              Kernel parameters that change enumeration, from the kernel's parameter list:

              The real boot log, line by line

              Linux 6.8 on the QEMU machine (dmesg, timestamps removed). Press a line:

              Nothing selected yet.

              Try it in QEMU

              The machine on this page is one command (QEMU 8.2, any kernel; add a root filesystem or an initramfs to get a shell, then lspci -tv and dmesg | grep pci):

              qemu-system-x86_64 -M q35 -m 512 -nographic -kernel vmlinuz -append console=ttyS0 \
               -device pcie-root-port,id=rp1,bus=pcie.0,chassis=1,slot=1,addr=0x1c.0,multifunction=on \
               -device pcie-root-port,id=rp2,bus=pcie.0,chassis=2,slot=2,addr=0x1c.1 \
               -device pcie-root-port,id=rp3,bus=pcie.0,chassis=5,slot=5,addr=0x1c.2 \
               -device x3130-upstream,id=up,bus=rp2 \
               -device xio3130-downstream,id=dn1,bus=up,chassis=3,slot=3 \
               -device xio3130-downstream,id=dn2,bus=up,chassis=4,slot=4 \
               -device e1000e,bus=rp1,netdev=n0,romfile= -netdev user,id=n0 \
               -device nvme,bus=dn1,serial=x,drive=d0 -drive if=none,id=d0,file=disk.img,format=raw

              What lspci -tv prints for it:

              -[0000:00]-+-00.0  Intel Corporation 82G33/G31/P35/P31 Express DRAM Controller
                         +-01.0  Device 1234:1111
                         +-1c.0-[01]----00.0  Intel Corporation 82574L Gigabit Network Connection
                         +-1c.1-[02-05]----00.0-[03-05]--+-00.0-[04]----00.0  Red Hat, Inc. QEMU NVM Express Controller
                         |                               \-01.0-[05]--
                         +-1c.2-[06]--
                         +-1f.0  Intel Corporation 82801IB (ICH9) LPC Interface Controller
                         +-1f.2  Intel Corporation 82801IR/IO/IH (ICH9R/DO/DH) 6 port SATA Controller [AHCI mode]
                         \-1f.3  Intel Corporation 82801I (ICH9 Family) SMBus Controller

              Then change one thing and predict the log before you boot: move the drive to dn2, add a fourth root port, or boot with pci=assign-busses. On your own PC, sudo lspci -tv and sudo dmesg | grep -E 'pci (0000|bus)' show the same story for real hardware.

              Sources