Advanced Systems Programming - Lesson 19: Detecting PCI devices

Detecting PCI devices On identifying the peripheral equipment installed in our PC Early PCs • Peripheral devices in the early PCs used fixed i/o-ports and fixed memory-addresses, e.g.: – Video memory address-range: 0xA0000-0xBFFFF – Programmable timer i/o-ports: 0x40-0x43 – Keyboard and mouse i/o-ports: 0x60-0x64 – Real-Time Clock’s i/o-ports: 0x70-0x71 – Hard Disk controller’s i/o-ports: 0x01F0-01F7 – Graphics controller’s i/o-ports: 0x03C0-0x3CF – Serial-port controller’s i/o-ports: 0x03F8-0x03FF – Parallel-port controller’s i/o-ports: 0x0378-0x037A The PC’s evolution • It became clear in the 1990s that there would be contention among equipment vendors for ‘fixed’ resource-addresses, which of course were in limited supply • Among the goals that motivated the PCI Specification was the creation of a more flexible scheme for allocating addresses that future peripheral devices could use PCI Configuration Space PCI Configuration Space Body (48 doublewords – variable format) 64 doublewords PCI Configuration Space Header (16 doublewords – fixed format) A non-volatile parameter-storage area for each PCI device-function PCI Configuration Header Status Register Command Register Device ID Vendor ID BIST Cache Line Size Class Code Class/SubClass/ProgIF Revision ID Base Address 0 Subsystem Device ID Subsystem Vendor ID CardBus CIS Pointer reserved capabilities pointer Expansion ROM Base Address Minimum Grant Interrupt Pin reserved Latency Timer Header Type Base Address 1 Base Address 2Base Address 3 Base Address 4Base Address 5 Interrupt Line Maximum Latency 31 031 0 16 doublewords Dwords 1 - 0 3 - 2 5 - 4 7 - 6 9 - 8 11 - 10 13 - 12 15 - 14 Three IA-32 address-spaces memory space (4GB) i/o space (64KB) PCI configuration space (16MB) accessed using a large variety of processor instructions (mov, add, or, shr, push, etc.) and virtual-to-physical address-translation accessed only by using the processor’s special ‘in’ and ‘out’ instructions (without any translation of port-addresses) i/o-ports 0x0CF8-0x0CFF dedicated to accessing PCI Configuration Space reserved Interface to PCI Configuration Space CONFADD ( 0x0CF8) CONFDAT ( 0x0CFC) 31 23 16 15 11 10 8 7 2 0 E N bus (8-bits) device (5-bits) doubleword (6-bits) function (3-bits) 00 PCI Configuration Space Address Port (32-bits) PCI Configuration Space Data Port (32-bits) 31 0 Enable Configuration Space Mapping (1=yes, 0=no) Reading PCI Configuration Data • Step one: Output the desired longword’s address (bus, device, function, and dword) with bit 31 set to 1 (to enable access) to the Configuration-Space Address-Port • Step two: Read the designated data from the Configuration-Space Data-Port: # read the PCI Header-Type field (byte 2 of dword 3) for bus=0, device=0, function=0 movl $0x8000000C, %eax # setup address in EAX movw $0x0CF8, %dx # setup port-number in DX outl %eax, %dx # output address to port mov $0x0CFC, %dx # setup port-number in DX inl %dx, %eax # input configuration longword shr $16, %eax # shift word 2 into AL register movb %al, header_type # store Header Type in variable Demo Program • We created a short Linux utility that searches for and reports all of your system’s PCI devices • It’s named “pciprobe.cpp” on our CS635 website • It uses some C++ macros that expand to Intel input/output instructions -- which normally are ‘privileged’ instructions that a Linux application- program is not allowed to execute (segfault!) • Our system administrator (Alex Fedosov) has created a utility (named “iopl3”) that will allow your command-shell to acquire I/O privileges Example: network interface • We identify the network interface controller in our classroom PC’s by class-code 0x02 • The subclass-code 0x00 is for ‘ethernet’ • We can identify the NIC from its VENDOR and DEVICE identification-numbers: • VENDOR_ID = 0x14E4 • DEVICE_ID = 0x1677 • You can use the ‘grep’ command to search for these numbers in this header-file: Vendor’s identity • The VENDOR-ID 0x14E4 belongs to the Broadcom Corporation (headquarters in Irvine, California) • Information about this firm may be learned from the corporation’s website: • The DEVICE-ID 0x1677 is used to signify Broadcom’s BCM5751 ethernet product nic Typical NIC TX FIFO RX FIFO transceiver LAN cableB U S main memory packet buffer CPU Packet filtering capability • Network Interface’s hardware needs to implement ‘filtering’ of network packets • Otherwise the PC’s memory-usage and processor-time will be wasted handling packets not meant for this PC to receive network packet’s layout Destination-address (6-bytes) Source-address (6-bytes) Each data-packet begins with the 6-byte device-address of the network interface which is intended to receive it Your NIC’s unique address • You can see the Hardware Address of the ethernet controller on your PC by typing: $ /sbin/ifconfig • Look for it in the first line of screen-output that is labeled ‘eth0’, for example: • (The NIC’s filter-register stores this value) eth0 Link encap: Ethernet HWaddr 00:11:43:C9:50:3A Our ‘tigon3.c demo • We wrote a kernel module that lets users see certain register-values which pertain to the BCM5751 network interface in your classroom workstation: – (1) the PCI Configuration Space registers – (2) the Media Access Controller’s address • It also shows your machine’s node-name (in case you want to save the information) How we got the MAC-address • We do not have Broadcom’s programming datasheet -- but we do have Linux source code for the ‘tigon3’ device-driver, which includes a header-file ‘tg3.h’ found here: • If you scroll through the #define directives you will see the offset where the hardware address is stored in the memory-mapped register-space of the ‘tigon3’ interface Driver’s authors • The Linux kernel’s open-source driver for the Broadcom ‘tigon3’ network controller was jointly written by David S. Miller (see photo below) and Jeff Garzik David Miller’s announcement in Feb 2002 of their driver’s BETA version is online. It includes his candid comments about the challenge of writing such a driver when the vendor does not make available its device’s programming documentation. How we got tigon3 registers Status Register Command Register DeviceID 0x1677 VendorID 0x14E4 BIST Cache Line Size Class Code Class/SubClass/ProgIF Revision ID Base Address 0 Subsystem Device ID Subsystem Vendor ID CardBus CIS Pointer reserved capabilities pointer Expansion ROM Base Address Minimum Grant Interrupt Pin reserved Latency Timer Header Type Base Address 1 Base Address 2Base Address 3 Base Address 4Base Address 5 Interrupt Line Maximum Latency 31 031 0 16 doublewords Dwords 1 - 0 3 - 2 5 - 4 7 - 6 9 - 8 11 - 10 13 - 12 15 - 14 Linux helper-functions #include struct pci_dev *devp; unsigned int iomem_base, iomem_size; void *io; devp = pci_get_device( 0x14E4, 0x1677, NULL ); if ( !devp ) return –ENODEV; iomem_base = pci_resource_start( devp, 0 ); iomem_size = pci_resource_len( devp, 0 ); io = ioremap( iomem_base, iomem_size ); if ( !io ) return -EBUSY; Big-Endian to Little-Endian mac 1 mac 0 mac 5 mac 4 mac 3 mac 2 0x0410 0x0411 0x0412 0x0413 0x0414 0x0415 0x0416 0x0417 Broadcom network interface storage-addresses Intel IA-32 character-array storage mac 0 mac 1 mac 2 mac 3 mac 4 mac 5 In-class exercise • Copy the ‘tigon3.c’ source-module to your own directory, then rename it ‘anchor.c’ • Your assignment is to modify it so that it will show information about the Intel NICs in our ‘anchor’ cluster’s machines: #define VENDOR_ID 0x8086 // Intel Corp #define DEVICE_ID 0x109A // 82573L NIC • Intel’s filter-register at offset 0x5400 uses the ‘little endian’ storage-convention Little-Endian to Little-Endian mac 0 mac 1 mac 2 mac 3 mac 4 mac 5 0x5400 0x5401 0x5402 0x5403 0x5404 0x5405 0x5406 0x5407 Intel network interface storage-addresses Intel IA-32 character-array storage mac 0 mac 1 mac 2 mac 3 mac 4 mac 5