I/O Architecture
bus
the primary communication channels inside the computer.Any computer has a system bus that connects most of the internal hardware devices. A typical system bus is the PCI (Peripheral Component Interconnect) bus.
the same computer includes several buses of different types, linked together by hardware devices called bridges. Two high-speed buses are dedicated to the data transfers to and from the memory chips: the frontside bus connects the CPUs
to the RAM controller, while the backside bus connects the CPUs directly to the external hardware cache. The host bridge links together the system bus and the frontside bus.
I/O bus
I/O bus is the data path that connects a CPU to an I/O device.
The I/O bus, in turn, is connected to each I/O device by means of a hierarchy of hardware components including up to
three elements: I/O ports, interfaces, and device controllers.
I/O Ports
Each device connected to the I/O bus has its own set of I/O addresses, which are usually called I/O ports. In the IBM PC architecture, the I/O address space provides up to 65,536 8-bit I/O ports.
in,ins,out,outs : Four special assembly language instructions called in, ins, out, and outs allow the CPU to read from and write into an I/O port. While executing one of these instructions, the CPU selects the required I/O port and transfers the data between a CPU register and the port.
mapping i/o ports to physical address space : I/O ports may also be mapped into addresses of the physical address space. The processor is then able to communicate with an I/O device by issuing assembly language instructions that operate directly on memory.
the I/O ports of each device are structured into a set of specialized registers. The CPU writes the commands to be sent to the device into the device control register and reads a value that represents the internal state of the device from the device status register. The CPU also fetches data from the device by reading bytes from the device input register and pushes data to the device by writing bytes into the device output register.
Accessing I/O ports
The in, out, ins, and outs assembly language instructions access I/O ports. The following auxiliary functions are included in the kernel to simplify such accesses:
//Read 1, 2, or 4 consecutive bytes, respectively, from an I/O port.
inb(), inw(), inl()
//Read 1, 2, or 4 consecutive bytes, respectively, from an I/O port, and then execute a “dummy” instruction to introduce a pause.
inb_p(), inw_p(), inl_p()
//Write 1, 2, or 4 consecutive bytes, respectively, to an I/O port. execute "dummy" if it suffix '_p'
outb(), outw(), outl(), outb_p(), outw_p(), outl_p()
//Read sequences of consecutive bytes in groups of 1, 2, or 4, respectively, from an I/O port. The length of the sequence is specified as a parameter of the functions.
insb(), insw(), insl()
//Write sequences of consecutive bytes, in groups of 1, 2, or 4, respectively, to an I/O port.
outsb(), outsw(), outsl()
the kernel keeps track of I/O ports assigned to each hardware device by means of resources.A **resource **represents a portion of some entity that can be exclusively assigned to a device driver. In our case, a resource represents a range of I/O port addresses. All resources of the same kind are inserted in a treelike data structure; for instance, all resources representing I/O port address ranges are included in a tree rooted at the node ioport_resource.
I/O Interfaces
An I/O interface is a hardware circuit inserted between a group of I/O ports and the corresponding device controller. It acts as an interpreter that translates the values in the I/O ports into commands and data for the device. It detects changes in the device state and correspondingly updates the I/O port that plays the role of status register. This circuit can also be connected through an IRQ line to a Programmable Interrupt Controller, so that it issues interrupt requests on behalf of the device.
Custom I/O interfaces
Devoted to one specific hardware device. The devices attached to a custom I/O interface can be either internal devices (devices located inside the PC’s cabinet) or external devices (devices located outside the PC’s cabinet).
General-purpose I/O interfaces
Used to connect several different hardware devices. Devices attached to a general-purpose I/O interface are usually external devices.
Custom I/O interfaces
e.g.
Keyboard interface
Graphic interface
Disk interface
Bus mouse interface
Network interface
General-purpose I/O interfaces
Modern PCs include several general-purpose I/O interfaces, which connect a wide range of external devices. The most common interfaces are:
- Parallel port
Traditionally used to connect printers, it can also be used to connect removable disks, scanners, backup units, and other computers. Transferred 8 bits at a time.
- Serial port
Like the parallel port, but the data is transferred 1 bit at a time.
- PCMCIA interface
Included mostly on portable computers. The external device, which has the shape of a credit card, can be inserted into and removed from a slot without rebooting the system.(hard disks, network cards)
- SCSI (Small Computer System Interface) interface
A circuit that connects the main PC bus to a secondary bus called the SCSI bus. The SCSI-2 bus allows up to eight PCs and external devices.The SCSI standard is the communication protocol used to connect devices via the SCSI bus.
- Universal serial bus (USB)
A general-purpose I/O interface that operates at a high speed and may be used for the external devices traditionally connected to the parallel port, the serial port, and the SCSI interface.
Device Controllers
A complex device may require a device controller to drive it.
- It interprets the high-level commands received from the I/O interface and forces the device to execute specific actions by sending proper sequences of electrical signals to it.
- It converts and properly interprets the electrical signals received from the device and modifies (through the I/O interface) the value of the status register.
Several hardware devices include their own memory, which is often called I/O shared memory. For instance, all recent graphic cards include tens of megabytes of RAM in the frame buffer, which is used to store the screen image to be displayed on the monitor.
The Device Driver Model
The framework is called the device driver model, implement a set of op-function and helper-function that offer a unifying view of all buses, devices, and device drivers in the system;
The sysfs Filesystem
The sysfs filesystem is a special filesystem similar to /proc that is usually mounted on the /sys directory. The /proc filesystem was the first special filesystem designed to allow User Mode applications to access kernel internal data structures. The /sysfs filesystem has essentially the same objective, but it provides additional information
on kernel data structures; furthermore, /sysfs is organized in a more structured way than /proc.
Kobjects
The core data structure of the device driver model is a generic data structure named kobject, which is inherently tied to the sysfs filesystem: each kobject corresponds to a directory in that filesystem.
Kobjects are embedded inside larger objects—the so-called “containers”—that describe the components of the device driver model. The descriptors of buses, devices, and drivers are typical examples of containers;
ksets
The kobjects can be organized in a hierarchical tree by means of ksets. A kset is a collection of kobjects of the same type—that is, included in the same type of container.
subsystems
Collections of ksets called subsystems also exist. A subsystem may include ksets of different types, and it is represented by a subsystem data structure.
Even the subsystem data structure can be embedded in a larger “container” object; the reference counter of the container is thus the reference counter of the embedded subsystem—that is, the reference counter of the kobject embedded in the kset embedded in the subsystem. The subsys_get() and subsys_put() functions respectively increase and decrease this reference counter.
subsystem and kset can both treated like one type of the wrapper of the kobjects
Components of the Device Driver Model
The device driver model is built upon a handful of basic data structures, which represent buses, devices, device drivers.
Devices
struct device
The devices are organized hierarchically: a device is the “parent” of some “children” devices if the children devices cannot work properly without the parent device. For instance, in a PCI-based computer, a bridge between the PCI bus and the USB bus is the parent device of every device hosted on the USB bus.
the driver field points to the descriptor of the device driver. For each bus type, moreover, there is a list including all devices that are hosted on the buses of the given type; the bus field of the device object stores the pointers to the bus type descriptor.
The device_register() function inserts a new device object in the device driver model, and automatically creates a new directory for it under /sys/devices. Conversely, the device_unregister() function removes a device from the device driver model.
Drivers
ops bind on the struct device_struct
The device_driver object includes four methods for handling hot-plugging, plug and play, and power management. The probe method is invoked whenever a bus device driver discovers a device that could possibly be handled by the driver; the corresponding function should probe the hardware to perform further checks on the device. The remove method is invoked on a hot-pluggable device whenever it is removed; it is also invoked on every device handled by the driver when the driver itself is unloaded. The shutdown, suspend, and resume methods are invoked on a device when the kernel must change its power state.
Usually, the device_driver object is statically embedded in a larger descriptor.(e.g. usb_driver)
Buses
Each bus type supported by the kernel is described by a bus_type object.Each bus_type object includes an embedded subsystem; the subsystem stored in the bus_subsys variable collects all subsystems embedded in the bus_type objects.The per-bus subsystem typically includes only two ksets named drivers and devices (corresponding to the drivers and devices fields of the bus_type object, respectively).
The bus_for_each_drv() and bus_for_each_dev() functions iterate over the elements of the
lists of drivers and devices, respectively.
Classes
Each class is described by a class object. All class objects belong to the class_subsys subsystem associated with the /sys/class directory.
Each class object includes a list of class_device descriptors, each of which represents a single logical device belonging to the class. The class_device structure includes a dev field that points to a device descriptor, thus a logical device always refers to a given device in the device driver model.
However, there can be several class_device descriptors that refer to the same device. In fact, a hardware device
might include several different sub-devices, each of which requires a different User Mode interface.
Device drivers in the same class are expected to offer the same functionalities to the User Mode applications.
The classes of the device driver model are essentially aimed to provide a standard method for exporting to User Mode applications the interfaces of the logical devices. Each class_device descriptor embeds a kobject having an attribute (special file)named dev. Such attribute stores the major and minor numbers of the device file that is needed to access to the corresponding logical device
Device Files
I/O devices are treated as special files called device files.
A device file is usually a real file stored in a filesystem. Its inode, however, doesn’t need to include pointers to blocks of data on the disk (the file’s data) because there are none. Instead, the inode must include an identifier of the hardware device corresponding to the character or block device file.
identifier
this identifier consists of the type of device file (character or block) and a pair of numbers. The first number, called the major number, identifies the device type. Traditionally, all device files that have the same major number and the same type share the same set of file operations, because they are handled by the same device driver. The second number, called the minor number, identifies a specific device among a group of devices that share the same major number.
Notice that character and block devices have independent device number.
mknod() syscall
it is used to creates a filesystem node (file, device special file, or named pipe). when creating device file, it receives the name of the device file, its type, and the major and minor numbers as its parameters.
User Mode Handling of Devices Files
in Linux 2.6: the major number is now encoded in 12 bits, while the minor number is encoded in 20 bits.
Dynamic device number assignment
Each device driver specifies in the registration phase the range of device numbers that it is going to handle.The
driver can, however, require the allocation of an interval of device numbers without specifying the exact values: in this case, the kernel allocates a suitable range of numbers and assigns them to the driver.
device drivers of new hardware devices no longer require an assignment in the official registry of device numbers; they can simply use whatever numbers are currently available in the system.
Dynamic device file creation
The Linux kernel can create the device files dynamically.
udev
A set of User Mode programs, collectively known as the udev toolset, must be installed in the system. At the system startup the /dev directory is emptied, then a udev program scans the subdirectories of /sys/class looking for
the dev files. For each such file, which represents a combination of major and minor number for a logical device supported by the kernel, the program creates a corresponding device file in /dev. It also assigns device filenames and creates symbolic links according to a configuration file.Eventually, /dev is filled with the device files of all devices
supported by the kernel on this system, and nothing else.
Eventually, /dev is filled with the device files of all devices supported by the kernel on this system, and nothing else.
VFS Handling of Device Files
Device files live in the system directory tree but are intrinsically different from regular files and directories. When a process accesses a regular file, it is accessing some data blocks in a disk partition through a filesystem; when a process accesses a device file, it is just driving a hardware device. To do this, the VFS changes the default file operations of a device file when it is opened.
Device Drivers
A device driver is the set of kernel routines that makes a hardware device respond to the programming interface defined by the canonical set of VFS functions (open, read, lseek, ioctl, and so forth) that control a device.
Device Driver Registration
struct device_driver
Device Driver Initialization
Registering a device driver and initializing it are two different things. A device driver is registered as soon as possible, so User Mode applications can use it through the corresponding device files. In contrast, a device driver is initialized at the last possible moment. In fact, initializing a driver means allocating precious resources of the system, which are therefore not available to other drivers.
Monitoring I/O Operations
the device driver that started an I/O operation must rely on a monitoring technique that signals either the
termination of the I/O operation or a time-out. The two techniques available to monitor the end of an I/O operation are called the polling mode and the interrupt mode.
Polling mode
busy-loop within predefined count or loop with scheduler()
Interrupt mode
(the device provide the interrupt handler which can wake up the sleeping device-access ops)
Interrupt mode can be used only if the I/O controller is capable of signaling, via an IRQ line, the end of an I/O operation.
dev_read() Executes wait_event_interruptible to suspend the process until the intr flag becomes 1. After some time, our device issues an interrupt to signal that the I/O operation is completed and that the data is ready in the proper DEV_FOO_DATA_PORT data port.
Accessing the I/O Shared Memory
//ops
ioremap(),ioremap_nocache()
// create a new vm_struct descriptor for a linear address interval that has the size of the required I/O shared memory area.
readb(), readw(), readl()
//Reads 1, 2, or 4 bytes, respectively, from an I/O shared memory location
writeb(), writew(), writel()
//Writes 1, 2, or 4 bytes, respectively, into an I/O shared memory location
memcpy_fromio(), memcpy_toio()
//Copies a block of data from an I/O shared memory location to dynamic memory and vice versa
memset_io()
//Fills an I/O shared memory area with a fixed value
Direct meory Access
DMA circuits
auxiliary DMA circuits can transfer data between the RAM and an I/O device. Once activated by the CPU, the DMA is able to continue the data transfer on its own; when the data transfer is completed, the DMA issues an interrupt request.
The conflicts that occur when CPUs and DMA circuits need to access the same memory location at the same time are resolved by a hardware circuit called a memory arbiter.
Because setup time for the DMA is relatively high, it is more efficient to directly use the CPU for the data transfer when the number of bytes is small.(suit to disk drivers)
Synchronous and asynchronous DMA
correspond to Reactor and Proactor
A device driver can use the DMA in two different ways called synchronous DMA and asynchronous DMA. In the first case, the data transfers are triggered by processes; in the second case the data transfers are triggered by hardware devices.
Bus address
before activating the transfer, the device driver must ensure that the DMA circuit can directly access the RAM
locations.
four type of address
1.logical address 2.linear address 3.physical address 4.bus address
bus address is the memory addresses which used by all hardware devices except the CPU to drive the data bus. in a DMA
operation, the data transfer takes place without CPU intervention; the data bus is driven directly by the I/O device and the DMA circuit. Therefore, when the kernel sets up a DMA operation, it must write the bus address of the memory buffer
involved in the proper I/O ports of the DMA or I/O device.
Cache coherency
the DMA helper functions must take into consideration the hardware cache when implementing DMA mapping operations.(In case of that when device fetch data from RAM, the data still in the cache and not update to the memory)
Device driver developers may handle DMA buffers in two different ways by making use of two different classes of helper functions(two different DMA mapping types).
- Coherent DMA mapping
the kernel ensures that there will be no cache coherency problems between the memory and the hardware device; this means that every write operation performed by the CPU on a RAM location is immediately visible to the hardware device, and vice versa. This type of mapping is also called “synchronous” or “consistent.”
- Streaming DMA mapping(higher priority option)
When using this mapping, the device driver must take care of cache coherency problems by using the proper synchronization helper functions. This type of mapping is also called “asynchronous” or “non-coherent.”
Helper functions for coherent DMA mappings
Usually, the device driver allocates the memory buffer and establishes the coherent DMA mapping in the initialization phase; it releases the mapping and the buffer when it is unloaded.
pci_alloc_consistent() dma_alloc_coherent()
Helper functions for streaming DMA mappings
Memory buffers for streaming DMA mappings are usually mapped just before the transfer and unmapped thereafter.
//allocate bus address
pci_map_single()
dma_map_single()
/*ache coherency*/
//before device read invokes this to flush the cache lines corresponding to the DMA buffer.
pci_dma_sync_single_for_device()
dma_sync_single_for_device()
//before driver reads invokes this to invalidate the corresponding hardware cache lines.
pci_dma_sync_single_for_cpu()
dma_sync_single_for_cpu()
Even buffers in high memory can be used for DMA transfers; the developer uses pci_map_page()—or dma_map_page()—passing to it the descriptor address of the page including the buffer and the offset of the buffer inside the page. to release the mapping of the high memory buffer, the developer uses pci_unmap_page() or dma_unmap_page().
Levels of Kernel Support
the iopl() and ioperm() system calls grant a process the privilege to access I/O ports.