Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Synchronous RNG: Writing the Driver

In the previous chapter we accessed the EDU-RNG-SYNC device directly from code inserted in kernel_init. This confirmed that we can talk to the device correctly, but it is not how a real driver works: our test code was not reusable, was not exposed to user-space applications, and lived in the wrong place (the kernel’s boot sequence, rather than a dedicated driver file).

We will now write a proper Linux driver for the EDU-RNG-SYNC device. This driver will:

  1. Be a self-contained piece of kernel code, compiled as part of the kernel image, that initialises itself once at boot.
  2. Map the device’s registers into kernel virtual memory, exactly like we did in the previous part of the exercise.
  3. Expose the device to user-space applications as a character device, i.e. a special file (also called pseudo-file), typically under /dev, that applications can open() and interact with.
  4. Define a small custom interface, based on ioctl(), so that user space can ask the driver to generate random numbers or to seed the generator.

Where the Driver Lives in the Kernel Source Tree

Real Linux drivers do not live inside core kernel files such as init/main.c; they live in their own source files, generally under drivers/, grouped by subsystem (drivers/net for network cards, drivers/gpu for graphics cards, etc.). Our RNG device does not fit any existing subsystem, so, like many simple/educational devices, we will place it in drivers/misc, a directory reserved (per its own Makefile’s comment) for “misc devices that really don’t fit anywhere else”.

Create a new, empty file for our driver:

root@container:~/workspace/linux-6.6# touch drivers/misc/edu-rng-sync.c

For the kernel build system to actually compile this file into the kernel image, you need to reference it from drivers/misc/Makefile. Open that file and add the following line, near the top:

obj-y += edu-rng-sync.o

obj-y vs obj-m. In the kernel build system, obj-y means “always compile this file into the kernel image” (the code is built-in), whereas obj-m means “compile this file separately as a dynamically loadable module” (a .ko file inserted at runtime with insmod and removed with rmmod). Since the deliverable for this exercise is a single kernel source patch, and to keep things simple, we compile our RNG driver as built-in, using obj-y. One consequence is that our driver will always be initialised, once, very early during every boot of the VM, and that it can never be unloaded at runtime.

From now on, every time you run make in linux-6.6, this new file will be compiled (as long as it contains valid C) and linked into arch/x86_64/boot/bzImage, just like any other kernel source file.

Recap: Kernel Modules, module_init and module_exit

Despite being built-in rather than dynamically loadable, we still write our driver using the standard Linux kernel module API. Every Linux driver, whether built-in or loadable, follows the same basic skeleton:

  • An initialisation function, registered with the module_init() macro, called exactly once, either at boot (built-in code) or when the module is loaded (insmod, loadable code). This is where a driver typically discovers/maps its device and registers itself with the relevant kernel subsystems.
  • A cleanup function, registered with the module_exit() macro, called when the compute shuts down ((built-in code) or when the module is removed (rmmod). Because our driver is built-in, nothing will ever call this function while the VM runs. We still define it, both as good practice and because it is expected of every properly structured Linux driver.

Mapping the Device’s Registers

The first part of our driver’s initialisation function is the same thing you already did in the previous chapter: map the device’s MMIO area with ioremap, using the BAR (physical base address) you previously determined with lspci -v, and store the resulting kernel virtual address in a global variable so that the rest of the driver (in particular, the ioctl handler we will write below) can use it:

#include <linux/ioctl.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/fs.h>
#include <linux/uaccess.h>
#include <linux/io.h>

#define DEVICE_BASE_PHYS_ADDR   /* TODO complete here */

void *devmem = 0x0;

static int __init edu_rng_sync_init(void) {
    devmem = ioremap(DEVICE_BASE_PHYS_ADDR, 4096);

    if(!devmem) {
        printk(KERN_ERR "Failed to map device registers in memory");
        return -1;
    }

    /* the rest of initialisation goes here, see below */

    printk(KERN_INFO "edu_rng_sync driver loaded\n");
    return 0;
}

Notice the optional macro coming before the first parameter passed to printk, indicating the level of log: information, error, warning, etc.

Exposing the Device as a Character Device

Now that we can talk to the device’s registers from kernel code, we need a way for user-space applications to trigger these MMIO accesses. Linux drivers expose themselves to user space through device pseudo files, special entries typically found under /dev (e.g. /dev/sda, /dev/tty0). There are several kinds of device files; we are interested in character devices here, which are accessed sequentially, one “unit” of data at a time (as opposed to block devices, such as disks, addressable in fixed-size blocks).

Whenever user space performs a system call (open, read, write, ioctl, close, …) on a device file, the kernel’s virtual filesystem layer forwards it to the driver responsible for that device, by calling the corresponding function pointer stored in a struct file_operations:

static struct file_operations my_fops = {
    .unlocked_ioctl = my_ioctl,
};

We will implement my_ioctl in a moment. For now, note that we only fill in the .unlocked_ioctl field: our device does not need .open, .read, .write, etc., since all interactions with it will happen through a single custom ioctl interface (see below for why).

To make the kernel aware of our character device, we must register it, from our init function, with the register_chrdev function, which takes a major number, a name, and a file_operations structure.

Every character device is identified by a pair of (major, minor) numbers: the major number identifies the driver responsible for the device, and the minor number distinguishes between several devices handled by the same driver. We only have one device, so we won’t worry about minor numbers here. Some major numbers are reserved for well-known devices; by convention, a range of numbers is set aside for local/experimental drivers such as ours. For this exercise we simply pick an arbitrary, currently unused major number, e.g. 255. At the top of the source file near the other #define, add the following constant definition:

#define CHARDEV_MAJOR_NUMBER    255

Add the registration call to edu_rng_sync_init, right after the ioremap call:

if (register_chrdev(CHARDEV_MAJOR_NUMBER, "edu_rng_sync", &my_fops) < 0) {
    printk(KERN_ERR "Failed to register edu_rng_sync\n");
    return -1;
}

The string "edu_rng_sync" is simply the name that will appear for our driver in /proc/devices; it is not the same thing as the path of a device file in /dev. register_chrdev does not create that device file for us: we will have to create it ourselves, manually, with the mknod command, in the VM once the driver is loaded. This is covered in the next part of the exercise, when we start writing user-space code that needs to open() the device.

Defining an ioctl-Based Interface

Our device supports exactly two operations, “generate and return a random number” and “seed the generator”, neither of which maps naturally onto the generic read()/write() system calls (in particular, “generate a number” is triggered by reading a device register, but is conceptually closer to a command than to reading a byte stream). This kind of situation, a device-specific command with optional input and/or output data, is precisely what the ioctl() system call was designed for.

An ioctl() call from user space looks like ioctl(fd, request, arg), where fd is the open file descriptor for our device, request is a driver-specific numeric command, and arg is an optional argument, typically a pointer to a user space value/data structure for the driver to read from and/or write to. On the kernel side, our .unlocked_ioctl handler receives the same cmd and arg, and decides what to do based on cmd.

Kernel convention (see the kernel documentation on ioctl numbers) is to build request/cmd values with the _IO, _IOR, _IOW, _IOWR macros, which encode a “magic” character identifying our driver, a sequence number, and the size/direction of the associated data. Define our two commands right after the includes:

#define EDU_RNG_SYNC_IOCTL_RAND _IOR('q', 1, unsigned int)
#define EDU_RNG_SYNC_IOCTL_SEED _IOW('q', 1, unsigned int)

EDU_RNG_SYNC_IOCTL_RAND uses _IOR (“read”, i.e. data flows from the kernel to user space): calling it makes the driver return a random number to the caller. EDU_RNG_SYNC_IOCTL_SEED uses _IOW (“write”, data flows from user space to the kernel): calling it lets the caller supply a new seed value. In both cases the associated data is a single unsigned int, matching the width of the device’s registers. If you expand these macros (or simply print their value on the kernel log), you will find that EDU_RNG_SYNC_IOCTL_RAND evaluates to 0x80047101 and EDU_RNG_SYNC_IOCTL_SEED to 0x40047101: the top bits encode the direction (read/write), the next bits the size of the data (4 bytes), then the magic character 'q' (0x71), and finally the sequence number (1).

These two constants need to be known both by the driver (to implement my_ioctl) and by any user-space application wanting to use the device (to know what value to pass to ioctl()). For now, we simply duplicate these #defines in the user-space test application we will write in the next part of the exercise; sharing a single definition between kernel and user space is left as a known limitation that we will address soon in an enhanced version of the driver.

Implementing the ioctl Handler

We can now write my_ioctl, the function referenced by .unlocked_ioctl above:

static long my_ioctl(struct file *file, unsigned int cmd, unsigned long arg) {
    unsigned int data = 0x0;

    switch (cmd) {
        // Get a new random number
        case EDU_RNG_SYNC_IOCTL_RAND:
            /* Application requests a new random number */
            /* TODO implement that feature */

            break;

        // Seed the RNG
        case EDU_RNG_SYNC_IOCTL_SEED:
            /* Application requests to seed the RNG */
            /* TODO implement that feature */

            break;

        default:
            return -ENOTTY; // unknown command
    }

    return 0;
}

Here the cmd parameter contains the exact ioctl command that was issued by the application. We use a switch statement to separate the processing according to what the application requests, either EDU_RNG_SYNC_IOCTL_RAND or EDU_RNG_SYNC_IOCTL_SEED.

It will be your responsibility to implement these commands. Draw inspiration from how we accessed the device in the previous section of this exercise. A few important points to note:

  • When either reading or writing data from/to the device through ioctl, the arg parameter will contain:
    • The address in user space of the data to write to the device, in the case of a write operation.
    • The address in user space where the data to be read should be stored, in the case of a read operation.
  • It is unsafe for the kernel to read/write from/to user-space addresses directly: a malicious user-space application could have, for example, passed a NULL with the aim to crash to the kernel. To properly access these addresses, you need to use copy_to_user when copying data read from the device into user-space memory, and copy_from_user when reading data from user space in order to write it to the device. See here for these functions’ prototypes and examples of their usage. Both functions perform various security checks on the user space buffer for the kernel to access.

Cleaning Up: the Exit Function

To complete the driver, add the cleanup function, undoing everything the init function set up, in reverse order:

static void __exit edu_rng_sync_exit(void) {
    unregister_chrdev(CHARDEV_MAJOR_NUMBER, "edu_rng_sync");

    if(devmem)
        iounmap(devmem);

    printk(KERN_INFO "edu_rng_sync driver unloaded\n");
}

And register both functions with the kernel module API, at the end of the file:

module_init(edu_rng_sync_init);
module_exit(edu_rng_sync_exit);

Building and Verifying the Driver Loads

With the code of your driver complete, rebuild the kernel and relaunch the VM as usual:

root@container:~/workspace/linux-6.6# make

Once the VM has booted, check the kernel log for the message our driver prints on initialisation:

alpine:~# dmesg | grep edu_rng_sync
[    x.xxxxxx] edu_rng_sync driver loaded

You can also confirm that the character device was correctly registered, with the major number you chose, by inspecting /proc/devices:

alpine:~# cat /proc/devices | grep edu_rng_sync
255 edu_rng_sync

If both checks succeed, the driver is correctly loaded and registered. Note that there is, at this stage, still no /dev/edu_rng_sync file: nothing has created it yet, and no user-space application can talk to the driver. We fix this next and write a small test application exercising both ioctls.