Advanced Systems Programming - Lesson 18: Kernel timing issues

Kernel timing issues An introduction to the use of kernel timers and work queues Kernel timers • Linux offers a facility that lets drivers put a process to sleep until a fixed amount of time has elapsed (as measured in jiffies) • When the timer expires, a driver-defined action will be performed, which can ‘wake up’ the process that was put to sleep, or could perform some alternative action (for example, the kernel timer could re-start) jiffies • unsigned long volatile jiffies; • global kernel variable (used by scheduler) • initialized to zero when system reboots • gets incremented during a timer interrupt • so it counts ‘clock-ticks’ since cpu restart • ‘tick-frequency’ is a ‘configuration’ option • On our machines: HZ=250 (in ‘.config’) jiffies overflow • Won’t overflow for at least 16 months • Linux kernel got modified to ‘fix’ overflow • Now the declaration is in ‘linux/jiffies.h’: unsigned long long jiffies_64; and a new instruction in ‘do_timer()’ (*(u64*)&jiffies_64)++; which compiles to assembly language as add $1, jiffies+0 adc $0, jiffies+4 Kernel timer syntax • Declare a timer: struct timer_list mytimer; • Initialize this timer: init_timer( &mytimer ); mytimer.func = mytimeraction; mytimer.data = (unsigned long)mydata; mytimer.expires = • Install this timer: add_timer( &mytimer ); • Modify this timer: mod_timer( &mytimer, ); • Delete this timer: del_timer( &mytimer ); • Delete it safely: del_timer_sync( &mytimer); A kernel-timer caution • A kernel timer’s timeout-action cannot do anything that could cause the current task to ‘sleep’ (such as copying data between user-space and kernel-space, or trying to allocate more kernel memory) • However, to aid debugging, a timer CAN use ‘printk()’ within its timeout-routine ‘trytimer.c’ • We have posted an example that shows how a Linux kernel timer can be used to perform a periodic action (such as using ‘printk()’ to issue a message every time the time expires, then restart the timer • Notice that our demo is not able to issue messages directly to the console – its timer-function executes without a ‘tty’ Delaying work • If a device-driver needs to perform actions that require using process resources (like a tty), or that may possibly ‘sleep’, then it can defer that work – using a ‘workqueue’ Programming syntax • Declare: struct workqueue_struct *myqueue; struct work_struct thework; • Define: void dowork( void *data ) { /* actions */ }; • Initialize: myqueue = create_singlethread_workqueue( “mywork” ); INIT_WORK( &thework, dowork, ); • Schedule: queue_dalayed_work( myqueue, &thework, ); • Cleanup: if ( !cancel_delayed_work( &thework ) ) flush_workqueue( myqueue ); destroy_workqueue( myqueue ); ‘tryworkq.c’ and ‘defermsg.c’ • We have posted demo-modules that show the use of workqueues to perform actions later, either as soon as a ‘process context’ is available, or after a prescribed time • Further details on the options for using an existing kernel workqueue or a workqueue of your own creation may be found in our textbook (Chapter 7 of LDD3) Applying these ideas • To demonstrate a programming situation in which using kernel timers is valuable, we created the ‘foo.c’ device-driver, plus an application that uses it (‘watchfoo.cpp’) • You can compile and install the module, then execute the application: $ watchfoo • But you will notice there are two ‘problems’ (excess cpu usage and loop-termination) Reducing CPU’s usage • The ‘watchfoo’ program rereads ‘/dev/foo’ constantly (numerous times per second), much faster than the human eye can see • If you run the ‘top’ utility, you will see that a high percentage of the available CPU time is being consumed by ‘watchfoo’ • You can add a kernel timer to the ‘foo.c’ driver to curtail this excessive reading In-class exercise • Modify ‘foo.c’ (call it ‘timedfoo.c’) as follows • Create an integer flag-variable (‘ready’) as a global object in your module • When your ‘read()’ function gets called, it should sleep until ‘ready’ equals TRUE; it should set ‘ready’ equal to FALSE when it awakens, but should set a timer to expire after 1/10 seconds • Your timer’s action-function should set ‘ready’ back to TRUE, then wake up any sleeping tasks Implementation hints • You need a wait-queue (so your driver’s ‘reader’ tasks can sleep on it) • You need a timer action-function • You need to organize your timer-function’s data-items into a single structure (because the timer-function has only one argument) • Your timer-function must do two things: – Change ‘ready’ to TRUE – Wake up any ‘sleepers’ Deferring work • Linux supports a ‘workqueue’ mechanism which allows the kernel to defer specified work until some later point in time • This mechanism has been ‘reworked’ in a major way since our texts were published • So any device-driver has to be modified if it made any use of a kernel ‘workqueue’ • Changes require grasp of some ‘macros’ ‘sizeof’ and ‘offsetof’ • Our GNU compilers permit use of these C/C++ operators on object types • The ‘sizeof’ operator returns the number of bytes of memory the compiler allocated for storing the specified object • The ‘offsetof’ operator returns the number of bytes in a structure which precede the specified structure-member A ‘struct’ example struct mystruct { char w; short x; long y; long long z; } my_instance; You can use the ‘sizeof’ operator to find out how much memory gets allocated to any ‘struct mystruct’ object, like this: int nbytes = sizeof( my_instance ); You can use the ‘offsetof’’ operator to find out where within a given structure a particular field occurs, like this: int offset_z = offsetof( struct mystruct, z ); The ‘container_of()’ macro • Recent versions of the Linux kernel have introduced a further operator on ‘structs’ container_of( ptr, type, member ); • When given a pointer to a field within a structure-object, and the type-name for that that structure-object, and the field- name for that structure’s field-member, then it returns the structure’s address Using ‘container_of()’ struct mystruct { char w; short x; long y; long long z; } my_instance = { 1, 2, 3, 4 }; If you have a pointer to one of the fields in some instance of a this kind of ‘struct’ object, then you could use the ‘container_of()’ macro to get a pointer to that ‘struct’ object itself, like this: long *ptr = &my_instance.y; struct mystruct *p = container_of( ptr, struct mystruct, y ); This would be useful if you now wanted to access other members: printk( “w=%d x=%d y=%d z=%d \n”, p->w, p->x, p->y, p->z ); • #include • void dowork( struct work_struct *data ); • DECLARE_DELAYED_WORK( mywork, dowork ); • struct workqueue_struct *myqueue; • myqueue = create_singlethread_workqueue( “mywork” ); ‘workqueue’ syntax #include struct workqueue_struct *myqueue; // pointer to your workqueue void dowork( struct work_struct *data ); // your function’s prototype DECLARE_DELAYED_WORK( mywork, dowork ); int init_module( void ) { myqueue = create_singlethread_workqueue( “mywork” ); if ( !queue_delayed_work( myqueue, &mywork, HZ*5 ) ) return –EBUSY; return 0; // SUCCESS } void cleanup_module( void ) { destroy_workqueue( myqueue ); } ‘tryworkq.c’ void dowork( struct work_struct *data ) { printk( “\n\n I am doing the delayed work right now \n” ); } In this example the delayed work consists of simply printing a message to the kernel’s logfile -- you can view by typing the ‘dmesg’ command Notice that the ‘action’ function in this example ignores its ‘data’ argument An improved example • Our ‘announce.c’ module shows how an LKM could display its messages within a window on the Linux graphical desktop • It uses the ‘tty_struct’ object which exists in the process-descriptor for the ‘insmod’ task which you launch to install the LKM • We shall see how this same idea can be used in a waitqueue’s ‘action’ functions ‘timer’ verses ‘workqueue’ • Any kernel-timer’s action-function will be executed in ‘atomic’ context – just like an interrupt service routine: it cannot ‘sleep’, and it cannot access any user-space data • But any workqueue’s action-function will be executed by a kernel-thread – and thus it possesses a ‘process’ context, so it can be ‘scheduled’ and ‘sleep’ if necessary – though it, too, cannot access user-space If ‘dowork()’ needs data // data items needed by your ‘dowork’ function are packaged in a ‘struct’ struct mydata { char *msg; struct tty_struct *tty; } my_data = { “\nHello\n”, NULL }; // your module-initialization function sets up your ‘struct delayed_work’ object // and it can also finish initializing your ‘my_data’ object’s member-fields myqueue = create_singlethread_workqueue( “mywork” ); INIT_DELAYED_WORK( &mywork, dowork ); my_data.tty = current->signal->tty; // then your action-function can access members of your ‘my_data’ object like this void dowork( struct work_struct *data ) { struct mydata *dp = container_of( &my_data.msg, struct mydata, msg ); struct tty_struct *tty = dp->tty; tty->driver->write( tty, dp->msg, strlen( dp->msg ) ); } ‘defermsg.c’ • This LKM will display a message within a desktop window after a 10-second delay • It illustrates a use of the ‘container_of()’ macro (as is needed by the reworked API for the Linux kernel’s workqueues) • Our course-website has a link to an online article by author Jonathan Corbet giving details of ongoing kernel changes in 2.6 Summary of tonight’s demos • ‘foo.c’ and ‘watchfoo.cpp’ • ‘announce.c’ • ‘trytimer.c’ • ‘trymacro.c’ • ‘tryworkq.c’ • ‘defermsg.c’ • EXERCISE: Modify the ‘foo.c’ device-driver to use a kernel timer in it’s ‘read()’ method