This week­end I made some progress in the lin­ux­u­la­tor:

• I MFCed the report­ing of some linux-syscalls to 9-stable and 8-stable.
• I updated my lin­ux­u­la­tor-dtrace patch to a recent -cur­rent. I already com­piled it on i386 and arundel@ has it com­piled on amd64. I counted more than 500 new DTrace probes. Now that DTrace res­cans for SDT probes when a ker­nel mod­ule is loaded, there is no ker­nel panic any­more when the linux mod­ule is loaded after the DTrace mod­ules and you want to use DTrace. I try to com­mit this at a morn­ing of a day where I can fix things dur­ing the day in case some prob­lems show up which I did not notice dur­ing my testing.
• I cre­ated a PR for portmgr@ to repocopy a new linux_base port.
• I set the expi­ra­tion date of linux_base-fc4 (only used by 7.x and upstream way past its EoL) and all depen­dent ports. It is set to the EoL of the last 7.x release, which can not use a later linux_base port. I also added a com­ment which explains that the date is the EoL of the last 7.x release.

Last week­end I com­mit­ted some dummy-syscalls to the lin­ux­u­la­tor in FreeBSD-cur­rent. I also added some com­ments to syscalls.master which should give a hint which linux ker­nel had them for the first time (if the linux man–page I looked this up in is cor­rect). So if some­one wants to exper­i­ment with a higher compat.linux.osrelease than 2.6.16 (as it is needed for a Cen­tOS based linux_base), he should now get some ker­nel mes­sages about unim­ple­mented syscalls instead of a silent fail­ure.

There may be some low-hanging fruits in there, but I did not really ver­ify this by check­ing what the dummy syscalls are sup­posed to do in linux and if we can eas­ily map this to exist­ing FreeBSD fea­tures. In case some­one has a look, please send an email to emu­la­tion@FreeBSD.org.

It seems my HOWTO cre­ate a new linux_base port was not too bad. There is now a PR for a Cen­tOS 6 based linux_base port. I had a quick look at it and it seems that it is nearly usable to include into the Ports Col­lec­tion (the SRPMs need to be added, but that can be done within some minutes).

When FreeBSD 8.3 is released and the Ports Col­lec­tion open for sweep­ing com­mits again, I will ask port­mgr to do a repo-copy for the new port and com­mit it. This is just the linux_base port, not the com­plete infra­struc­ture which is needed to com­pletely replace the cur­rent default lin­ux­u­la­tor user­land. This is just a start. The process of switch­ing to a more recent linux_base port is a long process, and in this case depends upon enough sup­port in the sup­ported FreeBSD releases.

Atten­tion: Any­one installing the port from the PR should be aware that using it is a highly exper­i­men­tal task. You need to change the lin­ux­u­la­tor to imper­son­ate him­self as a linux 2.6.18 ker­nel (described in the pkg-message of the port), and the code in FreeBSD is far from sup­port­ing this. Any­one who wants to try it is wel­come, but you have to run FreeBSD-current as of at least the last week­end, and watch out for ker­nel mes­sages about unsup­ported syscalls. Reports to emulation@FreeBSD.org please, not here on the webpage.

I got a lit­tle bit of time to update my 3 year old work of adding sta­tic DTrace probes to the lin­ux­u­la­tor.

The changes are not in HEAD, but in my linuxulator-dtrace branch. The revi­sion to have a look at is r230910. Included are some DTrace scripts:

• script to check inter­nal locks
• script to trace futexes
• script to gen­er­ate stats for DTracified lin­ux­u­la­tor parts
• script to check for errors:
  • emu­la­tion errors (unsup­ported stuff, unknown stuff, …)
  • ker­nel errors (resource shortage, …)
  • pro­gram­ming errors (errors which can hap­pen if some­one made a mis­take, but should not happen)

The programming-error checks give hints about user­land pro­gram­ming errors respec­tively a hint about the rea­son of error return val­ues due to resource short­age or maybe a wrong com­bi­na­tion of para­me­ters. An exam­ple error mes­sage for this case is “Appli­ca­tion %s issued a sysctl which failed the length restric­tions.nThe length passed is %d, the min length sup­ported is 1 and the max length sup­ported is %d.n”.

The stats-script (tai­lored spe­cially to the lin­ux­u­la­tor, but this can eas­ily be extended to the rest of the ker­nel) can report about:

• num­ber of calls to a ker­nel func­tion per exe­cutable binary (not per PID!): allows to see where an opti­miza­tion would be ben­e­fi­cial for a given application
• graph of CPU time spend in ker­nel func­tions per exe­cutable binary: together with the num­ber of calls to this func­tion this allows to deter­mine if a ker­nel opti­miza­tion would be ben­e­fi­cial / is pos­si­ble for a given application
• graph of longest run­ning (CPU-time!) ker­nel func­tion in total
• tim­ing sta­tis­tics for the emul_lock
• graph of longest held (CPU-time!) locks

Unfor­tu­nately this can not be com­mit­ted to HEAD as-is. The DTrace SDT provider can not han­dle probes which are added to the ker­nel after the SDT provider is already loaded. This means that you either have to com­pile the lin­ux­u­la­tor sta­t­i­cally into the ker­nel, or you have to load the SDT ker­nel mod­ule after the lin­ux­u­la­tor mod­ule is loaded. If you do not respect this, you get a ker­nel panic on first access of one of the providers in the lin­ux­u­la­tor (AFAIR this includes list­ing the probes avail­able in the kernel).

After giv­ing an overview of the in-kernel basics of the Lin­ux­u­la­tor, I want now to describe how to add sup­port for new ioctls to the Lin­ux­u­la­tor.

Where are the files to modify?

The plat­form inde­pen­dent code for the ioctls is in SRC/sys/compat/linux/linux_ioctl.c. The defines to have names for the ioctl val­ues are in SRC/sys/compat/linux/linux_ioctl.h.

How to mod­ify them?

First of all cre­ate a new header which will con­tain all the struc­tures, named val­ues and macros for those new ioctls. As writ­ten above, the ioctl val­ues (e.g. #define LINUX_VIDIOC_ENCODER_CMD 0x564d /* 0xc028564d */) do not belong there, they shall be added to linux_ioctl.h. Dur­ing the course of adding sup­port for ioctls, you will need this new header. Add it in the SRC/sys/compat/linux/ direc­tory, and pre­fix the name with a linux_. It would be good to decide on a com­mon tag here (ref­er­enced as yourtag in the fol­low­ing), and stay with it. Use it wher­ever you need to have some spe­cific name for the ioctl-set you want to add. In this case it would result in linux_yourtag.h (or even linux_ioctl_yourtag.h, depend­ing if this is used for some­thing very spe­cific to the ioctls, or some generic linux fea­ture) as the name of the header file. This was not done in the past, so do not expect that the names inside the linux_ioctl.c file will be con­sis­tent to this nam­ing scheme, but it is never too late to cor­rect mis­takes of the past (at least in Open Source soft­ware development).

Now add this header to linux_ioctl.c (you need to include compat/linux/linux_yourtag.h). After that add the ioctl val­ues to linux_ioctl.h. As can be seen above, the defines should be named the same as on linux, but with a LINUX_ pre­fix (make sure they where not defined before some­where else). The ioctl val­ues need to be the same hex val­ues as in Linux, off course. Sort them accord­ing to their hex value. When you added all, you need to add two more defines. The LINUX_IOCTL_yourtag_MIN and LINUX_IOCTL_yourtag_MAX ones. The MIN-one needs to be an alias for the first (sorted accord­ing to the hex value) ioctl you added, and MAX needs to be an alias for the last (again, sorted accord­ing to the hex value) ioctl you added.

The next step is to let the Lin­ux­u­la­tor know that it is able to han­dle the ioctls in the LINUX_IOCTL_yourtag_MIN to LINUX_IOCTL_yourtag_MAX range. Search the sta­tic linux_ioctl_function_t sec­tion of linux_ioctl.c and add such a vari­able for your ioctl set. The name of the vari­able should be some­thing like linux_ioctl_yourtag.

Sim­i­lar for the handler-definition for this. Search the sta­tic struct linux_ioctl_handler sec­tion and add a yourtag_handler. Set it to { linux_ioctl_yourtag, LINUX_IOCTL_yourtag_MIN, LINUX_IOCTL_yourtag_MAX }. To make this han­dler known to the Lin­ux­u­la­tor, you need to add it to the DATA_SET sec­tion. Add DATA_SET(linux_ioctl_handler_set, yourtag_handler) there.

Now the meat, the func­tion which han­dles the ioctls. You already defined it as linux_ioctl_function_t, but now you need to write it. The out­line of it looks like this:

static int
linux_ioctl_yourtag(struct thread *td, struct linux_ioctl_args *args)
{
        struct file *fp;
        int error;
        switch (args->cmd & 0xffff) {
        case LINUX_an_easy_ioctl:
                break;
        case LINUX_a_not_so_easy_ioctl:
                /* your handling of the ioctl */
                fdrop(fp, td);
                return (error);
        /* some more handling of your ioctls */
        default:
       return (ENOIOCTL);
        }
        error = ioctl(td, (struct ioctl_args *)args);
        return (error);
}

An easy ioctl in the switch above is an ioctl where you do not have to do some­thing but can pass the ioctl through to FreeBSD itself. The not so easy ioctl case is an ioctl where you need to do e.g. a fget(td, args->fd, &fp). This is just an exam­ple, there are also other pos­si­bil­i­ties where you need to do addi­tional stuff before the return, or where you do not pass the ioctl to FreeBSD. A typ­i­cal exam­ple of what needs to be done here is to copy val­ues from linux struc­tures to FreeBSD struc­tures (and the other way too), or to trans­late between 64bit and 32bit. Linux pro­grams on amd64 are 32bit exe­cuta­bles and 32bit structures/pointers. To make this work on amd64, you need to find a way to map between the two. There are exam­ples in the ker­nel where this is already the case. The more promi­nent exam­ples in the 64bit<->32bit regard are the v4l and v4l2 ioctls.

The tedious part is to research if a trans­la­tion has to be done and if yes what needs to be trans­lated how. When this is done, most of the work is not so hard. The linux_yourtag.h should con­tain the struc­tures you need for this trans­la­tion work.

It is also pos­si­ble to add ioctls in a ker­nel mod­ule, but this is not sub­ject to this descrip­tion (I will update this post­ing with a link to it when I get time to write about it).