A while ago I committed the linuxulator D-Trace probes I talked about earlier. I waited a little bit for this announcement to make sure I have not broken anything. Nobody complained so far, so I assume nothing obviously bad crept in.
The >500 probes I committed do not cover the entire linuxulator, but are a good start. Adding new ones is straight forward, if someone is interested in a junior–kernel-hacker task, this would be one. Just ask me (or ask on emulation@), and I can guide you through it.
In case you have not noticed yet, KDTRACE_HOOKS is now in the GENERIC kernel in FreeBSD-current. This means you just need to load the DTrace modules and can use DTrace with the GENERIC kernel.
In case you do not know what you can do with DTrace, take the time to have a look at the DTrace blog. It is worth any minute you invest reading it.
This weekend I made some progress in the linuxulator:
- I MFCed the reporting of some linux-syscalls to 9–stable and 8-stable.
- I updated my linuxulator-dtrace patch to a recent -current. I already compiled it on i386 and arundel@ has it compiled on amd64. I counted more than 500 new DTrace probes. Now that DTrace rescans for SDT probes when a kernel module is loaded, there is no kernel panic anymore when the linux module is loaded after the DTrace modules and you want to use DTrace. I try to commit this at a morning of a day where I can fix things during the day in case some problems show up which I did not notice during my testing.
- I created a PR for portmgr@ to repocopy a new linux_base port.
- I set the expiration date of linux_base-fc4 (only used by 7.x and upstream way past its EoL) and all dependent ports. It is set to the EoL of the last 7.x release, which can not use a later linux_base port. I also added a comment which explains that the date is the EoL of the last 7.x release.
I got a little bit of time to update my 3 year old work of adding static DTrace probes to the linuxulator.
The changes are not in HEAD, but in my linuxulator-dtrace branch. The revision to have a look at is r230910. Included are some DTrace scripts:
- script to check internal locks
- script to trace futexes
- script to generate stats for DTracified linuxulator parts
- script to check for errors:
- emulation errors (unsupported stuff, unknown stuff, …)
- kernel errors (resource shortage, …)
- programming errors (errors which can happen if someone made a mistake, but should not happen)
The programming-error checks give hints about userland programming errors respectively a hint about the reason of error return values due to resource shortage or maybe a wrong combination of parameters. An example error message for this case is “Application %s issued a sysctl which failed the length restrictions.\nThe length passed is %d, the min length supported is 1 and the max length supported is %d.\n”.
The stats-script (tailored specially to the linuxulator, but this can easily be extended to the rest of the kernel) can report about:
- number of calls to a kernel function per executable binary (not per PID!): allows to see where an optimization would be beneficial for a given application
- graph of CPU time spend in kernel functions per executable binary: together with the number of calls to this function this allows to determine if a kernel optimization would be beneficial / is possible for a given application
- graph of longest running (CPU-time!) kernel function in total
- timing statistics for the emul_lock
- graph of longest held (CPU-time!) locks
Unfortunately this can not be committed to HEAD as-is. The DTrace SDT provider can not handle probes which are added to the kernel after the SDT provider is already loaded. This means that you either have to compile the linuxulator statically into the kernel, or you have to load the SDT kernel module after the linuxulator module is loaded. If you do not respect this, you get a kernel panic on first access of one of the providers in the linuxulator (AFAIR this includes listing the probes available in the kernel).