Ker­nel fea­tures patch­set (from GSoC 2010)

I am play­ing around with the patch­set “my” stu­dent gen­er­ated dur­ing this years GSoC (the code for all pro­jects is avail­able from Google). In short, it gives you the pos­sib­il­ity to query from user­land, which op­tion­al ker­nel fea­tures are avail­able. I have let him mostly do those fea­tures, which are not so easy to de­tect from user­land, or where the de­tec­tion could trig­ger an auto­load of a ker­nel mod­ule.

I let the out­put speak for him­self, first the out­put be­fore his patch­set:

kern.features.compat_freebsd7: 1
kern.features.compat_freebsd6: 1
kern.features.posix_shm: 1

And now with his patch­set:

kern.features.compat_freebsd6: 1
kern.features.compat_freebsd7: 1
kern.features.ffs_snapshot: 1
kern.features.geom_label: 1
kern.features.geom_mirror: 1
kern.features.geom_part_bsd: 1
kern.features.geom_part_ebr: 1
kern.features.geom_part_ebr_compat: 1
kern.features.geom_part_mbr: 1
kern.features.geom_vol: 1
kern.features.invariant_support: 1
kern.features.kdtrace_hooks: 1
kern.features.kposix_priority_scheduling: 1
kern.features.ktrace: 1
kern.features.nfsclient: 1
kern.features.nfsserver: 1
kern.features.posix_shm: 1
kern.features.pps_sync: 1
kern.features.quota: 1
kern.features.scbus: 1
kern.features.softupdates: 1
kern.features.stack: 1
kern.features.sysv_msg: 1
kern.features.sysv_sem: 1
kern.features.sysv_shm: 1
kern.features.ufs_acl: 1

With his patches we have a total of 84 ker­nel fea­tures which can be quer­ied (ob­vi­ously I do not have all op­tion­al op­tions en­abled in the ker­nel which pro­duces this out­put). All of the fea­tures also have a de­scrip­tion, and it is easy to add more fea­tures. As an ex­ample I present what is ne­ces­sary to pro­duce the kern.features.stack out­put:

./kern/subr_stack.c:FEATURE(stack, “Sup­port for cap­tur­ing ker­nel stack”);

There is also a little user­land ap­plic­a­tion (and a lib­rary in­ter­face) which al­lows to query sev­er­al fea­tures from scripts/​applications with the pos­sib­il­ity to pre­tend a fea­ture is not there (the re­quire­ment for this was for ports; pre­tend­ing a fea­ture is there if it is not was ruled out be­cause such run-​time de­tec­tion is only ne­ces­sary for things which have to run soon and pre­tend­ing some fea­ture is there while it is not will cause big prob­lems). Un­for­tu­nately the man page for the ap­plic­a­tion is not yet ready, but I’m sure you can fig­ure out how to use it.

The names of the fea­tures and the de­scrip­tion fol­lows an easy scheme, what is writ­ten down in NOTES is used as a name and a de­scrip­tion for the fea­ture (an ex­cep­tion is geom_​part_​X, there we de­cided to use a com­mon theme (“GEOM par­ti­tion­ing class for XXX”) which is dis­tinct from the cor­res­pond­ing geom_​X class). If you have com­plains about what is used in a spe­cif­ic fea­ture, do not com­plain to him: change it in NOTES and the fea­ture will fol­low.

If you have ques­tions, sug­ges­tions, or some oth­er in­terest to con­tact him, his FreeBSD ad­dress is kibab@. Feel free to en­cour­age him to go ahead with the next steps (fin­ish­ing the man page, split­ting up the patches in­to sens­ible pieces and present­ing them on ap­pro­pri­ate mailing­lists for re­view). 🙂

In­ter­est­ing pro­jects in the GSoC

I coun­ted 18 pro­jects which are giv­en to FreeBSD in this years GSoC. For 3 of them I have some com­ments.

Very in­ter­est­ing to me is the pro­ject which is named Col­lect­ive lim­its on set of pro­cesses (a.k.a. jobs). This looks a bit like the Sol­ar­is contract/​project IDs. If this pro­ject res­ults in some­thing which al­lows the user­land to query which PID be­longs to which set, than this al­lows some nice im­prove­ment for start scripts. For ex­ample at work on Sol­ar­is each ap­plic­a­tion is a mix of sev­er­al pro­jects (apache = “name:web” pro­ject, tom­cat = “name:app” pro­ject, Or­acle DB = “name:ora” pro­ject). Our man­age­ment frame­work (writ­ten by a co-​worker) al­lows to eas­ily do some­thing with those pro­jects, a “show” dis­plays the prstat (sim­il­ar to top) info just for pro­cesses which be­long to the pro­ject, a “kill” sends a kill-​signal to all pro­cesses of the pro­ject, and so on. We could do some­thing sim­il­ar with our start scripts by de­clar­ing a namespace (FreeBSD:base:XXX /​ FreeBSD:ports:XXX?) and maybe num­ber space (de­pend­ing on the im­ple­ment­a­tion) as re­served and use it to see if pro­cesses which be­long to a par­tic­u­lar script are still run­ning or kill them or whatever.

The oth­er two pro­jects I want to com­ment upon here are Com­plete libp­kg and cre­ate new pkg tools and Com­plete Pack­age sup­port in the pkg_​install tools and cleanup. Both pro­jects ref­er­ence libp­kg in their de­scrip­tion. I hope the ment­ors of both pro­jects pay some at­ten­tion to what is go­ing on in the oth­er pro­ject to not cause dependencies/​clashes between the stu­dents.

That I do not men­tion oth­er pro­jects does not mean that they are not in­ter­est­ing or sim­il­ar, it is just that I do not have to say some­thing valu­able about them…

Ment­or­ing again in the GSoC

Seems that I will act­ively ment­or again in this Google Sum­mer of Code (as op­posed to just re­view the sub­mis­sions from stu­dents and/​or act­ing as a fall-​back ment­or).

The pro­ject I will ment­or is the “Make op­tion­al ker­nel sub­sys­tems re­gister them­selves via sy­sctl”-one from the FreeBSD ideas page.

The stu­dent already got in­to con­tact with me and it looks like he is mo­tiv­ated (he is already sub­scribed to sev­er­al FreeBSD mailing­lists, which is not a re­quire­ment we have in our GSoC docs).

Google’s new RE en­gine

I stumbled over Google’s new RE en­gine. Un­for­tu­nately it is not hand­ling back­refer­ences, so it is not a drop-​in re­place­ment for the reg­u­lar ex­pres­sions code in FreeBSD. It has a POSIX mode, but this only seems to be enough for the egrep syn­tax. For people which need back­refer­ences, they refer to the Google Chrome’s RE en­gine ir­reg­exp which in turn ref­er­ences a pa­per from 2007 which is titled Reg­u­lar Ex­pres­sion Match­ing Can Be Simple And Fast.

The tech­niques in the pa­per can not be ap­plied to the ir­reg­exp en­gine, but maybe could help to speed up awk, egrep and sim­il­ar pro­grams.

I think it would be in­ter­est­ing to com­pare those re­cent de­vel­op­ments to what we have in FreeBSD, and if they are faster, to see if it is pos­sible to im­prove the FreeBSD im­ple­ment­a­tion based upon them (either by writ­ing new code, or by im­port­ing ex­ist­ing code, de­pend­ing on the cor­res­pond­ing li­cense and the lan­guage the code is writ­ten in).

Maybe a can­did­ate for the GSoC?

Catch­ing up… linuxu­lat­or.

The linuxu­lat­or is synced on amd64 with i386 (since a while). This means TLS is work­ing now and we have the same (a little bit buggy) fu­texes.

Ro­man is slowly work­ing on the *at() com­mands. He also ap­plied for the GSoC this year again. Kib is will­ing to ment­or (in case Ro­man gets a free seat in the SoC). I re­jec­ted the ment­or­ing po­s­i­tion this time, as I don’t know if I will have enough time this sum­mer, but I hope I will be around.