I stum­bled over Google’s new RE engine. Unfor­tu­nately it is not han­dling back­ref­er­ences, so it is not a drop-in replace­ment for the reg­u­lar expres­sions code in FreeBSD. It has a POSIX mode, but this only seems to be enough for the egrep syn­tax. For peo­ple which need back­ref­er­ences, they refer to the Google Chrome’s RE engine irreg­exp which in turn ref­er­ences a paper from 2007 which is titled Reg­u­lar Expres­sion Match­ing Can Be Sim­ple And Fast.

The tech­niques in the paper can not be applied to the irreg­exp engine, but maybe could help to speed up awk, egrep and sim­i­lar programs.

I think it would be inter­est­ing to com­pare those recent devel­op­ments to what we have in FreeBSD, and if they are faster, to see if it is pos­si­ble to improve the FreeBSD imple­men­ta­tion based upon them (either by writ­ing new code, or by import­ing exist­ing code, depend­ing on the cor­re­spond­ing license and the lan­guage the code is writ­ten in).

Maybe a can­di­date for the GSoC?

This WE I was told that FreeNAS seems to want to move from FreeBSD to Linux (since then it seems there could be a linux and a FreeBSD ver­sion). One of the rea­sons seems to be a miss­ing sen­sors framework.

As I was com­mit­ting a port of the OpenBSD sen­sors frame­work (pro­duced as part of the Google Sum­mer of Code 2007) to FreeBSD and had to remove it after­wards because one com­mit­ter com­plained very loudly, I was asked what the sta­tus of this is.

The short sta­tus is: Nobody is doing some­thing about it.

Before I explain the long sta­tus, I give  a short overview what this sen­sors frame­work is:

• a ker­nel API which allows to add sensors
• an inter­face for the user­land to query the sen­sor data
• some basic user­land code to show and log the sen­sor info

The API and the query inter­face are more or less inde­pen­dent. For the user­land code it was more a log­ging infra­struc­ture than a real mon­i­tor­ing solu­tion. The rea­son was the real mon­i­tor­ing solu­tions already exist (Nagios, snmpd, …) and can be adapted to query the sen­sors. Ide­ally a query in user­land should be han­dled by a library instead of directly access­ing the sysctl inter­face, this way the kernel<->userland inter­face would be abstracted away (and could b replaced as needs arise). This was not done, it was some­thing to be done later (Rome was not build in a day).

The user­land inter­face also only cared about dumb sen­sors (those which you need to query man­u­ally to get the infor­ma­tion), smart sen­sors (those which are able to send events them­self) where not taken care about in the sense of really send­ing sensor-triggered events, but the ker­nel API allowed to add such sen­sors. The sysctl inter­face has no way of send­ing events, but FreeBSD already has an event inter­face (devd is tak­ing care about it). It would have been not a prob­lem to send events via this chan­nel and let an user­land library take care about the deliv­ery together with other sensor-data in userland.

And now the long sta­tus is:

PHK com­plained loudly about it. First he said he did not look at it but he com­plained that is not good regard­less. After a lot of nag­ging from me he had a look at it and was not happy about the time stuff in it (short: the FreeBSD time­counter code is bet­ter). This was not a prob­lem in my opin­ion, we could have dis­abled this part with­out prob­lems. After such an offer from me, he com­plained that the sen­sors frame­work uses the sysctl inter­face instead of an entry in /dev.

At this point in time already sev­eral user­land util­i­ties used the sysctl frame­work to query for sta­tus data in the ker­nel. So there was already prece­dence for such an use of it. Later some more such uses where added too (e.g. the proc­stat stuff by core team mem­ber Robert Watson).

I saved some of the cor­re­spond­ing mails (to pub­lic mail­ing lists) in a mbox file, read the mess your­self if you want.

The bot­tom line is: Sev­eral com­mit­ters (even some which we could call high pro­file com­mit­ters) told me that they do not see a prob­lem in the use of the sysctl inter­face. They do not seem to want to tell it in pub­lic (nobody of them voiced their opin­ion in the thread, so do not ask me who those peo­ple are). I am not inter­ested in invest­ing more of my spare time into fight­ing wind­mills (it looks like this to me).

So, if some­one is inter­sted in the code, r172631 has it. In the per­force repos­i­tory you can maybe find some sen­sors. I think most of it can still be used with­out much changes.

If some­one tries it with a more recent FreeBSD, please drop me a note if it just applies fine, or a patch (or an URL to it) if it needs some mod­i­fi­ca­tions. Who knows, maybe in a future project it may be use­ful for me.

If there is enough inter­est by sev­eral peo­ple, I can even put up a wiki page where those peo­ple can coor­di­nate, but that is most prob­a­bly all I am will­ing to invest fur­ther into this (at least in my unpaid time).

Addi­tion­ally to the WP plu­g­ins I already talked about, I installed some more since then:

• aLinks: auto­matic link gen­er­a­tion based upon rules/modules/keywords
• All in one SEO Pack: Search Engine Opti­miza­tion, e.g. auto­matic meta tag gen­er­a­tion and more
• Global Trans­la­tor: auto­matic machine–trans­la­tion of your posts into other lan­guages (see the coun­try flags in the sidebar)
• Google XML Sitemaps: auto­mat­i­cally gen­er­ates a sitemap and announces the update (after a post) to sev­eral searchengines
• http:BL Word­Press plu­gin: checks vis­i­tors against the Project Honey Pot black­list (email har­vest­ing) and rejects access
• InfoLink: adds a but­ton to look up marked text in the edi­tor and link the markup to the first result if desired
• One-Time Pass­word: allows to use one–time pass­words (RFC 2289) for the WP login
• Smartlinker: another but­ton for the edi­tor regard­ing auto­matic link­ing (gives other auto­matic links than the other auto­matic link­ing plugins)
• Update Noti­fier: sends me a mail when there is a update for WP
• WP-Print: printer friendly page of the post­ing (see the link below the title of this post), I had to patch it to not print the links from the WP AJAX Edit Com­ments plugin
• WP AJAX Edit Com­ments: pro­vides an AJAX edit inter­face for comments

We got a lot of good pro­pos­als. Google is will­ing to give us a very nice amount of stu­dents. We didn’t expected this much. Thanks!

Now we need to rate the stu­dent appli­ca­tions and find suit­able men­tors… not that easy. It’s easy for the strongest pro­pos­als, but for the rest I expect that there will be some shuf­fling around until the very end.

Now that we have (or “soon will have”) an offi­cial FreeBSD blog, I decided to give this kind of elec­tronic and pub­lic diary a try. At least to report the sta­tus and progress of some FreeBSD related projects I’m involved in.

A lot hap­pened in the last days. We’re past the dead­line of the Google Sum­mer of Code rat­ing process and now the lucky stu­dents are cho­sen. It wasn’t easy. We had more than 120 pro­pos­als (out of ~6400). From those we where will­ing to men­tor 40 – 50 (based upon our resources and the qual­ity of the pro­pos­als). Google granted 14 to us (a big thank you to Google!). I expect an offi­cial announce­ment “soon”. Hope­fully some of those stu­dents Google wasn’t able to fund are will­ing to work with us regard­less of the money, there are a lot of very nice pro­pos­als (I will add some items to the ideas list based upon them later). We’re at least will­ing to pro­vide the same amount of men­tor­ship as if they where selected.

I will work with two stu­dents. One of them will work on sync­ing the OSS API from recent releases from 4Front with our sound sys­tem. The other one will work on improv­ing the lin­ux­o­la­tor. I will announce this on the cor­re­spond­ing mail­inglists after the offi­cial announce­ment. ATM we need to do some admin­is­tra­tive stuff (hand­ing out access to the wiki and per­force, set­ting up email aliases, …).

I already mailed some gen­eral guides to the stu­dents (note to men­tors: it’s in the wiki on the “hid­den” and restricted to men­tors SoC page). For the sound stuff I don’t have a nice TODO list, but luck­ily the stu­dent already inves­ti­gated the 4Front stuff for the pro­posal and is eager to start the work (basi­cally this gives us the user­land inter­face for multi-channel mixer sup­port). Regard­ing the lin­ux­o­la­tor improve­ments the stu­dent has a lit­tle bit less luck. I com­piled a nice TODO list which is based upon his own pro­posal and all the var­i­ous things I know about the lin­ux­o­la­tor (PR’s, mes­sages on emulation@, pri­vate con­ver­sa­tions, things we noticed while work­ing on the update of the linux base to a recent Fedora Core, …). I will be impressed if he man­ages to do every­thing on the TODO list (don’t worry, he knows that not every­thing has to be done to declare “suc­cess” to Google).

Both of them may be can­di­dates for a com­mit bit (not within the SoC, but maybe later), both are eager to do the work, inter­ested, moti­vated, and don’t need hand-holding.

Fur­ther news in the area of the sound sys­tem: Ariff told me he is work­ing on multi-channel and endi­aness issues/support, and I may be able to com­mit two more sound dri­vers to the tree. One dri­ver is the emu10kx dri­ver cur­rently avail­able in the Ports Col­lec­tion, and the other one is a dri­ver for some envy24 chips. Cur­rently I’m in con­tact with the author of the emu10kx dri­ver and a vol­un­teer who wants to improve an exist­ing envy24 dri­ver (author of the dri­ver con­tacted; since it’s BSD licensed, this is a “don’t be evil” action).

And some news regard­ing the user­land part of the lin­ux­o­la­tor: We’re wait­ing for a repo-copy form linux_base-fc3 to linux_base-fc4. It seems FC4 is com­pat­i­ble enough so that we can dire­clty move from Red­Hat 8 to FC 4. FC 5 is out of the game for a while, it doesn’t want to run with the old linux ker­nel ver­sion our lin­ux­o­la­tor announces. Chang­ing the ver­sion via the sysctl doesn’t help (prob­a­bly because of the changed seman­tic of the linux clone syscall between 2.4.x and 2.6.x), so it has to wait until the lin­ux­o­la­tor SoC finishes.