A phoronix bench­mark cre­ates a huge bench­mark­ing discussion

The recent Phoronix bench­mark which com­pared a release can­di­date of FreeB­SD 9 with Ora­cle Lin­ux Serv­er 6.1 cre­at­ed a huge dis­cus­sion in the FreeB­SD mail­inglists. The rea­son was that some peo­ple think the num­bers pre­sent­ed there give a wrong pic­ture of FreeB­SD. Part­ly because not all bench­mark num­bers are pre­sent­ed in the most promi­nent page (as linked above), but only at a dif­fer­ent place. This gives the impres­sion that FreeB­SD is infe­ri­or in this bench­mark while it just puts the focus (for a rea­son, accord­ing to some peo­ple) on a dif­fer­ent part of the bench­mark (to be more spe­cif­ic, blog­bench is doing disk reads and writes in par­al­lel, FreeB­SD gives high­er pri­or­i­ty to writes than to reads, FreeB­SD 9 out­per­forms OLS 6.1 in the writes while OLS 6.1 shines with the reads, and only the reads are pre­sent­ed on the first page). Oth­er com­plaints are that it is told that the default install was used (in this case UFS as the FS), when it was not (ZFS as the FS).

The author of the Phoronix arti­cle par­tic­i­pat­ed in parts of the dis­cus­sion and asked for spe­cif­ic improve­ment sug­ges­tions. A FreeB­SD com­mit­ter seems to be already work­ing to get some issues resolved. What I do not like per­son­al­ly, is that the arti­cle is not updat­ed with a remark that some things pre­sent­ed do not reflect the real­i­ty and a retest is necessary.

As there was much talk in the thread but not much obvi­ous activ­i­ty from our side to resolve some issues, I start­ed to improve the FreeB­SD wiki page about bench­mark­ing so that we are able to point to it in case some­one wants to bench­mark FreeB­SD. Oth­ers already chimed in and improved some things too. It is far from per­fect, some more eyes – and more impor­tant­ly some more fin­gers which add con­tent – are need­ed. Please go to the wiki page and try to help out (if you are afraid to write some­thing in the wiki, please at least tell your sug­ges­tions on a FreeB­SD mail­inglist so that oth­ers can improve the wiki page).

What we need too, is a wiki page about FreeB­SD tun­ing (a first step would be to take the man-page and con­vert it into a wiki page, then to improve it, and then to feed back the changes to the man-page while keep­ing the wiki page to be able to cross ref­er­ence parts from the bench­mark­ing page).

I already told about this in the thread about the Phoronix bench­mark: every­one is wel­come to improve the sit­u­a­tion. Do not talk, write some­thing. No mat­ter if it is an improve­ment to the bench­mark­ing page, tun­ing advise, or a tool which inspects the sys­tem and sug­gests some tun­ing. If you want to help in the wiki, cre­ate a First­name­Last­name account and ask a FreeB­SD comit­ter for write access.

A while ago (IIRC we have to think in months or even years) there was some frame­work for auto­mat­ic FreeB­SD bench­mark­ing. Unfor­tu­nate­ly the author run out of time. The frame­work was able to install a FreeB­SD sys­tem on a machine, run some spec­i­fied bench­mark (not much bench­marks where inte­grat­ed), and then install anoth­er FreeB­SD ver­sion to run the same bench­mark, or to rein­stall the same ver­sion to run anoth­er bench­mark. IIRC there was also some DB behind which col­lect­ed the results and maybe there was even some way to com­pare them. It would be nice if some­one could get some time to talk with the author to get the frame­work and set it up some­where, so that we have a con­trolled envi­ron­ment where we can do our own bench­marks in an auto­mat­ic and repeat­able fash­ion with sev­er­al FreeB­SD versions.

Rants about JASS (Solaris Secu­ri­ty Toolkit)

Recent­ly I switched to a new client where the Solaris Secu­ri­ty Toolk­it (JASS) is exten­sive­ly used. I am now in the process of updat­ing some things, among them are JET and JASS. As part of this work I reeval­u­ate the local JASS mod­i­fi­ca­tions. Pre­vi­ous­ly a cus­tom JASS pack­age was used, but in case JASS is updat­ed by Ora­cle at some point in time (and an update is real­ly need­ed, see below), this would need some amount of work to find out the dif­fer­ences and to for­ward port them to the new ver­sion. If every­thing is well doc­u­ment­ed, this should not be hard to do, but the per­son doing the work also needs to find the up-to-date docs.

To make it more easy I decid­ed to change this. I now install the offi­cial JASS pack­age via JET togeth­er with the lat­est patch for it, and then let JET copy our mod­i­fi­ca­tions over the installed pack­age. Instead of mod­i­fy­ing exist­ing dri­vers, I cre­at­ed our own dri­vers with a ref­er­ence to the dri­ver which served as a base.

While doing this I encoun­tered sev­er­al short­com­ings of JASS on Solaris 10.

There are sev­er­al FS based checks which do not make sense to do for the FS of zones in a glob­al zone (at least not the way I use JASS, so maybe a con­fig­urable way of chang­ing the behav­ior should serve for every­one). If zones are installed in /zones, you do not need to check for files with­out valid UIDs (you sure­ly find a lot of files, as the users are defined inside the zones and not in the glob­al zone) or sim­i­lar things (even not for world writable files, as the zones are installed in a root-access-only sub­tree and inside the zones there may be oth­er secu­ri­ty con­straints con­fig­ured inside JASS, read: it is the respon­si­bil­i­ty of JASS inside the zone to do this). An easy solu­tion would be to exclude those FS which con­tain zones (and as we only have one sub­tree, I just hard­cod­ed this in sev­er­al scripts).

I also miss the pos­si­bil­i­ty (maybe I over­looked a sim­ple way) for the ssh check to lim­it the Allow­Root­Lo­gin to spe­cif­ic hosts. JASS only checks yes or no, but can not lim­it it to spe­cif­ic hosts (e.g. via “Match IP/hostname”). Often you do not need to per­mit root-logins (RBAC/sudo/…), but some­times it is the only way to han­dle a par­tic­u­lar edge-case (or to speed up an action dra­mat­i­cal­ly), and in such cas­es you do not want to allow root-logins more than necessary.

OCP: Ora­cle Solaris 10 Sys­tem Administrator

After work­ing a long time with Solaris (I start­ed with 2.5.1 at about the time when Solaris 7 was released in 1998), my cur­rent boss decid­ed that it is time that I do a cer­ti­fi­ca­tion for it (clients like it). Two exams lat­er I have it now:

Oracle Certified Professional, Oracle Solaris 10 System Administrator

Since yes­ter­day I am offi­cial­ly an Ora­cle Cer­ti­fied Pro­fes­sion­al, Ora­cle Solaris 10 Sys­tem Admin­is­tra­tor.

The exam ques­tions where a bit strange. I asked myself if a real admin was proof read­ing them or not, but most prob­a­bly some­one with­out much knowl­edge about Solaris admin­is­tra­tion just took the study guides and tried to make some ques­tions out of it.

Any­way, my boss should be hap­py now, and I have some­thing to add to my CV.

HeatMaps again…

Today I stum­bled again over some HeatMaps from Bren­dan Gregg (of DTrace-fame). This time it was the PDF of his pre­sen­ta­tion at the LISA 2010 con­fer­ence. It shows nice­ly how he plans to evolve it from a single-machine (like in Ana­lyt­ics for Ora­cle Stor­age prod­ucts) to the cloud. It is a very good overview about what kind of intu­itive per­for­mance visu­al­iza­tion you can do with this.

I would love to see some­thing for FreeB­SD (or oth­er per­for­mance data). Maybe some­one could take the DTrac­eTaz­Tool as a base or hint and write some­thing gener­ic which works for a lot of things…

There are just too much nice and inter­est­ing things out there, and not enough time for all of them.

Rant about Berke­ley­DB docs

I was build­ing Berke­ley­DB (4.7, yes I know, there are more recent ver­sions avail­able) on a Solaris machine. First try was to unpack, cd into the direc­to­ry, run con­fig­ure. It failed, there is no con­fig­ure script. Bah. 🙁

Sec­ond try: search­ing for docs… found some… in HTML (the README refers to it and tells noth­ing else). This is a remote machine, I do not want to use a HTML brows­er remote­ly (I may not even have one installed there…). Bah. 🙁

Ok, dist/configure exists, no spe­cial options need­ed for my case, it seems.

There is even a Solaris spe­cif­ic HTML file, but from a quick glance at it with ‘less’, it looks like a FAQ.

Usabil­i­ty from a com­mand line: zero.
Pos­si­bil­i­ty to com­pile from a GUI (unix): I doubt it.

What is wrong with plain text files? If I down­load the source and want to com­pile it (and for Solaris this is the nor­mal way of work­ing), why the hell do I need some GUI instead of get­ting a plain text file with the required descrip­tion (which is not graph­i­cal­ly enhanced in the HTML ver­sion either)? You can even gen­er­ate a plain text ver­sion of the docs auto­mat­i­cal­ly dur­ing the src-packaging process.

Hey Ora­cle, there is room for improve­ment here!