Solaris: script to check var­i­ous set­tings of a sys­tem if they com­ply to some pre-defined settings

Prob­lem

If you set­up a sys­tem, you want to make sure that it com­plies to a pre-defined con­fig. You can do that with some con­fig­u­ra­tion man­age­ment sys­tem, but there are cas­es where it use­ful to do that out­side of this context.

Solu­tion

The shell script below I start­ed to write in 2008. Over time (until 2016) it extend­ed into some­thing which is able to out­put a report of over 1000 items. You can con­fig­ure it via ${HOME}/.check_host.cfg and /etc/check_host.cfg (it reads both in this order, first con­fig wins and oth­er con­fig is not read). You can use option “-h” to see the usage text. Option “-n” sup­press­es mes­sages which help to fix issues, “-a” prints out sim­ple HTML instead of text.

Solaris: script to cre­ate com­mands to set­up LDOMs based upon out­put from “ldm ls”

Prob­lem

You have a LDOM which you want to clone to some­where else and all you have to per­form that is the ldm com­mand on the tar­get system.

Solu­tion

Down­load the AWK script below. Use the out­put of “ldm ls ‑l ‑p <ldom>” as the input of this AWK script. The out­put will be a list of com­mands to re-create the con­fig for VDS, VDISK, VSW and NETWORK.

I wrote this in 2013, so changes to the out­put of “ldm ls” since then are not account­ed for.

Self-signed cer­tifi­cates and LDAPS (OpenL­DAP) in PHP (or python)

This is not about how to gen­er­ate a self-signed cer­tifi­cate, this is about how to con­fig­ure an ldap client to con­nect secure­ly to a ldap serv­er which has a self-signed certificate.

Recent­ly I was search­ing a lot how to make this kind of set­up work, but it seems nobody is using the key­words of the head­line in their HOW­TOs, or every­one is not real­ly set­ting up a real­ly secure con­nec­tion with self-signed cer­tifi­cates. As such here my try to doc­u­ment this for those which are inter­est­ed in a secure setup.

How OpenL­DAP is check­ing the cer­tifi­cates normally

OpenL­DAP is using the cer­tifi­cate store which is con­fig­ured for OpenSSL. So any cer­tifi­cate which is signed by one of the CAs in the OpenSSL cert-ctore are trusted.

Secure set­up

Most of the time you do not expose an LDAP serv­er to the out­side where a cer­tifi­cate from one of the trusted-by-default CAs is need­ed. A cer­tifi­cate from your inter­nal CA is enough, and in some cas­es a self-signed cer­tifi­cate is suf­fi­cient too.

An easy solu­tion could be to add either the root-certificate of your CA or the self-signed cer­tifi­cate into the trust-store of OpenSSL (not every OS / dis­tri­b­u­tion has this in the same loca­tion, you have to check where this is for your OS, for FreeB­SD 13+ this is /usr/local/etc/ssl/certs/, see also certctl(8) there). But this would mean you trust the cer­ti­ti­fa­cate which you put there addi­tion­al­ly to the default cer­tifi­cates (mod­u­lo any black­list­ing you made your­self). The­o­ret­i­cal­ly this means any­one who is able to get hold of a cer­tifi­cate from a public-CA for your LDAP serv­er, could per­form a man-in-the-middle attack (you need to con­sid­er your­self how fea­si­ble this is in your infra­struc­ture set­up and how like­ly this is to happen).

More secure operation

Let’s say you run a ser­vice which needs to be able to make TLS ses­sions to sys­tems which use cer­tifi­cates from pub­lic CAs and you want to make sure a con­nec­tion to the LDAP back­end can not use cer­tifi­cates from pub­lic CAs.

To tight­en the set­up in this case, you need to spec­i­fy that the client which uses OpenLDAP-client libraries is using a dif­fer­ent trust-store for the cer­tifcate validation.

For the openl­dap client util­i­ties there is a glob­al con­fig file for this (on FreeB­SD this is /usr/local/etc/openldap/ldap.conf). For oth­er tools, like PHP, this needs to be done in the per-user con­fig file ~/.ldaprc. Both file have the same syntax.

With php-ldap you nor­mal­ly run the ser­vice either in php-fpm or in an apache-php-module. In both cas­es the process which runs is con­fig­ured to run as a non-root user which may or may not have a home direc­to­ry (in FreeB­SD the www user which is typ­i­cal­ly used for that has no home directory).

HOWTO

  1. cre­ate a home directory
  2. cre­ate a sep­a­rate trust-store for LDAP
  3. con­fig­ure php-ldap / py-ldap to make use of the sep­a­rate trust-store

Step 1 – cre­ate a home directory

Chose a place which is suit­able, and cre­ate a direc­to­ry there. It does­n’t need to be in /home, it can be any­where. The impor­tant part is, that it is read­able by the user which runs the appli­ca­tion which is using php-ldap. It does not need to be writable by this user. In there you need to cre­ate the .ldaprc file (again, needs only be read­able by the user) with the con­tent from step 3.

Step 2 – cre­ate a sep­a­rate trust-store for LDAP

In FreeB­SD the glob­al ldap con­fig is in /usr/local/etc/openldap/ldap.conf. The­o­ret­i­cal­ly you can put the trust-store for LDAP in any place wou want. In my set­up I con­sid­er it to belong into /usr/local/etc/openldap/ssl/. So make a direc­to­ry – like /usr/local/etc/openldap/ssl – for the trust-store, and copy the cer­tifi­cate of the LDAP serv­er there.

Atten­tion! Only the pub­lic cer­tifi­cate, not the pri­vate key! If you only have one file on the serv­er for this, it is the com­bined key+certificate (if you don’t know or are able to deduct by look­ing into the file how to get rid of the key… there is a lot of info out there in the WWW which explains it). The direc­to­ry and the cer­tifi­cate need to be acces­si­ble (read for the file, exe­cute for the direc­to­ry) by any user which shall make use of this. It does not hurt to have it acces­si­ble by every­one (you made sure there is not the private-key from the serv­er, right?).

Step 3 – con­fig­ure php-ldap / py-ldap to make use of the sep­a­rate trust-store

If you use php-fpm, you need to con­fig­ure a home direc­to­ry in the FPM pool con­fig­ure­ation sec­tion. As already said above, it does not need to be inside /home, but it dpends upon your needs. Here in this exam­ple let me use /home. The FPM con­fig line to add is then some­thing like:
env[HOME] = /home/php-fpm
You could achieve the same via chang­ing the home direc­to­ry in the pass­word data­base, but this would have an effect on all process­es run with this user, where­as here it is just for the php-fpm process­es (and childs).

If you use apache instead of php-fpm, you need to con­fig­ure some­thing sim­i­lar for the cor­re­spond­ing vir­tu­al host:
SetEnv HOME /home/php-fpm

With this you can now con­fig­ure /home/php-fpm/.ldaprc to point to the LDAP trust-store:
TLS_CACERT /usr/local/etc/openldap/ssl/ldap_server_cert.pem
TLS_CACERTDIR /usr/local/etc/openldap/ssl

If you use some python based appli­ca­tion, you have to do some­thing sim­i­lar… if all else fails, it needs to be via a real home direc­to­ry in the pass­word database.

If you want to use the ldap client tools with any user, you need to add those lines to the /usr/local/etc/openldap/ldap.conf file too (there you can also set the default BASE – e.g. “BASE dc=example,dc=com” – and URI – e.g. “URI ldaps://ldap.example.com:639″).

After restart­ing php-fpm or apache, you should now be able to make real­ly secure con­nec­tions to the ldap server.

Some impor­tant things

  • Every time you change the cer­tifi­cate of the LDAP serv­er, you need to update the cer­ti­facte in the clients.
  • There are two TLS modes for the LDAP serv­er, one is “ldaps”, and one is “ldap+starttls”. If you have your LDAP serv­er run­ning in ldaps-mode (typ­i­cal­ly on port 639), you do not need to spec­i­fy in your php-ldap using appli­ca­tion to enable TLS (which is doing a start­tls after con­nect­ing… typ­i­cal­ly on port 389), but you need to spec­i­fy “ldaps://servername:639” (assum­ing it runs on port 639) instead of just “server­name” at the place in your appli­ca­tion where you are told to enter the serv­er name. For py-ldap I have checked just one appli­ca­tion (net­da­ta), and there TLS needs to be enabled, and the serv­er name has to be with­out “ldaps://” as net­da­ta is pre­fix­ing the “ldaps://” itself if tls is enabled.
  • Some places in the inter­net are telling to add “TLS_REQCERT nev­er” into ldap.conf / .ldaprc. Tech­ni­cal­ly this is not need­ed. Depend­ing on your point of view this can either be good or bad (spec­i­fy­ing it saves some CPU cycles on the serv­er and the client, and some trans­fer time over the net­work – not spec­i­fy­ing it allows to val­i­date the cer­tifi­cat­ed received to be com­pared to the cer­tifcate being avail­able local­ly, but I do not know if OpenL­DAP is doing this, nor did I spend some time to eval­u­ate if this improves secu­ri­ty (if the impor­tant parts of the cer­tifi­cate are out-of-sync, the con­nec­tion will fail)).

Fight­ing the Coro­n­avirus with FreeB­SD (Folding@Home)

Pho­to by Fusion Med­ical Ani­ma­tion on Unsplash

Here is a quick HOWTO for those which want to pro­vide some FreeB­SD based com­pute resources to help find­ing vac­cines. I have not made a port out of this and do not know yet if I get the time to make one. If some­one wants to make a port, go ahead, do not wait for me.

UPDATE 2020-03-22: 0mp@ made a port out of this, it is in “biology/linux-foldingathome”.

  • Down­load the lin­ux RPM of the Folding@Home client (this cov­ers fah­client only).
  • Enable the lin­ux­u­la­tor (ker­nel moduls and linux_base (first part of chap­ter 10.2) is enough).
  • Make sure linprocfs/linsysfs are mount­ed in /compat/linux/{proc|sys}.
  • cd /compat/linux
  • tar -xf /path/to/fahclient....rpm
  • add the “fah­client” user (give it a real home directory)
  • make sure there is no /compat/linux/dev or alter­na­tive­ly mount devfs there
  • mkdir /compat/linux/etc/fahclient
  • cp /compat/linux/usr/share/doc/fahclient/sample-config.xml /compat/linux/etc/fahclient/config.xml
  • chown -R fahclient /compat/linux/etc/fahclient
  • edit /compat/linux/fahclient/config.xml: mod­i­fy user (manda­to­ry) / team (option­al: FreeB­SD team is 11743) / passkey (option­al) as appro­pri­ate (if you want to con­trol the client remote­ly, you need to mod­i­fy some more parts, but some­how the client “los­es” a filedescrip­tor and stops work­ing as it should if you do that on FreeBSD)
  • If you have the home direc­to­ries of the users as no-exec (e.g. seper­ate ZFS datasets with exec=off): make sure the home direc­to­ry of the fah­client user has exec per­mis­sions enabled
  • cd ~fahclient (impor­tant! it tries to write to the cur­rent work direc­to­ry when you start it)
  • Start it: /usr/sbin/daemon /compat/linux/usr/bin/FAHClient /compat/linux/etc/fahclient/config.xml --run-as fahclient --pid-file=/var/run/fahclient.pid >/dev/null 2>&1

Per default it will now pick up some SARS-CoV‑2 (COVID-19) relat­ed fold­ing tasks. There are some more con­fig options (e.g. how much of the sys­tem resources are used). Please refer to the offi­cial Folding@Home site for more infor­ma­tion about that. Be also aware that there is a big rise in com­pute resources donat­ed to Folding@Home, so the pool of avail­able work units may be emp­ty from time to time, but they are work­ing on adding more work units. Be patient.

Strate­gic think­ing, or what I think what we need to do to keep FreeB­SD relevant

Since I par­tic­i­pate in the FreeB­SD project there are from time to time some voic­es which say FreeB­SD is dead, Lin­ux is the way to go. Most of the time those voic­es are trolls, or peo­ple which do not real­ly know what FreeB­SD has to offer. Some­times those voic­es wear blind­ers, they only see their own lit­tle world (were Lin­ux just works fine) and do not see the big pic­ture (like e.g. com­pe­ti­tion stim­u­lates busi­ness, …) or even dare to look what FreeB­SD has to offer.

Some­times those voic­es raise a valid con­cern, and it is up to the FreeB­SD project to fil­ter out what would be ben­e­fi­cial. Recent­ly there were some mails on the FreeB­SD lists in the sense of “What about going into direc­tion X?”. Some peo­ple just had the opin­ion that we should stay were we are. In my opin­ion this is sim­i­lar­ly bad to blind­ly say­ing FreeB­SD is dead and fol­low­ing the mass­es. It would mean stag­na­tion. We should not hold peo­ple back in explor­ing new / dif­fer­ent direc­tions. Some­one wants to write a ker­nel mod­ule in (a sub­set of) C++ or in Rust… well, go ahead, give it a try, we can put it into the Ports Col­lec­tion and let peo­ple get expe­ri­ence with it.

This dis­cus­sion on the mail­inglists also trig­gered some kind of “where do we see us in the next years” / strate­gic think­ing reflec­tion. What I present here, is my very own opin­ion about things we in the FreeB­SD project should look at, to stay rel­e­vant in the long term. To be able to put that into scope, I need to clar­i­fy what “rel­e­vant” means in this case.

FreeB­SD is cur­rent­ly used by com­pa­nies like Net­flix, NetApp, Cis­co, Juniper, and many oth­ers as a base for prod­ucts or ser­vices. It is also used by end-users as a work-horse (e.g. mailservers, web­servers, …). Stay­ing rel­e­vant means in this con­text, to pro­vide some­thing which the user base is inter­est­ed in to use and which makes it more easy / fast for the user base to deliv­er what­ev­er they want or need to deliv­er than with anoth­er kind of sys­tem. And this in terms of time to mar­ket of a solu­tion (time to deliv­er a ser­vice like a web-/mail-/whatever-server or prod­uct), and in terms of per­for­mance (which not only means speed, but also secu­ri­ty and reli­a­bil­i­ty and …) of the solution.

I have cat­e­go­rized the list of items I think are impor­tant into (new) code/features, docs, pol­ish­ing and project infra­struc­ture. Links in the fol­low­ing usu­al­ly point to documentation/HOWTOs/experiences for/with FreeB­SD, and not to the canon­i­cal entry points of the projects or tech­nolo­gies. In a few cas­es the links point to an expla­na­tion in the wikipedia or to the web­site of the top­ic in question.

Code/features

The vir­tu­al­iza­tion train (Open­Stack, Open­Neb­u­la, oVirt, Cloud­Stack, Kuber­netes, Dock­er, Pod­man, …) is run­ning on full speed. The marked is as big/important, that solu­tion providers even do joint ven­tures on cross­ing bor­ders between each oth­ers, e.g. VMware is open­ing up to inte­grate their solu­tion with solu­tions from Amazon/Azure/Google. The under­ly­ing infra­struc­ture is get­ting more and more unim­por­tant, as long as the ser­vices which shall be run per­form as need­ed. Ease of use and time to mar­ket are the key-drivers (the last lit­tle piece of per­for­mance is most­ly impor­tant for com­pa­nies which go to the “edge” (both mean­ings intend­ed in a non-exclusive-or way) like Net­flix for their FreeB­SD based CDN). FreeB­SD is not real­ly par­tic­i­pat­ing in this world. Yes, we had jails way before any­one else out there had some­thing smi­lar, and some even do not have that right now. But if you are real­is­tic, FreeB­SD does not play a major role here. You can do nice things with jails and bhyve, but you have to do it “by hand” (ezjail, iocage and such are improve­ments on the ease of use side, but that is not enough as this is still lim­it­ed on a host cen­tric view). The world has moved on to admin­is­ter­ing a dat­a­cen­ter (to avoid the buzz­words “cloud” or “private-cloud”) with a single-click. In my opin­ion we would need to port sev­er­al of the ini­tialy men­tioned cloud/container man­age­ment solu­tions to FreeB­SD and have them able to han­dle their work via jails and/or bhyve. If FreeB­SD is not able to serve as a build­ing block in this big pic­ture, we will fall off the edge in this par­tic­u­lar IT-area in the long run.

With all the ready-made con­tain­ers avail­able in the inter­net, we should improve our lin­ux­o­la­tor. Our kernel-support for this is lim­it­ed to a 2.6.32-ish ABI ver­sion (it is less than 2.6.32, more like 2.6.16, we are miss­ing epoll and ino­ti­fy sup­port, among oth­ers, but this is the low­est ver­sion glibc in the Cen­tOS 7 based linux_base port is able to run on… and glibc checks the ver­sion num­ber). We need to catch-up to a more recent ver­sion if we want to be able to run those ready-made lin­ux con­tain­ers with­out issue (we can put a lin­ux sys­tem into a jail and start that). If some­one would like to work on that, a good start would be to run the Lin­ux Test Project tests via the lin­ux­u­la­tor and start fix­ing bugs. The last time I did that was in 2007 and about 16% of the test cas­es failed back then. It would be also quite nice if we could inte­grate those lin­ux­u­la­tor tests into the FreeB­SD CI. With improve­ments in the lin­ux­u­la­tor and the above men­tioned vir­tu­al­iza­tion sup­port, we should be able to run more/those linux-images … ehrm, sor­ry, docker/kubernetes/…-containers with­in the linuxulator.

Fin­ish the work regard­ing ker­beros in base. Cy Schu­bert is/was work­ing on this. I do not know the sta­tus of this, but the “fix­ing 800 ports” part in his mail from May 2018 looks like some more help­ing hands would be ben­e­fi­cial. This would bring us to a bet­ter start­ing point for a more seam­less inte­gra­tion (some ports need “the oth­er” kerberos).

We have one port (as far as I was able to deter­mine… the exact amount does not real­ly mat­ter) in terms of SDN – net/openvswitch – but the doc­u­men­ta­tion of this … leaves room for improve­ment (ker­nel sup­port / netmap sup­port and func­tion­al­i­ty / FreeB­SD spe­cif­ic HOWTO). As part of the vir­tu­al­i­sa­tion of every­thing we (yes: we – as part of the FreeB­SD hand­book, see docs cat­e­go­ry below for more on this) need to pro­vide this info so that FreeB­SD is able to par­tic­i­pate in this area. We should also have a look at port­ing some more SDN soft­ware, e.g Open­Con­trail now tung­sten­fab­ric (there is an old con­trail port­ing project), Open­Stack Neu­tron, Open­Day­light, … so that users have a choice, respec­tive­ly FreeB­SD can be inte­grat­ed into exist­ing het­ero­ge­neous environments. 

Sen­sors (tem­per­a­ture, volt­age, fans, …), a top­ic with his­to­ry. Short: in the Google Sum­mer of Code 2007 code was pro­duced, com­mit­ted, and then removed again due to a dis­pute. My per­son­al under­stand­ing (very sim­pli­fied) is “remove every­thing because some of the data han­dled by this frame­work shall not be han­dled by this frame­work” (instead of e.g. “remove sen­sor X, this data shall not be han­dled in this way”), and “remove every­thing as this does not han­dle sen­sors of type X which are not used in servers but in enter­prise class >99% non-IT-related sen­sors”. Noth­ing bet­ter has shown up since then. If I look at Win­dow, VMware, Solaris and Lin­ux, I can query sen­sors on my mainboard/chassis/disks/whatever (yes, I am mix­ing some apples with oranges here), plot them in mon­i­tor­ing sys­tems, and get alarms. In FreeB­SD we fail on this top­ic (actu­al­ly mul­ti­ple top­ics) which I con­sid­er to be some­thing basic and manda­to­ry. I do not sug­gest that we com­mit the Google Sum­mer of Code 2007 code. I sug­gest to have a look at what makes sense to do here. Take the exist­ing code and com­mit it, or improve on this code out­side the tree and then com­mit it, or write some­thing new. In the end it does not mat­ter (for an user) which way it is han­dled, as long as we have some­thing which users can use in the end. It sure­ly makes sense to have an OS-provided frame­work of reg­is­ter­ing sen­sors in a cen­tral place (it would sure­ly be nice if you could get the temp/fan val­ues of your graph­ics card… ooops… sor­ry… AI/HPC accel­er­a­tor togeth­er with oth­er sim­i­lar hard­ware data in your hardware).

To con­tin­ue play­ing well (not only) in the high-availability area, we should also have a look at get­ting an imple­men­ta­tion of MPTCP (Mut­li­path TCP) into the tree. Apple (and oth­ers) is already using it since 2013 with good ben­e­fits (most prob­a­bly not only for Siri users). There exists some code for FreeB­SD, but it is far from usable and it does not look like there is progress since 2016. We say we have the pow­er to serve, but with the cloud­i­fi­ca­tion of the recent years, all users expect that every­thing is always-on and nev­er fails, and being able to pro­vide the serv­er side of this client-server relat­ed tech­nol­o­gy for those peo­ple which have such high demands is nec­es­sary to not fall behind (do not let us rest on our laurels).

Secure­Boot needs also some help­ing hands. At some point oper­at­ing sys­tems which do not sup­port it will not be con­sid­ered by com­pa­nies anymore.

Anoth­er item we should have a look at is to pro­vide means to write ker­nel code in dif­fer­ent lan­guages. Not in the base sys­tem, but at least in ports. If some­one wants to write a ker­nel mod­ule in  C++ or Rust, why not? It offers pos­si­bil­i­ties to explore new areas. There are even reports of expe­ri­ences with dif­fer­ent lan­guages. It does not fit your needs? Well, ignore it and con­tin­ue writ­ing ker­nel code in C, but let oth­er peo­ple which want to use a screw­driv­er instead of a ham­mer do what they want, they will either learn that they should have used a ham­mer, or can report about ben­e­fits about the screwdriver. 

Docs

I think we can improve our end-user docs to the next lev­el. The base sys­tem is already well cov­ered (we can sure­ly find some fea­tures which we could doc­u­ment), but an user does not use FreeB­SD to use FreeB­SD. An user sure­ly has a goal in mind which requires to set­up some kind of ser­vice (mail serv­er, web serv­er, dis­play serv­er (desk­top sys­tem), …). While one could argue that it is the 3rd par­ty project which needs to doc­u­ment how to run their soft­ware on FreeB­SD, I think we need to do our share here too. There are a lot of HOW­TOs for Lin­ux, and then you have to find some tips and tricks to make some­thing work (bet­ter) on FreeB­SD. What I have in mind here is that we should doc­u­ment how to make FreeB­SD par­tic­i­pate in a Win­dows Active Direc­to­ry envi­ron­ment, or in an LDAP envi­ron­ment (as a client), improve the Sam­ba part with FreeB­SD spe­cif­ic parts (like how to make Sam­ba use ZFS snap­shots for Win­dows Shad­ow Copies), con­fig­u­ra­tion man­age­ment tools and so on. I do not talk about pro­vid­ing in-depth docs about the 3rd par­ty soft­ware, but lit­tle HOW­TOs with FreeB­SD spe­cif­ic parts / tips and tricks, and a ref­er­ence to the 3rd par­ty docs. Peo­ple come to us for real-world needs and if we pro­vide them with a head-start of the most com­mon items (e.g. also cov­er­ing nginx or what­ev­er and not only apache httpd) and then guide them to fur­ther docs will improve the val­ue of our hand­book even more for end-users (spe­cial­ly for new­com­ers, but also for expe­ri­enced FreeB­SD users which out of a sud­den now need to do some­thing which they nev­er did before…).

We should also review our docs. The hand­book lists e.g. proc­mail (just an exam­ple…). With proc­mail not being mainained any­more since a long time and known vul­ner­a­bil­i­ties we should replace the info there with info about mail­drop (or any suit­able replace­ment). Care­ful review may also find sim­i­lar items which need some care.

One more item I have in mind in terms of docs for user is the restruc­tur­ing of some parts. Now the world is more think­ing in terms of XaaS (“some­thing as a ser­vice”) we should also have a “cloud” sec­tion (going beyond of what we have in terms of vir­tu­al­iza­tion already) in out hand­book. We can put there items like the exist­ing descrip­tion of vir­tu­al­i­sa­tion items, but also should put there new items like glus­terfs or object stor­age or the hope­ful­ly upcom­ing pos­si­bil­i­ty of how to set­up OpenStack/kubernetes/… on FreeB­SD. This goes into the same direc­tion as the first docs-item in terms of pro­vide more doc­u­men­ta­tion how to achieve goals of our users. 

In my opin­ion we are also lack­ing on the developer-documentation side. Yes, we have man pages which describe the offi­cial API (in most cas­es). Where I see room for improve­ment is the source code doc­u­men­ta­tion. Some­thing like doxy­gen (or what­ev­er the tool of the day is – which one does not real­ly mat­ter, any kind of extractable-from-source doc­u­men­ta­tion is bet­ter than no extractable doc­u­men­ta­tion) is already used in sev­er­al places in our source (search for it via: egrep ‑R ‘\\(brief|file)’ /usr/src/) and we have already some infra­struc­ture to extract and ren­der (HTML / PDF) them. The more acces­si­ble / easy it is to start devel­op­ment in FreeB­SD, the more attrac­tive it will be (addi­tion­al to the exist­ing ben­e­fits) to peo­ple / com­pa­nies to dive in. The best exam­ples about doc­u­ment­ing source code in our code I have found so far is the isci and ocs_fc device code. 

Polishing

Pol­ish­ing some­thing in the top­ic of stay­ing rel­e­vant? Yes! It is the details which mat­ter. If peo­ple have 2 options with rough­ly the same fea­tures (noth­ing miss­ing what you need, same price), which one do they take, the one which has every­thing con­sis­tent and well inte­grat­ed, or the one with some quirks you can cir­cum­vent with a lit­tle bit of work on their side?

We have some nice fea­tures, but we are not using it to the extend pos­si­ble. One of the items which come to my mind is DTrace. The area which I think needs pol­ish­ing is to add more probes, and to have some kind of probe-convention about com­mon top­ics. For exam­ple I/O relat­ed nam­ing con­ven­tion (maybe area spe­cif­ic, like stor­age I/O and net­work I/O) and cov­er­ing all dri­vers to com­ply. We should also look into mak­ing it more acces­si­ble by pro­vid­ing more easy inter­faces (no mat­ter if text based (thanks to Devin Teske for dwatch, more of this mag­ic please…), web based, or what­ev­er) to make it real­ly easy (= start a com­mand or click around and you get the result for a spe­cif­ic set of probes/conditions/…). Some exam­ples are statemaps, flamegraphs and most promi­nent­ly the Oracle/Sun ZFS Stor­age Ana­lyt­ics to give you an idea what is pos­si­ble with DTrace and and how to make it acces­si­ble to peo­ple with­out knowl­edge about the ker­nel inter­nals and programming.

Some pol­ish­ing in the ports col­lec­tion would be to revis­it the defaults options for ports with options. The tar­get here should be to have con­sis­tent default set­tings (e.g. serv­er soft­ware should not depend upon X11 by default (direct­ly or indi­rect­ly), most peo­ple should not need to build the port with non-default options). One could argue that it is the respon­s­abil­i­ty of the main­tain­er of the port, and to some extend it is, but we do not have guide­lines which help here. So a lit­tle team of peo­ple to review all ports (and mod­i­fy them if nec­es­sary) and come up with guide­lines and exam­ples would be great.

Addi­tion­al­ly we should come up with meta-ports for spe­cif­ic use case, e.g. web­serv­er (dif­fer­ent flavours… apache/nginx/…), data­base (dif­fer­ent flavours, with some use­ful tools like mytop or mysql­tuner or sim­i­lar) and then even ref­er­ence them in the hand­book (this goes along with my sug­ges­tion above to doc­u­ment real-world use cas­es instead of “only” the OS itself).

Recent­ly ‑cur­rent has seen some low lev­el per­for­mance improve­ments (via ifuncs, …). We should con­tin­ue this and even extend it to revise default set­tings / val­ues (non auto-tuned and even auto-tuned ones). I think it would be ben­e­fi­cial in the long run if we tar­get more cur­rent hard­ware (with­out los­ing the abil­i­ty to run on old­er hard­ware) and for those val­ues which can not be auto-tuned pro­vide some way of down-tuning (e.g. in a failsafe-boot set­ting in the loader or in doc­u­ment­ed set­tings for rc.conf or wher­ev­er those defaults can be changed). 

Project infrastructure

We have a CI (con­tin­u­ous inte­gra­tion) sys­tem, but it is not very promi­nent­ly placed. Just recent­ly it gained some more atten­tion from the devel­op­er side and we even got the first sta­tus report about it (nice! vis­i­bil­i­ty helps mak­ing it a part of the com­mu­ni­ty effort). There is a FreeBSD-wiki page about the sta­tus and the future ideas, but it was not updat­ed since sev­er­al months. There is also a page which talks in more details about using it for per­for­mance test­ing, which is some­thing peo­ple have talked about since years but nev­er became avail­able (and is not today).

I think we need to improve here. The goals I think which are impor­tant are to get var­i­ous test­ing, san­i­tiz­ing and fuzzing tech­nolo­gies inte­grat­ed into our CI. On the con­fig repos­i­to­ry I have not found any inte­gra­tion of e.g. the cor­re­spond­ing clang tech­nolo­gies (fuzzing, ASAN, UBSAN, MSAN (still exper­i­men­tal, so maybe not to tar­get before oth­er mature tech­nolo­gies)) or any oth­er such technology.

We should also make our CI more public/visible (build sta­tus linked some­where on www.freebsd.org, nag peo­ple more about issues found by it, have some docs how to add new tests (maybe from ports), so that more peo­ple can help in extend what we auto­mat­i­cal­ly test (e.g. how could I inte­grate the LTP (Lin­ux Test Project) tests to test our lin­ux­u­la­tor? This requires the down­load of a lin­ux dist port, the LTP itself, and then to run the tests). There are a lot of nice ideas float­ing around, but I have the impres­sion we are lack­ing some help­ing hands to get var­i­ous items integrated. 

Wrap-Up 

Var­i­ous items I talked above are not sexy. Those are typ­i­cal­ly not the things peo­ple do just for fun. Those are typ­i­cal­ly items peo­ple get paid for. It would be nice if some of the com­pa­nies which ben­e­fit from FreeB­SD would be so nice to lend a help­ing hand for one or anoth­er item. Maybe the FreeB­SD Foun­da­tion has some con­tacts they could ask about this? 

It could also be that for some of the items I men­tioned here there is more ongo­ing that I know of. This means then that the cor­re­spond­ing work could be made more known on the mail­inglists. When it is more known, maybe some­one wants to pro­vide a help­ing hand.