I noticed that we do not have some auto­matic way of scrub­bing a ZFS pool peri­od­i­cally. A quick poll on fs@ revealed, that there is inter­est in some­thing like this. So I took a lit­tle bit of time to write a peri­odic daily script which checks if the last scrub is X days ago and scrubs a pool accord­ingly. The script has options to scrub all pools, or just a spe­cific sub­set. It also allows to spec­ify a time-interval between scrubs for each pool with dif­fer­ent lev­els of fall-back (if no pool-specific inter­val is set, the default inter­val is used, which is set to 30 days if no other default inter­val is specified).

The dis­cus­sion about this is hap­pen­ing over at fs@, so go there and have a look for the CFT (with a link to the WIP of the script) and the dis­cus­sion if you are interested.

So far there are some minor details to sort out (and a lit­tle bit of doc­u­men­ta­tion to write) before I can com­mit it… prob­a­bly next week.

Bren­dan Gregg of Sun Ora­cle fame made a good expla­na­tion how to visu­al­ize latency to get a bet­ter under­stand­ing of what is going on (and as such about how to solve bot­tle­necks). I have seen all this already in var­i­ous posts in his blog and in the Ana­lyt­ics pack­age in an Open­Stor­age pre­sen­ta­tion, but the ACM arti­cle sum­ma­rizes it very good.

Unfor­tu­nately Ana­lyt­ics is AFAIK not avail­able in Open­So­laris, so we can not go out and adapt it for FreeBSD (which would prob­a­bly require to port/implement some addi­tional dtrace stuff/probes). I am sure some­thing like this would be very inter­est­ing to all those com­pa­nies which use FreeBSD in an appli­ance (regard­less if it is a stor­age appli­ance like NetApp, or a net­work appli­ance like a Cisco/Juniper router, or any­thing else which has to per­form good).

The dis­cus­sion about direct and indi­rect depen­den­cies is com­ing up again on the FreeBSD mail­inglists. Seems I should make some blog post about it, maybe it makes this topic more find­able than my post­ings in the mailinglists.

Some def­i­n­i­tions:

• A direct depen­dency from A to B is when program/port A uses sym­bols from library/port B.
• An indi­rect depen­dency from A to C is when program/port A uses sym­bols from library/port B but no sym­bols from library/port C, and library/port B uses sym­bols from library/port C.
• An explicit depen­dency from A to C is when it is a direct or indi­rect depen­dency A to C, and when the compiler-time-linker added an explicit ref­er­ence to C to the program/lib of A.

Ide­ally we have no indi­rect depen­den­cies in the explicit depen­den­cies, only direct depen­den­cies. Unfor­tu­nately in real­ity we also have indi­rect depen­den­cies there. This has at least two causes:

1. libtool (at least 1.x) does not (or was not) come with a hint on FreeBSD, which tells that the run-time-linker is recur­sively resolv­ing dependencies.
2. Some pkg-config setups list indi­rect depen­den­cies as explicit depen­den­cies (IIRC it depends if Requires.private and/or Libs.private is used in the .pc file or not; if it is used, there should be no indi­rect depen­dency appear from this soft­ware, but I am not 100% sure about this).

Three years ago I wrote /usr/ports/Tools/scripts/explicit_lib_depends.sh, it looks at the files of a given port (it needs to be installed), and prints out explicit depen­den­cies. Because of the indi­rect depen­den­cies which could be listed there, this list is not a list of ports which are real depen­den­cies from a source code point of view, but it reflects the link-time real­ity. If a port C shows up there, the port which is checked needs to be rebuild in case the ABI of library/port C changes.

After all the big-impact com­mits (Gnome/gettext/KDE/X11/…) have set­tled now, I took the time to update audio/lame (I iden­ti­fied more than 100 ports with an (implicit) depen­dency on lame, 45 of them needed a portre­vi­sion bump; if I forgot/overlooked some, bump the revi­sion your­self or notify me please). That is the first update of my ports where miwi@ did not beat me in com­mit­ting an update since a year (he has implicit approval to do any­thing he wants with my ports).

I can be happy that he is/was this fast (and that we have such a pro­duc­tive and effi­cient com­mit­ter), or I can be sad that I do not have the time any­more to be faster than I am with such things… or both. Hmmm… I think I will go the happy way.

I have an old sys­tem (only the hard­ware, it runs –cur­rent) which reboots itself from time to time (mostly dur­ing the daily periodic(8) run, but also dur­ing a lot of com­pil­ing (por­tup­grade)). There is no obvi­ous rea­son (no panic) why it is doing this. It could be that there is some hard­ware defect, or some­thing else. It is not impor­tant enough to get a high enough pri­or­ity that I try hard to ana­lyze the prob­lem with this machine. The annoy­ing part is, that some­times after a restart apache does not start. So if this hap­pens, the solu­tion is to login and start the web­server. If the web­server would start each time, nearly nobody would detect the reboot (root gets an EMail on each reboot via an @reboot crontab entry).

My prag­matic solu­tion (for ser­vices started via a good rc.d script which has a work­ing sta­tus com­mand) is a crontab entry which checks peri­od­i­cally if it is run­ning and which restarts the ser­vice if not. As an exam­ple for apache and an inter­val of 10 minutes:

*/10 * * * *    /usr/local/etc/rc.d/apache22 status >/dev/null 2>&1 || /usr/local/etc/rc.d/apache22 restart

For the use case of this service/machine, this is enough. In case of a prob­lem with the ser­vice, a mail with the restart out­put would arrive each time it runs, else only after a reboot for which the ser­vice did not restart.