I noticed that we do not have some automatic way of scrubbing a ZFS pool periodically. A quick poll on fs@ revealed, that there is interest in something like this. So I took a little bit of time to write a periodic daily script which checks if the last scrub is X days ago and scrubs a pool accordingly. The script has options to scrub all pools, or just a specific subset. It also allows to specify a time-interval between scrubs for each pool with different levels of fall-back (if no pool-specific interval is set, the default interval is used, which is set to 30 days if no other default interval is specified).
The discussion about this is happening over at fs@, so go there and have a look for the CFT (with a link to the WIP of the script) and the discussion if you are interested.
So far there are some minor details to sort out (and a little bit of documentation to write) before I can commit it… probably next week.
Brendan Gregg of Sun Oracle fame made a good explanation how to visualize latency to get a better understanding of what is going on (and as such about how to solve bottlenecks). I have seen all this already in various posts in his blog and in the Analytics package in an OpenStorage presentation, but the ACM article summarizes it very good.
Unfortunately Analytics is AFAIK not available in OpenSolaris, so we can not go out and adapt it for FreeBSD (which would probably require to port/implement some additional dtrace stuff/probes). I am sure something like this would be very interesting to all those companies which use FreeBSD in an appliance (regardless if it is a storage appliance like NetApp, or a network appliance like a Cisco/Juniper router, or anything else which has to perform good).
The discussion about direct and indirect dependencies is coming up again on the FreeBSD mailinglists. Seems I should make some blog post about it, maybe it makes this topic more findable than my postings in the mailinglists.
- A direct dependency from A to B is when program/port A uses symbols from library/port B.
- An indirect dependency from A to C is when program/port A uses symbols from library/port B but no symbols from library/port C, and library/port B uses symbols from library/port C.
- An explicit dependency from A to C is when it is a direct or indirect dependency A to C, and when the compiler-time-linker added an explicit reference to C to the program/lib of A.
Ideally we have no indirect dependencies in the explicit dependencies, only direct dependencies. Unfortunately in reality we also have indirect dependencies there. This has at least two causes:
- libtool (at least 1.x) does not (or was not) come with a hint on FreeBSD, which tells that the run-time-linker is recursively resolving dependencies.
- Some pkg-config setups list indirect dependencies as explicit dependencies (IIRC it depends if Requires.private and/or Libs.private is used in the .pc file or not; if it is used, there should be no indirect dependency appear from this software, but I am not 100% sure about this).
Three years ago I wrote /usr/ports/Tools/scripts/explicit_lib_depends.sh, it looks at the files of a given port (it needs to be installed), and prints out explicit dependencies. Because of the indirect dependencies which could be listed there, this list is not a list of ports which are real dependencies from a source code point of view, but it reflects the link-time reality. If a port C shows up there, the port which is checked needs to be rebuild in case the ABI of library/port C changes.
After all the big-impact commits (Gnome/gettext/KDE/X11/…) have settled now, I took the time to update audio/lame (I identified more than 100 ports with an (implicit) dependency on lame, 45 of them needed a portrevision bump; if I forgot/overlooked some, bump the revision yourself or notify me please). That is the first update of my ports where miwi@ did not beat me in committing an update since a year (he has implicit approval to do anything he wants with my ports).
I can be happy that he is/was this fast (and that we have such a productive and efficient committer), or I can be sad that I do not have the time anymore to be faster than I am with such things… or both. Hmmm… I think I will go the happy way. 😉