The dis­cus­sion about direct and indi­rect depen­den­cies is com­ing up again on the FreeBSD mail­inglists. Seems I should make some blog post about it, maybe it makes this topic more find­able than my post­ings in the mailinglists.

Some def­i­n­i­tions:

• A direct depen­dency from A to B is when program/port A uses sym­bols from library/port B.
• An indi­rect depen­dency from A to C is when program/port A uses sym­bols from library/port B but no sym­bols from library/port C, and library/port B uses sym­bols from library/port C.
• An explicit depen­dency from A to C is when it is a direct or indi­rect depen­dency A to C, and when the compiler-time-linker added an explicit ref­er­ence to C to the program/lib of A.

Ide­ally we have no indi­rect depen­den­cies in the explicit depen­den­cies, only direct depen­den­cies. Unfor­tu­nately in real­ity we also have indi­rect depen­den­cies there. This has at least two causes:

1. libtool (at least 1.x) does not (or was not) come with a hint on FreeBSD, which tells that the run-time-linker is recur­sively resolv­ing dependencies.
2. Some pkg-config setups list indi­rect depen­den­cies as explicit depen­den­cies (IIRC it depends if Requires.private and/or Libs.private is used in the .pc file or not; if it is used, there should be no indi­rect depen­dency appear from this soft­ware, but I am not 100% sure about this).

Three years ago I wrote /usr/ports/Tools/scripts/explicit_lib_depends.sh, it looks at the files of a given port (it needs to be installed), and prints out explicit depen­den­cies. Because of the indi­rect depen­den­cies which could be listed there, this list is not a list of ports which are real depen­den­cies from a source code point of view, but it reflects the link–time real­ity. If a port C shows up there, the port which is checked needs to be rebuild in case the ABI of library/port C changes.

Seems that I will actively men­tor again in this Google Sum­mer of Code (as opposed to just review the sub­mis­sions from stu­dents and/or act­ing as a fall-back mentor).

The project I will men­tor is the “Make optional ker­nel sub­sys­tems reg­is­ter them­selves via sysctl”-one from the FreeBSD ideas page.

The stu­dent already got into con­tact with me and it looks like he is moti­vated (he is already sub­scribed to sev­eral FreeBSD mail­inglists, which is not a require­ment we have in our GSoC docs).

Due to the prob­lems with a 7–sta­ble machine, I had a look at some unmerged fixes for ZFS (58 changes not merged).

I back­ported some of those changes from 8-stable to 7-stable, I have this run­ning on one 7-stable machine. I would like to get some more feed­back for it (even an “it works for me” would be great). The main part of this change is that the FreeBSD taskqueue is used now instead of the open­so­laris one (and some other changes which may improve the ZFS experience).

It would also be nice if some­one could have a look at the FIRST_THREAD_IN_PROC part. Can there be more than one thread at this place (I do not think so) and I should use FOREACH_THREAD_IN_PROC_instead?

How to apply:

• cd /usr/src/
• fetch http://www.Leidinger.net/FreeBSD/test/releng7_zfs_merge3.diff
• fetch http://www.Leidinger.net/FreeBSD/test/opensolaris_taskq.c
• fetch http://www.Leidinger.net/FreeBSD/test/taskq.h
• mv taskq.h sys/cddl/contrib/opensolaris/uts/common/sys/taskq.h
• mv opensolaris_taskq.c sys/cddl/compat/opensolaris/kern/opensolaris_taskq.c
• patch –p 0 –quiet <releng7_zfs_merge3.diff
• ignore the 2 .rej files
• rm –f sys/cddl/compat/opensolaris/sys/taskq_impl.h*
• rm –f sys/cddl/compat/opensolaris/sys/taskq.h*
• rm –f sys/cddl/contrib/opensolaris/uts/common/os/taskq.c*
• rebuild ker­nel

I do not list all of those 16 of 58 out­stand­ing patches which are cov­ered here, a detailed list can be found on the sta­ble and fs mail­inglists.

On the machine where I host this blog, I have/had some sta­bil­ity prob­lems.

Last week I updated the machine from FreeBSD 7.1-pX to 7.2-p5 (GENERIC ker­nel in both cases). 5 – 10 Min­utes after the reboot into the new ver­sion the machine had a dead­lock. After some road­blocks (order­ing a KVM-switch from the hoster, the KVM-switch not work­ing with a proxy (dur­ing lunchtime at work), a bro­ken video-capture of the KVM-switch and a replace­ment on Mon­day morn­ing to not pay the WE-fees), I spend a big part of the night to get it sta­ble. I tried dis­abling SMP, enabling INVARIANTS and WITNESS, chang­ing the sched­uler, cut­ting the soft­ware mir­ror (to rule out a mis­match between the con­tent of the disks after all the hard reboots) and updat­ing to 7-stable.

Unfor­tu­nately noth­ing helped.

Googling a lit­tle bit around (it is a AMD Dual–Core sys­tem with NVidia MCP61 chipset) was lead­ing me to a post on the mail­inglists from 2008 which talks about an issue with the buffer cache. I do not know if this is still an issue (I have send a email to kib@ to ask about it), and my sce­nario is not the same as the one which is described in the mail, but because of this I decided to switch one of the two UFS mir­rors to ZFS.

The first boot into the ZFS caused again a reboot after some min­utes (I do not know if it was because of a mem­ory exhausted panic, or because of a dead­lock), but as I did not tune the ker­nel for ZFS I am tempted to believe that I should not count that. Now, after tun­ing the ker­nel (increas­ing the kmem_size to 700M, no prefetch­ing, lim­it­ing the ARC to 40M) it is up since nearly 2h (as of this writ­ting… cross­ing fin­gers). Before it was not able to sur­vive more than some min­utes with just the jail for the mails up. Now I not only have the mail-jail up, but also the jail for the blog (one jail still dis­abled, but I will take care about that after this post).

I do not know if only increas­ing the kmem_size would have helped with the prob­lem, but as I was test­ing a GENERIC ker­nel + gmir­ror mod­ule in the begin­ning, I expected that the auto-tuning of this value should have been enough for such a sim­ple setup (2GB RAM, 2 disks with 3 par­ti­tions each, one par­ti­tion pair for root, one for swap, one for the jails).

I hope that I sta­bi­lized the sys­tem now. It may be the case that I will test some patches in case some­one comes up with some­thing, so do not be sur­prised if the blog and email to me is a lit­tle bit flaky.

After two weeks of hol­i­day I have about 5.400 emails (about 51 MB, down­loaded and fil­tered in 3h). Split up this is:

• 2763 from FreeBSD mail­inglists
• 320 SPAM (101 false pos­i­tives, mostly FreeBSD related… I should have a look at this and take appro­pri­ate actions)
• 115 in the nor­mal inbox = per­sonal + not fil­tered emails (18 false neg­a­tive SPAM)
• 16 post­mas­ter related, e.g. bounces from forged SPAM, 4 of them are not SPAM bounces but valid mails…
• 114 secu­rity related (Bugtraq, …)
• 220 sta­tus mes­sages (e.g. FreeBSD peri­odic mails from var­i­ous systems)
• the rest is var­i­ous other (fil­tered away) mails

It will take a while until I had a look at them…