I have an old sys­tem (only the hard­ware, it runs –cur­rent) which reboots itself from time to time (mostly dur­ing the daily periodic(8) run, but also dur­ing a lot of com­pil­ing (por­tup­grade)). There is no obvi­ous rea­son (no panic) why it is doing this. It could be that there is some hard­ware defect, or some­thing else. It is not impor­tant enough to get a high enough pri­or­ity that I try hard to ana­lyze the prob­lem with this machine. The annoy­ing part is, that some­times after a restart apache does not start. So if this hap­pens, the solu­tion is to login and start the web­server. If the web­server would start each time, nearly nobody would detect the reboot (root gets an EMail on each reboot via an @reboot crontab entry).

My prag­matic solu­tion (for ser­vices started via a good rc.d script which has a work­ing sta­tus com­mand) is a crontab entry which checks peri­od­i­cally if it is run­ning and which restarts the ser­vice if not. As an exam­ple for apache and an inter­val of 10 minutes:

*/10 * * * *    /usr/local/etc/rc.d/apache22 status >/dev/null 2>&1 || /usr/local/etc/rc.d/apache22 restart

For the use case of this service/machine, this is enough. In case of a prob­lem with the ser­vice, a mail with the restart out­put would arrive each time it runs, else only after a reboot for which the ser­vice did not restart.

I am tired of bad net­work cards on main­boards. Can someome who knows a mar­ket­ing guy of a main­board man­u­fac­turer please please tell them to inte­grate a good net­work card (Intel ones comes to my mind, but other non-RealTec ones are not bad either… if an Intel one is not pos­si­ble at all)? They just need to sell those main­boards as gam­ing and/or mul­ti­me­dia optimized:

• a good net­work card trans­fers more data per sec­ond (even while the CPU is used much)
• a good net­work card does not take much CPU (good DMA engine and some IP-stuff-in-hardware features)

Sim­i­lar things can be told about good/bad SATA con­trollers and USB con­trollers. If done right, they affect the sys­tem less and/or per­form bet­ter than not so good parts.

Yes, most of this does not affect gam­ing in a sig­nif­i­cant way, but a lot of other hard­ware gamers buy does not affect the gam­ing significantly.

The point is, that they are will­ing to give more money to get some­thing bet­ter. So please, give them the pos­si­bil­ity to pay a lit­tle bit more for good qual­ity so that good stuff will go mainstream.

And do not under­es­ti­mate the power of mouth-to-mouth mar­ket­ing. If you pro­duce some­thing bet­ter than other com­pa­nies for a fair price (no need to play the we-are-the-cheapest game, just take care to not be the most expen­sive one), it will be bought.

BTW: This is sim­i­lar to the mar­ket­ing with good capac­i­tors, just on an IT instead of an elec­tri­cal level.

Cur­rently I play a lit­tle bit around with my ZFS setup. I want to make it faster, but I do not want to spend a lot of money.

The disks are con­nected to an ICH 5 con­troller, so an obvi­ous improve­ment would be to either buy a con­troller for the PCI slot which is able to do NCQ with the SATA disks (a siis(4) based one is not cheap), or to buy a new sys­tem which comes with a chipset which knows how to do NCQ (this would mean new RAM, new CPU, new MB and maybe even a new PSU). A new con­troller is a lit­tle bit expen­sive for the old sys­tem which I want to tune. A new sys­tem would be nice, and read­ing about the specs of new sys­tems lets me want to get a Core i5 sys­tem. The prob­lem is that I think the cur­rent offers of main­boards for this are far from good. The sys­tem should be a lit­tle bit future proof, as I would like to use it for about 5 years or more (the cur­rent sys­tem is some­where between 5–6 years old). This means it should have SATA-3 and USB 3, but when I look at what is offered cur­rently it looks like there are only beta-versions of hard­ware with SATA-3 and USB 3 sup­port avail­able on the marked (accord­ing to tests there is a lot of vari­ance of the max speed the con­trollers are able to achieve, bugs in the BIOS, or the  con­trollers are attached to a slow bus which pre­vents to use the full band­width). So it will not be a new sys­tem soon.

As I had a 1GB USB-stick around, I decided to attach it to the one of the EHCI USB ports and use it as a cache device for ZFS. If some­one wants to try this too, be care­ful with the USB ports. My main­board has only 2 USB ports con­nected to an EHCI, the rest are UHCI ones. This means that only 2 USB ports are fast (sort of… 40 MBit/s), the rest is only usable for slow things like a mouse, key­board or a ser­ial line.

Be warned, this will not give you a lot of band­width (if you have a fast USB stick, the 40MBit/s of the EHCI are the limit which pre­vent a big stream­ing band­width), but the latency of the cache device is great when doing small ran­dom IO. When I do a gstat and have a look how long a read oper­a­tion takes for each involved device, I see some­thing between 3 msec and 20 msec for the hard­disks (depend­ing if they are read­ing some­thing at the cur­rent head posi­tion, or if the hard­disk needs to seek around a lot). For the cache device (the USB stick) I see some­thing between around 1 mssec and 5 msec. That is 1/3^th to 1/4^th of the latency of the harddisks.

With a “zfs send” I see about 300 IOops per hard­disk (3 disks in a RAIDZ). Obvi­ously this is an opti­mum stream­ing case where the disks do not need to seek around a lot. You see this in the low latency, it is about 2 msec in this case. In the random-read case, like for exam­ple when you run a find, the disks can not keep this amount of IOops, as they need to seek around. And here the USB-stick shines. I’ve seen upto 1600 IOops on it dur­ing run­ning a find (if the cor­re­spond­ing data is in the cache, off course). This was with some­thing between 0.5 and 0.8 msec of latency.

This is the machine at home which is tak­ing care about my mails (incom­ing and out­go­ing SMTP, IMAP and Web­mail), has a squid proxy and acts as a file server. There are not many users (just me and my wife) and there is no reg­u­lar usage pat­tern for all those ser­vices. Because of this I did not do any bench­mark to see how much time I can gain with var­i­ous work­loads (and I am not inter­ested in some arti­fi­cial per­for­mance num­bers of my web­mail ses­sion, as the brows­ing expe­ri­ence is highly sub­jec­tive in this case). For this sys­tem a 1 GB USB stick (which was just col­lect­ing dust before) seems to be a cheap way to improve the response time for often used small data. When I use the web­mail inter­face now, my sub­jec­tive impres­sion is, that it is faster. I am talk­ing about list­ing emails (sub­ject, date, sender, size) and dis­play­ing the con­tent of some emails. FYI, my maildir stor­age has 849 MB with 35000 files in 91 folders.

Bot­tom line is: do not expect a lot of band­width increase with this, but if you have a work­load which gen­er­ates ran­dom read requests and you want to decrease the read latency, it could be a cheap solu­tion to add a (big) USB stick as a cache device.

I exper­i­mented a lit­tle bit with the order of direc­to­ries to backup in tarsnap.

Cur­rently I use the fol­low­ing sort­ing algorithm:

1. least fre­quently changed directory-trees first
   Every change — even in meta-data — will affect the fol­low­ing data, as tarsnap is doing the de-duplication in fixed-width blocks (AFAIR 64k).
2. for those directory-trees which change with the same fre­quency: list the big­ger ones first
   Implic­itly I assume that the smaller ones are much smaller than the big­ger ones so that the smaller part which will be backed up will not be noticed because of the big­ger change. For my use cases of tarsnap this is true.
3. if changes in a directory-tree are much much big­ger than any­thing else, but the directory-tree has a medium change-frequency, put it even before less-frequently chang­ing stuff
   I do not want that a small change trig­gers a big backup, but a big backup can con­tain the remain­ing small part.
4. if you backup home direc­to­ries (even root’s one) and they do not con­tain much data, put them before directory-trees which change a lot daily
   I do not want that a login trig­gers the trans­fer of data in other directory-trees which have not changed.

I had a look at all my posts and deter­mined which ones pro­vide a mid-term / long-term ben­e­fit to read­ers. Those I clas­si­fied for myself as Very Impor­tant Posts (for an appro­pri­ate def­i­n­i­tion of very and impor­tant…) and they are listed on the VIP page now.