Tun­ing guide in the wiki

In the light of the re­cent bench­mark dis­cus­sion, a vo­lun­teer im­por­ted the tun­ing man-​page in­to the wiki. Some com­ments at some places for pos­sible im­prove­ments are already made. Please go over there, have a look, and par­ti­cip­ate please (testing/​verification/​discussion/​improvements/​…).

As al­ways, feel free to re­gister with First­nameLast­name and tell a FreeBSD com­mit­ter to add you to the con­trib­ut­ors group for write ac­cess (you also get the be­ne­fit to be able to re­gister for an email no­ti­fic­a­tion for spe­cif­ic pages).

How big are the buf­fers in FreeBSD drivers?

Today I have read an in­ter­est­ing in­vest­ig­a­tion and prob­lem ana­lys­is from Jim Gettys.

It is a set of art­icles he wrote over sev­er­al months and is not fin­ished writ­ing as of this writ­ing (if you are deeply in­ter­ested in it go and read them, the most in­ter­est­ing ones are from Decem­ber and Janu­ary and the com­ments to the art­icles are also con­trib­ut­ing to the big pic­ture). Ba­sic­ally he is telling that a lot of net­work prob­lems users at home (with ADSL/​cable or WLAN) ex­per­i­ence  are be­cause buf­fers in the net­work hard­ware or in op­er­at­ing sys­tems are too big. He also pro­poses work­arounds un­til this prob­lem is at­tacked by OS vendors and equip­ment man­u­fac­tur­ers.

Ba­sic­ally he is telling the net­work con­ges­tion al­gorithms can not do their work good, be­cause the net­work buf­fers which are too big come in­to the way of their work (not re­port­ing pack­et loss timely enough re­spect­ively try to not lose pack­ets in situ­ations where pack­et loss would be bet­ter be­cause it would trig­ger ac­tion in the con­ges­tion al­gorithms).

He in­vest­ig­ated the be­ha­vi­or of Linux, OS X and Win­dows (the sys­tem he had avail­able). I wanted to have a quick look at the situ­ation in FreeBSD re­gard­ing this, but it seems at least with my net­work card I am not able to see/​find the cor­res­pond­ing size of the buf­fers in drivers in 30 seconds.

I think it would be very good if this is­sue is in­vest­ig­ated in FreeBSD, and apart from maybe tak­ing some ac­tion in the source also write some sec­tion for the hand­book which ex­plains the is­sue (one prob­lem here is, that there are situ­ations where you want/​need to have such big buf­fers and as such we can not just downs­ize them) and how to bench­mark and tune this.

Un­for­tu­nately I even have too much on my plate to even fur­ther look in­to this. 🙁 I hope one of the net­work people in FreeBSD is pick­ing up the ball and starts play­ing.

Are USB memory sticks really that bad?

Last week my ZFS cache device – an USB memory stick – showed xxxM write er­rors. I got this stick for free as a promo, so I do not ex­pect it to be of high qual­ity (or wear-​leveling or sim­il­ar life-​saving things). The stick sur­vived about 9 months, dur­ing which it provided a nice speed-​up for the ac­cess to the cor­res­pond­ing ZFS stor­age pool. I re­placed it by an­oth­er stick which I got for free as a promo. This new stick sur­vived… one long week­end. It has now 8xxM write er­rors and the USB sub­sys­tem is not able to speak to it any­more. 30 minutes ago I is­sued an “us­b­con­fig re­set” to this device, which is still not fin­ished. This leads me to the ques­tion if such sticks are really that bad, or if some prob­lem crept in­to the USB sub­sys­tem?

If this is a prob­lem with the memory stick it­self, I should be able to re­pro­duce such a prob­lem on a dif­fer­ent ma­chine with a dif­fer­ent OS. I could test this with FreeBSD 8.1, Sol­ar­is 10u9, or Win­dows XP. What I need is an auto­mated test. This rules out the Win­dows XP ma­chine for me, I do not want to spend time to search a suit­able test which is avail­able for free and al­lows to be run in an auto­mated way. For FreeBSD and Sol­ar­is it prob­ably comes down to use some disk-​I/​O bench­mark (I think there are enough to chose from in the FreeBSD Ports Col­lec­tion) and run it in a shell-loop.