ZFS and NFS / on-disk-cache

In the FreeB­SD mail­inglists I stum­bled over  a post which refers to a blog-post which describes why ZFS seems to be slow (on Solaris).

In short: ZFS guar­an­tees that the NFS client does not expe­ri­ence silent cor­rup­tion of data (NFS serv­er crash and loss of data which is sup­posed to be already on disk for the client). A rec­om­men­da­tion is to enable the disk-cache for disks which are com­plete­ly used by ZFS, as ZFS (unlike UFS) is aware of disk-caches. This increas­es the per­for­mance to what UFS is deliv­er­ing in the NFS case.

There is no in-deep descrip­tion of what it means that ZFS is aware of disk-caches, but I think this is a ref­er­ence to the fact that ZFS is send­ing a flush com­mand to the disk at the right moments. Let­ting aside the fact that there are disks out there which lie to you about this (they tell the flush com­mand fin­ished when it is not), this would mean that this is sup­port­ed in FreeB­SD too.

So every­one who is cur­rent­ly dis­abling the ZIL to get bet­ter NFS per­for­mance (and accept silent data cor­rup­tion on the client side): move your zpool to ded­i­cat­ed (no oth­er real FS than ZFS, swap and dump devices are OK) disks (hon­est ones) and enable the disk-caches instead of dis­abling the ZIL.

I also rec­om­mend that peo­ple which have ZFS already on ded­i­cat­ed (and hon­est) disks have a look if the disk-caches are enabled.

IPv6 in my LAN

After enabling IPv6 in my WLAN router, I also enabled IPv6 in my FreeB­SD sys­tems. I have to tell that the IPv6 chap­ter in the FreeB­SD hand­book does not con­tain as much infor­ma­tion as I would like to have about this.

Con­fig­ur­ing the inter­faces of my two 9‑current sys­tems to also car­ry a spe­cif­ic IPv6 address (an easy one from the ULA I use) was easy after read­ing the man-page for rc.conf. After a lit­tle bit of exper­i­ment­ing it came down to:

ifconfig_rl0_ipv6=“inet6 ::2:1 pre­fixlen 64 accept_rtadv”
ipv6_defaultrouter=”<router address>”

Apart from this address (I chose it because the IPv4 address ends in “.2”, this way I can add some easy to remem­ber address­es for this machine if need­ed), I also have two auto­mat­i­cal­ly con­fig­ured address­es. One is with the same ULA and some not so easy to remem­ber end (con­struct­ed from the MAC address), and one is from the offi­cial pre­fix the router con­struct­ed out of the offi­cial IPv4 address from the ISP (+ the same end than the oth­er end).

Addi­tion­al­ly I also have all my jails on this machine with an IPv6 address now (yes, they are like “…:2:100” with the :100 because the IPv4 address ends in “.100”). Still TODO is the con­ver­sion of all the ser­vices in the jails to also lis­ten on the IPv6 address.

I already changed the con­fig of my inter­nal DNS to have the IPv6 address­es for all sys­tems, lis­ten on the IPv6 address (when I add an IPv6 net­work to allow-query/allow-query-cache/allow-recursion bind does not want to start). And as I was there, I also enabled the DNSSEC ver­i­fi­ca­tion (but I get a lot of error mes­sages in the logs: “unable to con­vert errno to isc_result: 42: Pro­to­col not avail­able”, one search result which talks exact­ly about this error tells it is a “cos­met­ic error”…).

I noticed that an IPv6 ping between two phys­i­cal machines takes a lit­tle bit more time than an IPv4 ping (no IPsec enabled). It sur­prised me that this is such a notice­able dif­fer­ence (not with­in the std-dev at all):

— m87.Leidinger.net ping statistics —
10 pack­ets trans­mit­ted, 10 pack­ets received, 0.0% pack­et loss
round-trip min/avg/max/stddev = 0.168÷0.193÷0.220÷0.017 ms

— m87.Leidinger.net ping6 statistics —
10 pack­ets trans­mit­ted, 10 pack­ets received, 0.0% pack­et loss
round-trip min/avg/max/std-dev = 0.207÷0.325÷0.370÷0.047 ms

The infor­ma­tion I miss in the FreeB­SD hand­book in the IPv6 chap­ter is what those oth­er IPv6 relat­ed ser­vices are and when/how to con­fig­ure them. I have an idea now what this rad­vd is, but I am not sure what the inter­ac­tion is with the accept_rtadv set­ting for ifcon­fig (and I do not think I need it, as my WLAN router seems to do it already). I know that I get the IPv6-friendly net­work neigh­bor­hood dis­played with ndp(8). I did not have a look at enabling IPv6 mul­ti­cast sup­port in FreeB­SD, and I do not know what those oth­er IPv6 options for rc.conf do.

What to do with the weath­er data?

As can be read in the pre­vi­ous blog post­ing, I have now some weath­er data around. Here the head­er from the CSV file I gen­er­ate out of the XML file of the software:

Date;Humidity (room);Temp (room);Humidity (out);Temp (out);Pressure (abs);Wind (ave);Wind (gust);Wind (dir);Rain

Cur­rent­ly I pro­grammed the weath­er sta­tion to save the data every 5 min­utes. The long-term goal is to decide if a giv­en wind tur­bine deliv­ers a sane amount of ener­gy (dur­ing a com­plete year) at a giv­en place. As I do not want to wait that long to get some infor­ma­tion out of this, the ques­tion aris­es, what I can do with this weath­er data?

Here an exam­ple of the out­put (ignore the rain and wind val­ues, the sen­sors are not attached at a place where there is wind or rain, the com­plete set is hor­i­zon­tal­ly on the floor instead of ver­ti­cal­ly how they are sup­posed to be; and do not be shocked about the room  val­ues, it is the “serv­er room” in the basement):

2010-09-02 12:55:01;52.0;18.700;30.0;27.000;978.600;0.0;0.0;135.0;4.200
2010-09-02 12:50:01;53.0;18.700;30.0;27.500;978.600;0.0;0.0;135.0;4.200
2010-09-02 12:45:01;53.0;18.600;30.0;27.300;978.500;0.0;0.0;135.0;4.200
2010-09-02 12:40:01;53.0;18.600;30.0;27.800;978.600;0.0;0.0;135.0;4.200
2010-09-02 12:35:01;53.0;18.600;30.0;27.700;978.500;0.0;0.0;135.0;4.200
2010-09-02 12:30:01;54.0;18.500;31.0;27.500;978.700;0.0;0.0;135.0;4.200

Some things I came up with myself:

  • A line-graph of the val­ues dur­ing a day/week/month.
  • A graph of the amount of accu­mu­lat­ed rain per hour/day/week/month/year.
  • The aver­age temperature/humidity (as an error bar, so see the min/max too) per day and night, but what to use as the times where the day starts/ends? I would like to have the day-part cov­er the real day­light time (minus some ramp-up and ramp-down time), but I do not have any idea how to get this for each day and for my region.Does it make sense to do the same per hour (with­out any ramp-up/-down)?
  • Does the com­bi­na­tion of tem­per­a­ture and humid­i­ty and maybe wind tell some­thing? If yes, how to com­bine them and how to inter­pret the result?
  • Sim­i­lar for the pres­sure. I do not know what it tells me, but in a graph I can maybe add some hor­i­zon­tal lines which tell some­thing (rain prob­a­bil­i­ty and maybe dan­ger zones?).
  • For the wind speed the instruc­tion man­u­al comes with a nice table of the beau­fort scale and a cor­re­spond­ing descrip­tion. This can be put into some col­ored hor­i­zon­tal lines which show more or less dan­ger­ous speeds.

For the plot­ting of the data, I intend to use gnu­plot with the CSV data as the input. It should allow me to auto­mate a lot of things, and some of the graphs should also be easy to real­ize inside gnu­plot itself with­out any exter­nal pro­cess­ing, the ques­tion is only how to real­ize it. For exam­ple for the aver­age of some val­ues I do not know if it makes sense to use some­thing else than the arith­metic mean.

Weath­er sta­tion read­out with FreeBSD

A while ago a wind tur­bine was installed not far away from my place. It is far enough to not dis­turb us, and it is near enough to notice that it turns a lot (IIRC I have seen it only once not turning).

This trig­gered a ques­tion. How much ener­gy would such a device (small­er of course) pro­duce at my place?

The answer depends upon sev­er­al fac­tors. The wind speed, the wind direc­tion and the wind-speed-to-power-output curve of the device. If you do not take a device which rotates around the hor­i­zon­tal axis but the ver­ti­cal axis, the wind direc­tion can be tak­en out of the ques­tion (prob­a­bly not com­plete­ly, but to answer my ques­tion this sim­pli­fi­ca­tion should be ok). The output-power curve depends upon the device, and I hope it is easy to get it from the ven­dors. The remain­ing open ques­tion it the wind speed at my place. Is there enough wind with enough speed?

To answer this ques­tion I bought a weath­er sta­tion with an anemome­ter (wind speed sen­sor). I searched a lit­tle bit until I decid­ed to buy a spe­cif­ic one (actu­al­ly I bought three of them, some cowork­ers got inter­est­ed too but they found only much more expen­sive ones, so soon there will be three more weath­er sta­tions in use in Bel­gium, France and Ger­many). The main point is, I can con­nect it to an USB port of a PC and there is some soft­ware for Lin­ux to read out the data. It also comes with some oth­er outdoor-sensors (tem­per­a­ture, rain, wind direc­tion, humid­i­ty, …) and an indoor-control-unit with some inter­nal sen­sors (tem­per­a­ture, humid­i­ty). The user inter­face is main­ly the touch­screen of the control-unit. There is also some Win­dows soft­ware, which is need­ed to pro­gram the inter­val in which the mea­sure­ments are tak­en and saved in the control-unit.

It seems the weath­er sta­tion is pro­duced by Fine Off­set Elec­tron­ics Co.,Ltd and sold with­in dif­fer­ent brands in dif­fer­ent loca­tions. The Lin­ux soft­ware can read all of them, as the ven­dor and prod­uct IDs are not changed.

Port­ing the soft­ware was easy, it uses libusb and I just had to cor­rect a lit­tle prob­lem for the non-portable func­tions which are used (I asked about them on usb@ and the response was that they just got imple­ment­ed upon my request and will be com­mit­ted to HEAD soon). I made a lit­tle patch for the soft­ware to only use them when avail­able (if you have not loaded the USB HID dri­ver, you do not need to care about them) and com­mit­ted it to the Ports Col­lec­tion as astro/fowsr.

Now I just need to attach the out­side sen­sors at the place where I would put the ver­ti­cal axis wind tur­bine, install some toolk­it which takes a series of mea­sure­ments and dis­plays them as a nice graph (while keep­ing all data val­ues) and write some glue code to feed the out­put of fowsr to it. After a year I can then cal­cu­late how much pow­er a giv­en wind tur­bine would have pro­duced dur­ing the year and cal­cu­late the return of invest­ment for it.

The Lin­ux soft­ware also ref­er­ences sev­er­al weath­er sites, for some of them you can get even an iGoogle wid­get so that you can view the data from wher­ev­er you want (as long as you have a suit­able inter­net con­nec­tion). I think this is also some­thing I will have a look at later.

Note to users in Europe, the device also comes with a DCF77 receiv­er. As the time is dis­trib­uted in UTC+1 (or +2, depend­ing on the day­light sav­ing time), you should adjust the time­zone set­ting accord­ing­ly to this, not to plain UTC (so for me the time­zone should be ‘0’ for the same timezone).

I switched my feed reader

Before I have read the news feeds I am inter­est­ed in via the Fire­fox plu­g­in “brief”. It did all I want­ed it to do, but I had all the data and meta­da­ta (all the feeds and read items) only in one brows­er. I was not able to have a shared state at work and at home.

Now I installed rnews on my web­serv­er. It is multi-user capa­ble, so that mul­ti­ple peo­ple can read the feeds they are inter­est­ed in, with­out the need to have mul­ti­ple instal­la­tions. I can use it from any place where I have an inter­net con­nec­tion, with­out los­ing the state.

It is in the FreeB­SD Ports Col­lec­tion as www/rnews.