Category: Jails

IPv6 in my LAN

After enabling IPv6 in my WLAN router, I also enabled IPv6 in my FreeB­SD sys­tems. I have to tell that the IPv6 chap­ter in the FreeB­SD hand­book does not con­tain as much infor­ma­tion as I would like to have about this.

Con­fig­ur­ing the inter­faces of my two 9‑current sys­tems to also car­ry a spe­cif­ic IPv6 address (an easy one from the ULA I use) was easy after read­ing the man-page for rc.conf. After a lit­tle bit of exper­i­ment­ing it came down to:

ifconfig_rl0_ipv6=“inet6 ::2:1 pre­fixlen 64 accept_rtadv”
ipv6_defaultrouter=”<router address>”

Apart from this address (I chose it because the IPv4 address ends in “.2”, this way I can add some easy to remem­ber address­es for this machine if need­ed), I also have two auto­mat­i­cal­ly con­fig­ured address­es. One is with the same ULA and some not so easy to remem­ber end (con­struct­ed from the MAC address), and one is from the offi­cial pre­fix the router con­struct­ed out of the offi­cial IPv4 address from the ISP (+ the same end than the oth­er end).

Addi­tion­al­ly I also have all my jails on this machine with an IPv6 address now (yes, they are like “…:2:100” with the :100 because the IPv4 address ends in “.100”). Still TODO is the con­ver­sion of all the ser­vices in the jails to also lis­ten on the IPv6 address.

I already changed the con­fig of my inter­nal DNS to have the IPv6 address­es for all sys­tems, lis­ten on the IPv6 address (when I add an IPv6 net­work to allow-query/allow-query-cache/allow-recursion bind does not want to start). And as I was there, I also enabled the DNSSEC ver­i­fi­ca­tion (but I get a lot of error mes­sages in the logs: “unable to con­vert errno to isc_result: 42: Pro­to­col not avail­able”, one search result which talks exact­ly about this error tells it is a “cos­met­ic error”…).

I noticed that an IPv6 ping between two phys­i­cal machines takes a lit­tle bit more time than an IPv4 ping (no IPsec enabled). It sur­prised me that this is such a notice­able dif­fer­ence (not with­in the std-dev at all):

— m87.Leidinger.net ping statistics —
10 pack­ets trans­mit­ted, 10 pack­ets received, 0.0% pack­et loss
round-trip min/avg/max/stddev = 0.168÷0.193÷0.220÷0.017 ms

— m87.Leidinger.net ping6 statistics —
10 pack­ets trans­mit­ted, 10 pack­ets received, 0.0% pack­et loss
round-trip min/avg/max/std-dev = 0.207÷0.325÷0.370÷0.047 ms

The infor­ma­tion I miss in the FreeB­SD hand­book in the IPv6 chap­ter is what those oth­er IPv6 relat­ed ser­vices are and when/how to con­fig­ure them. I have an idea now what this rad­vd is, but I am not sure what the inter­ac­tion is with the accept_rtadv set­ting for ifcon­fig (and I do not think I need it, as my WLAN router seems to do it already). I know that I get the IPv6-friendly net­work neigh­bor­hood dis­played with ndp(8). I did not have a look at enabling IPv6 mul­ti­cast sup­port in FreeB­SD, and I do not know what those oth­er IPv6 options for rc.conf do.

Share/Save

Sta­bi­liz­ing 7‑stable…

The 7‑stable sys­tem on which I have sta­bil­i­ty prob­lems after an update from 7.1 to 7.2/7‑stable is now semi-stable.

The watch­dog reboots after one minute of no reac­tion (cur­rent­ly it is able to run 3 – 4 hours), and the jails come up with­out prob­lems now.

The prob­lem with the jails was, that e.g. the mysql-server start­up went into the STOP state because TTY-input was “request­ed”. I solved the prob­lem by using /dev/null as input on jail-startup. On ‑cur­rent I do not see this behav­ior (I have a 9‑current sys­tem with a lot of jails which reboots every X days, and there mysql does not go into the STOP state).

I also start the jails in the back­ground, so that one block­ing jail does not block every­thing (done like in ‑cur­rent).

To say this with code:

--- /usr/src/etc/rc.d/jail      2009-02-07 15:04:35.000000000 +0100
+++ /etc/rc.d/jail      2009-12-16 17:03:12.000000000 +0100
@@ -556,7 +556,8 @@
 fi
 _tmp_jail=${_tmp_dir}/jail.$$
 eval ${_setfib} jail ${_flags} -i ${_rootdir} ${_hostname} \
-                       \\"${_addrl}\\" ${_exec_start} > ${_tmp_jail} 2>&1
+                       \\"${_addrl}\\" ${_exec_start} > ${_tmp_jail} 2>&1 \\
+                       </dev/null

 if [ "$?" -eq 0 ] ; then
 _jail_id=$(head -1 ${_tmp_jail})
@@ -623,4 +624,4 @@
 if [ -n "$*" ]; then
 jail_list="$*"
 fi
-run_rc_command "${cmd}"
+run_rc_command "${cmd}" &

I also iden­ti­fied 57 patch­es for ZFS which are in 8‑stable, but not in 7‑stable (I do not think they could solve the dead­lock, but I do not real­ly know, and now that there is one FS on ZFS, I would like to get as much fixed as pos­si­ble). Some of them should be merged, some would be nice to merge, and some I do not care much about (but if they are easy to merge, why not…). I already have all revi­sions and the cor­re­spond­ing com­mit logs avail­able in an email-draft.

Now I just need to write a lit­tle bit of text and find some peo­ple will­ing to help (some of the changes need a review if they are applic­a­ble to 7‑stable, and every­thing should be test­ed on a scratch-box).

Share/Save

Tarsnap usage statistics

The more time pass­es with tarsnap, the more impres­sive it is.

Fol­low­ing is a list of all my pri­vate­ly used sys­tems (2 machines which only host jails – here named Prison1 and Prison2 – and sev­er­al jails – here named accord­ing to their func­tion­al­i­ty) togeth­er with some tarsnap sta­tis­tics. For each back­up tarsnap prints out some sta­tis­tics. The amount of uncom­pressed stor­age space of all archives of this machine, the com­pressed stor­age space of all archives, the unique uncom­pressed stor­age space of all archives, the unique com­pressed stor­age space of all archives, and the same mount of info for the cur­rent archive. The unique stor­age space is after dedu­pli­ca­tion. The most inter­est­ing infor­ma­tion is the unique and com­pressed one. For a spe­cif­ic archive it shows the amount of data which is dif­fer­ent to all oth­er archives, and for the total amount it tells how much stor­age space is used on the tarsnap serv­er. I do not back­up all data in tarsnap. I do a full back­up on exter­nal stor­age (zfs snap­shot + zfs send | zfs receive) once in a while and tarsnap is only for the stuff which could change dai­ly or is very small (my mails belong to the first group, the con­fig of appli­ca­tions or the sys­tem to the sec­ond group). At the end of the post there is also an overview of the mon­ey I have spend so far in tarsnap for the backups.

Atten­tion: the fol­low­ing graphs are dis­play­ing small val­ues in KB, while the text is telling about sizes in MB or even GB!

Prison1

The back­up of one day cov­ers 1.1 GB of uncom­pressed data, the sub­trees I back­up are /etc, /usr/local/etc, /home, /root, /var/db/pkg, /var/db/mergemaster.mtree, /space/jails/flavours and a sub­ver­sion check­out of /usr/src (exclud­ing the ker­nel com­pile direc­to­ry; I back­up this as I have local mod­i­fi­ca­tions to FreeB­SD). If I want to have all days uncom­pressed on my hard­disk, I would have to pro­vide 10 GB of stor­age space. Com­pressed this comes down to 2.4 GB, unique uncom­pressed this is 853 MB, and unique com­pressed this is 243 MB. The fol­low­ing graph splits this up into all the back­ups I have as of this writ­ting. I only show the unique val­ues, as includ­ing the total val­ues would make the unique val­ues dis­ap­pear in the graph (val­ues too small).

chart


In this graph we see that I have a con­stant rate of new data. I think this is most­ly ref­er­ences to already stored data (/usr/src being the most like­ly cause of this, noth­ing changed in those directories).

Internal-DNS

One day cov­ers 7 MB of uncom­pressed data, all archives take 56 MB uncom­pressed, unique and com­pressed this comes down to 1.3 MB. This cov­ers /etc, /usr/local/etc, /root, /var/db/pkg, /var/named, and /var/db/mergemaster.mtree.

chart


This graph is strange. I have no idea why there is so much data for the sec­ond and the last day. Noth­ing changed.

Outgoing-Postfix

One day cov­ers 8 MB of uncom­pressed data, all archives take 62 MB uncom­pressed, unique and com­pressed this comes down to 1.5 MB. This cov­ers /etc, /usr/local/etc, /root, /var/db/pkg, /var/spool/postfix, and /var/db/mergemaster.mtree.

chart


This looks not bad. I was send­ing a lot of mails on the 25th. And the days in the mid­dle I was not send­ing much.

IMAP

One day cov­ers about 900 MB of uncom­pressed data, all archives take 7.2 GB uncom­pressed, unique and com­pressed this comes down to 526 MB. This cov­ers /etc, /usr/local/etc, /root, /var/db/pkg, /var/db/mergemaster.mtree, /home (mail fold­ers) and /usr/local/share/courier-imap.

chart


Obvi­ous­ly I have a not so small amount of change in my mail­box. As my spam­fil­ter is work­ing nice­ly this is direct­ly cor­re­lat­ed to mails from var­i­ous mail­inglists (most­ly FreeBSD).

MySQL (for the Horde web­mail interface)

One day cov­ers 100 MB of uncom­pressed data, all archives take 801 MB uncom­pressed, unique and com­pressed this comes down to 19 MB. This cov­ers /etc, /usr/local/etc, /root, /var/db/pkg, /var/db/mysql and /var/db/mergemaster.mtree.

chart


This is cor­re­lat­ed with the use of my web­mail inter­face, and as such is also cor­re­lat­ed with the amount of mails I get and send. Obvi­ous­ly I did not use my web­mail inter­face at the week­end (as the back­up cov­ers the change of the pre­vi­ous day).

Web­mail

One day cov­ers 121 MB of uncom­pressed data, all archives take 973 MB uncom­pressed, unique and com­pressed this comes down to 33 MB. This cov­ers /etc, /usr/local/etc, /root, /var/db/pkg, /var/db/mergemaster.mtree, /usr/local/www/horde and /home.

chart


This one is strange again. Noth­ing in the data changed.

Sam­ba

One day cov­ers 10 MB of uncom­pressed data, all archives take 72 MB uncom­pressed, unique and com­pressed this comes down to 1.9 MB. This cov­ers /etc, /usr/local/etc, /root, /var/db/pkg, /var/db/mergemaster.mtree and /var/db/samba.

chart


Here we see the changes to /var/db/samba, this should be most­ly my Wii access­ing mul­ti­me­dia files there.

Proxy

One day cov­ers 31 MB of uncom­pressed data, all archives take 223 MB uncom­pressed, unique and com­pressed this comes down to 6.6 MB. This cov­ers /etc, /usr/local/etc, /root, /var/db/pkg and /var/db/mergemaster.mtree.

chart


This is also a strange graph. Again, noth­ing changed there (the cache direc­to­ry is not in the backup).

php­MyAd­min

One day cov­ers 44 MB of uncom­pressed data, all archives take 310 uncom­pressed, unique and com­pressed this comes down to 11 MB. This cov­ers /etc, /usr/local/etc, /root, /var/db/pkg, /var/db/mergemaster.mtree, /home and /usr/local/www/phpMyAdmin.

chart


And again a strange graph. No changes in the FS.

Gallery

One day cov­ers 120 MB of uncom­pressed data, all archives take 845 MB uncom­pressed, unique and com­pressed this comes down to 25 MB. This cov­ers /etc, /usr/local/etc, /root, /var/db/pkg, /var/db/mergemaster.mtree, /usr/local/www/gallery2 and /home/gallery (exclud­ing some parts of /home/gallery).

chart


This one is OK. Friends and Fam­i­ly access­ing the pictures.

Prison2

One day cov­ers 7 MB of uncom­pressed data, all archives take 28 MB uncom­pressed, unique and com­pressed this comes down to 1.3 MB. This cov­ers /etc, /usr/local/etc, /root, /var/db/pkg, /var/db/mergemaster.mtree, /space/jails/flavours and /home.

chart


This one looks strange to me again. Same rea­sons as with the pre­vi­ous graphs.

Incoming-Postfix

One day cov­ers 56 MB of uncom­pressed data, all archives take 225 MB uncom­pressed, unique and com­pressed this  comes down to 5.4 MB. This cov­ers /etc, /usr/local/etc, /usr/local/www/postfixadmin, /root/, /var/db/pkg, /var/db/mysql, /var/spool/postfix and /var/db/mergemaster.mtree.

chart


This graph looks OK to me.

Blog-and-XMPP

One day cov­ers 59 MB of uncom­pressed data, all archives take 478 MB uncom­pressed, unique and com­pressed this comes down to 14 MB. This cov­ers /etc, /usr/local/etc, /root, /home, /var/db/pkg, /var/db/mergemaster.mtree, /var/db/mysql and /var/spool/ejabberd (yes, no back­up of the web-data, I have it in anoth­er jail, no need to back­up it again). 

chart


With the MySQL and XMPP data­bas­es in the back­up, I do not think this graph is wrong.

Totals

The total amount of stored data per sys­tem is: 

chart


Costs

Since I use tarsnap (8 days), I have spend 38 cents, most of this is band­width cost for the trans­fer of the ini­tial back­up (29.21 cents). Accord­ing to the graphs, I am cur­rent­ly at about 8 – 14 cents per week (or about half a dol­lar per month) for my back­ups (I still have a machine to add, and this may increase the amount in a sim­i­lar way than the Prison1 sys­tem with 2 – 3 jails). The amount of mon­ey spend in US-cents (round­ed!) per day is: 

chart


Share/Save

We got ZFS!

ZFS is there. Great! Thanks Pawel!

Now I wait a lit­tle bit until the first bugs are ironed out, and then I move all my stuff to it. The nice part: when you have 2 machines and every­thing you use is jailed, you just can do this with­out an “inter­rup­tion of ser­vice” (or at least only with a very small one). Just move the jails to the oth­er machine, replace the old FS with ZFS, and then move all jails back.

Share/Save

A desk­top envi­ron­ment in a jail.

Yeah! Final­ly I got time to fin­ish my work to put a desk­top envi­ron­ment (in this case GNOME) into a jail. At least I have a proof of con­cept (I write this with fire­fox run­ning in my “desk­jail”). No, I don’t do this for addi­tion­al secu­ri­ty (there’s more secu­ri­ty than in a non-jailed set­up, but less secu­ri­ty than in an ordi­nary jail, as you have to allow access to a lot more devices than in an ordi­nary jail), I do this for addi­tion­al flex­i­bil­i­ty: Mov­ing my desk­top is now only the install of FreeB­SD on a new machine and rsync­ing the jail over to it. As the machine will also be a host of sev­er­al jails where I have some com­mon users with the same UID in each jail, I don’t pol­lute the jail-host with the desk­top stuff and I have every­thing nice­ly separated.

With­out a ker­nel patch and good devfs rules you will not get Xorg up and run­ning in a jail (at least I did­n’t man­aged to let it rec­og­nize my graph­ic card with­out the ker­nel patch). Now I have to beef up the patch a lit­tle bit and ask for review (it weak­ens up the secu­ri­ty a lit­tle bit like the sysctl security.jail.sysvipc_allowed=1 or security.jail.allow_raw_sockets=1).

But first I have to fin­ish the move of all my ser­vices I use at home to the jail-host now.

Share/Save