Tarsnap us­age stat­ist­ics

The more time passes with tarsnap, the more impressive it is.

Following is a list of all my privately used systems (2 machines which only host jails -- here named Prison1 and Prison2 -- and several jails -- here named according to their functionality) together with some tarsnap statistics. For each backup tarsnap prints out some statistics. The amount of uncompressed storage space of all archives of this machine, the compressed storage space of all archives, the unique uncompressed storage space of all archives, the unique compressed storage space of all archives, and the same mount of info for the current archive. The unique storage space is after deduplication. The most interesting information is the unique and compressed one. For a specific archive it shows the amount of data which is different to all other archives, and for the total amount it tells how much storage space is used on the tarsnap server. I do not backup all data in tarsnap. I do a full backup on external storage (zfs snapshot + zfs send | zfs receive) once in a while and tarsnap is only for the stuff which could change daily or is very small (my mails belong to the first group, the config of applications or the system to the second group). At the end of the post there is also an overview of the money I have spend so far in tarsnap for the backups.

Attention: the following graphs are displaying small values in KB, while the text is telling about sizes in MB or even GB!

Prison1

The backup of one day covers 1.1 GB of uncompressed data, the subtrees I backup are /etc, /usr/local/etc, /home, /root, /var/db/pkg, /var/db/mergemaster.mtree, /space/jails/flavours and a subversion checkout of /usr/src (excluding the kernel compile directory; I backup this as I have local modifications to FreeBSD). If I want to have all days uncompressed on my harddisk, I would have to provide 10 GB of storage space. Compressed this comes down to 2.4 GB, unique uncompressed this is 853 MB, and unique compressed this is 243 MB. The following graph splits this up into all the backups I have as of this writting. I only show the unique values, as including the total values would make the unique values disappear in the graph (values too small).

chart


In this graph we see that I have a constant rate of new data. I think this is mostly references to already stored data (/usr/src being the most likely cause of this, nothing changed in those directories).

Internal-DNS

One day covers 7 MB of uncompressed data, all archives take 56 MB uncompressed, unique and compressed this comes down to 1.3 MB. This covers /etc, /usr/local/etc, /root, /var/db/pkg, /var/named, and /var/db/mergemaster.mtree.

chart


This graph is strange. I have no idea why there is so much data for the second and the last day. Nothing changed.

Outgoing-Postfix

One day covers 8 MB of uncompressed data, all archives take 62 MB uncompressed, unique and compressed this comes down to 1.5 MB. This covers /etc, /usr/local/etc, /root, /var/db/pkg, /var/spool/postfix, and /var/db/mergemaster.mtree.

chart


This looks not bad. I was sending a lot of mails on the 25th. And the days in the middle I was not sending much.

IMAP

One day covers about 900 MB of uncompressed data, all archives take 7.2 GB uncompressed, unique and compressed this comes down to 526 MB. This covers /etc, /usr/local/etc, /root, /var/db/pkg, /var/db/mergemaster.mtree, /home (mail folders) and /usr/local/share/courier-imap.

chart


Obviously I have a not so small amount of change in my mailbox. As my spamfilter is working nicely this is directly correlated to mails from various mailinglists (mostly FreeBSD).

MySQL (for the Horde webmail interface)

One day covers 100 MB of uncompressed data, all archives take 801 MB uncompressed, unique and compressed this comes down to 19 MB. This covers /etc, /usr/local/etc, /root, /var/db/pkg, /var/db/mysql and /var/db/mergemaster.mtree.

chart


This is correlated with the use of my webmail interface, and as such is also correlated with the amount of mails I get and send. Obviously I did not use my webmail interface at the weekend (as the backup covers the change of the previous day).

Webmail

One day covers 121 MB of uncompressed data, all archives take 973 MB uncompressed, unique and compressed this comes down to 33 MB. This covers /etc, /usr/local/etc, /root, /var/db/pkg, /var/db/mergemaster.mtree, /usr/local/www/horde and /home.

chart


This one is strange again. Nothing in the data changed.

Samba

One day covers 10 MB of uncompressed data, all archives take 72 MB uncompressed, unique and compressed this comes down to 1.9 MB. This covers /etc, /usr/local/etc, /root, /var/db/pkg, /var/db/mergemaster.mtree and /var/db/samba.

chart


Here we see the changes to /var/db/samba, this should be mostly my Wii accessing multimedia files there.

Proxy

One day covers 31 MB of uncompressed data, all archives take 223 MB uncompressed, unique and compressed this comes down to 6.6 MB. This covers /etc, /usr/local/etc, /root, /var/db/pkg and /var/db/mergemaster.mtree.

chart


This is also a strange graph. Again, nothing changed there (the cache directory is not in the backup).

phpMyAdmin

One day covers 44 MB of uncompressed data, all archives take 310 uncompressed, unique and compressed this comes down to 11 MB. This covers /etc, /usr/local/etc, /root, /var/db/pkg, /var/db/mergemaster.mtree, /home and /usr/local/www/phpMyAdmin.

chart


And again a strange graph. No changes in the FS.

Gallery

One day covers 120 MB of uncompressed data, all archives take 845 MB uncompressed, unique and compressed this comes down to 25 MB. This covers /etc, /usr/local/etc, /root, /var/db/pkg, /var/db/mergemaster.mtree, /usr/local/www/gallery2 and /home/gallery (excluding some parts of /home/gallery).

chart


This one is OK. Friends and Family accessing the pictures.

Prison2

One day covers 7 MB of uncompressed data, all archives take 28 MB uncompressed, unique and compressed this comes down to 1.3 MB. This covers /etc, /usr/local/etc, /root, /var/db/pkg, /var/db/mergemaster.mtree, /space/jails/flavours and /home.

chart


This one looks strange to me again. Same reasons as with the previous graphs.

Incoming-Postfix

One day covers 56 MB of uncompressed data, all archives take 225 MB uncompressed, unique and compressed this  comes down to 5.4 MB. This covers /etc, /usr/local/etc, /usr/local/www/postfixadmin, /root/, /var/db/pkg, /var/db/mysql, /var/spool/postfix and /var/db/mergemaster.mtree.

chart


This graph looks OK to me.

Blog-and-XMPP

One day covers 59 MB of uncompressed data, all archives take 478 MB uncompressed, unique and compressed this comes down to 14 MB. This covers /etc, /usr/local/etc, /root, /home, /var/db/pkg, /var/db/mergemaster.mtree, /var/db/mysql and /var/spool/ejabberd (yes, no backup of the web-data, I have it in another jail, no need to backup it again).

chart


With the MySQL and XMPP databases in the backup, I do not think this graph is wrong.

Totals

The total amount of stored data per system is:

chart


Costs

Since I use tarsnap (8 days), I have spend 38 cents, most of this is bandwidth cost for the transfer of the initial backup (29.21 cents). According to the graphs, I am currently at about 8-14 cents per week (or about half a dollar per month) for my backups (I still have a machine to add, and this may increase the amount in a similar way than the Prison1 system with 2-3 jails). The amount of money spend in US-cents (rounded!) per day is:

chart


1 thought on “Tarsnap us­age stat­ist­ics”

  1. I found now the reas­on why some of the sys­tems show a “big” trans­fer, while noth­ing has changed: There was a change (at least the date of a file), and tarsnap uses 64kb blocks for the de­du­plic­a­tion.

Leave a Reply

Your email address will not be published. Required fields are marked *