Tarsnap us­age stat­ist­ics

The more time passes with tarsnap, the more im­press­ive it is.

Fol­low­ing is a list of all my privately used sys­tems (2 ma­chines which only host jails – here named Prison1 and Prison2 – and sev­eral jails – here named ac­cord­ing to their func­tion­al­ity) to­gether with some tarsnap stat­ist­ics. For each backup tarsnap prints out some stat­ist­ics. The amount of un­com­pressed stor­age space of all archives of this ma­chine, the com­pressed stor­age space of all archives, the unique un­com­pressed stor­age space of all archives, the unique com­pressed stor­age space of all archives, and the same mount of info for the cur­rent archive. The unique stor­age space is after de­du­plic­a­tion. The most in­ter­est­ing in­form­a­tion is the unique and com­pressed one. For a spe­cific archive it shows the amount of data which is dif­fer­ent to all other archives, and for the total amount it tells how much stor­age space is used on the tarsnap server. I do not backup all data in tarsnap. I do a full backup on ex­ternal stor­age (zfs snap­shot + zfs send | zfs re­ceive) once in a while and tarsnap is only for the stuff which could change daily or is very small (my mails be­long to the first group, the con­fig of ap­plic­a­tions or the sys­tem to the second group). At the end of the post there is also an over­view of the money I have spend so far in tarsnap for the backups.

At­ten­tion: the fol­low­ing graphs are dis­play­ing small val­ues in KB, while the text is telling about sizes in MB or even GB!

Prison1

The backup of one day cov­ers 1.1 GB of un­com­pressed data, the sub­trees I backup are /​etc, /​usr/​local/​etc, /​home, /​root, /​var/​db/​pkg, /var/db/mergemaster.mtree, /​space/​jails/​flavours and a sub­ver­sion check­out of /​usr/​src (ex­clud­ing the ker­nel com­pile dir­ect­ory; I backup this as I have local modi­fic­a­tions to FreeBSD). If I want to have all days un­com­pressed on my hard­disk, I would have to provide 10 GB of stor­age space. Com­pressed this comes down to 2.4 GB, unique un­com­pressed this is 853 MB, and unique com­pressed this is 243 MB. The fol­low­ing graph splits this up into all the backups I have as of this writ­ting. I only show the unique val­ues, as in­clud­ing the total val­ues would make the unique val­ues dis­ap­pear in the graph (val­ues too small). 

chart


In this graph we see that I have a con­stant rate of new data. I think this is mostly ref­er­ences to already stored data (/​usr/​src be­ing the most likely cause of this, noth­ing changed in those dir­ect­or­ies).

Internal-​DNS

One day cov­ers 7 MB of un­com­pressed data, all archives take 56 MB un­com­pressed, unique and com­pressed this comes down to 1.3 MB. This cov­ers /​etc, /​usr/​local/​etc, /​root, /​var/​db/​pkg, /​var/​named, and /var/db/mergemaster.mtree.

chart


This graph is strange. I have no idea why there is so much data for the second and the last day. Noth­ing changed.

Outgoing-​Postfix

One day cov­ers 8 MB of un­com­pressed data, all archives take 62 MB un­com­pressed, unique and com­pressed this comes down to 1.5 MB. This cov­ers /​etc, /​usr/​local/​etc, /​root, /​var/​db/​pkg, /​var/​spool/​postfix, and /var/db/mergemaster.mtree.

chart


This looks not bad. I was send­ing a lot of mails on the 25th. And the days in the middle I was not send­ing much.

IMAP

One day cov­ers about 900 MB of un­com­pressed data, all archives take 7.2 GB un­com­pressed, unique and com­pressed this comes down to 526 MB. This cov­ers /​etc, /​usr/​local/​etc, /​root, /​var/​db/​pkg, /var/db/mergemaster.mtree, /​home (mail folders) and /​usr/​local/​share/​courier-​imap.

chart


Ob­vi­ously I have a not so small amount of change in my mail­box. As my spam­fil­ter is work­ing nicely this is dir­ectly cor­rel­ated to mails from vari­ous mailing­lists (mostly FreeBSD).

MySQL (for the Horde web­mail in­ter­face)

One day cov­ers 100 MB of un­com­pressed data, all archives take 801 MB un­com­pressed, unique and com­pressed this comes down to 19 MB. This cov­ers /​etc, /​usr/​local/​etc, /​root, /​var/​db/​pkg, /​var/​db/​mysql and /var/db/mergemaster.mtree.

chart


This is cor­rel­ated with the use of my web­mail in­ter­face, and as such is also cor­rel­ated with the amount of mails I get and send. Ob­vi­ously I did not use my web­mail in­ter­face at the week­end (as the backup cov­ers the change of the pre­vi­ous day).

Web­mail

One day cov­ers 121 MB of un­com­pressed data, all archives take 973 MB un­com­pressed, unique and com­pressed this comes down to 33 MB. This cov­ers /​etc, /​usr/​local/​etc, /​root, /​var/​db/​pkg, /var/db/mergemaster.mtree, /​usr/​local/​www/​horde and /​home.

chart


This one is strange again. Noth­ing in the data changed.

Samba

One day cov­ers 10 MB of un­com­pressed data, all archives take 72 MB un­com­pressed, unique and com­pressed this comes down to 1.9 MB. This cov­ers /​etc, /​usr/​local/​etc, /​root, /​var/​db/​pkg, /var/db/mergemaster.mtree and /​var/​db/​samba.

chart


Here we see the changes to /​var/​db/​samba, this should be mostly my Wii ac­cess­ing mul­ti­me­dia files there.

Proxy

One day cov­ers 31 MB of un­com­pressed data, all archives take 223 MB un­com­pressed, unique and com­pressed this comes down to 6.6 MB. This cov­ers /​etc, /​usr/​local/​etc, /​root, /​var/​db/​pkg and /var/db/mergemaster.mtree.

chart


This is also a strange graph. Again, noth­ing changed there (the cache dir­ect­ory is not in the backup).

phpMy­Ad­min

One day cov­ers 44 MB of un­com­pressed data, all archives take 310 un­com­pressed, unique and com­pressed this comes down to 11 MB. This cov­ers /​etc, /​usr/​local/​etc, /​root, /​var/​db/​pkg, /var/db/mergemaster.mtree, /​home and /​usr/​local/​www/​phpMyAdmin.

chart


And again a strange graph. No changes in the FS.

Gal­lery

One day cov­ers 120 MB of un­com­pressed data, all archives take 845 MB un­com­pressed, unique and com­pressed this comes down to 25 MB. This cov­ers /​etc, /​usr/​local/​etc, /​root, /​var/​db/​pkg, /var/db/mergemaster.mtree, /​usr/​local/​www/​gallery2 and /​home/​gallery (ex­clud­ing some parts of /​home/​gallery).

chart


This one is OK. Friends and Fam­ily ac­cess­ing the pic­tures.

Prison2

One day cov­ers 7 MB of un­com­pressed data, all archives take 28 MB un­com­pressed, unique and com­pressed this comes down to 1.3 MB. This cov­ers /​etc, /​usr/​local/​etc, /​root, /​var/​db/​pkg, /var/db/mergemaster.mtree, /​space/​jails/​flavours and /​home.

chart


This one looks strange to me again. Same reas­ons as with the pre­vi­ous graphs.

Incoming-​Postfix

One day cov­ers 56 MB of un­com­pressed data, all archives take 225 MB un­com­pressed, unique and com­pressed this  comes down to 5.4 MB. This cov­ers /​etc, /​usr/​local/​etc, /​usr/​local/​www/​postfixadmin, /​root/​, /​var/​db/​pkg, /​var/​db/​mysql, /​var/​spool/​postfix and /var/db/mergemaster.mtree.

chart


This graph looks OK to me.

Blog-​and-​XMPP

One day cov­ers 59 MB of un­com­pressed data, all archives take 478 MB un­com­pressed, unique and com­pressed this comes down to 14 MB. This cov­ers /​etc, /​usr/​local/​etc, /​root, /​home, /​var/​db/​pkg, /var/db/mergemaster.mtree, /​var/​db/​mysql and /​var/​spool/​ejabberd (yes, no backup of the web-​data, I have it in an­other jail, no need to backup it again). 

chart


With the MySQL and XMPP data­bases in the backup, I do not think this graph is wrong.

Totals

The total amount of stored data per sys­tem is: 

chart


Costs

Since I use tarsnap (8 days), I have spend 38 cents, most of this is band­width cost for the trans­fer of the ini­tial backup (29.21 cents). Ac­cord­ing to the graphs, I am cur­rently at about 8 – 14 cents per week (or about half a dol­lar per month) for my backups (I still have a ma­chine to add, and this may in­crease the amount in a sim­ilar way than the Prison1 sys­tem with 2 – 3 jails). The amount of money spend in US-​cents (roun­ded!) per day is: 

chart


StumbleUponXINGBalatarinBox.netDiggGoogle GmailNetvouzPlurkSiteJotTypePad PostYahoo BookmarksVKSlashdotPocketHacker NewsDiigoBuddyMarksRedditLinkedInBibSonomyBufferEmailHatenaLiveJournalNewsVinePrintViadeoYahoo MailAIMBitty BrowserCare2 NewsEvernoteMail.RuPrintFriendlyWaneloYahoo MessengerYoolinkWebnewsStumpediaProtopage BookmarksOdnoklassnikiMendeleyInstapaperFarkCiteULikeBlinklistAOL MailTwitterGoogle+PinterestTumblrAmazon Wish ListBlogMarksDZoneDeliciousFlipboardFolkdJamespotMeneameMixiOknotiziePushaSvejoSymbaloo FeedsWhatsAppYouMobdiHITTWordPressRediff MyPageOutlook.comMySpaceDesign FloatBlogger PostApp.netDiary.RuKindle ItNUjijSegnaloTuentiWykopTwiddlaSina WeiboPinboardNetlogLineGoogle BookmarksDiasporaBookmarks.frBaiduFacebookGoogle ClassroomKakaoQzoneSMSTelegramRenrenKnownYummlyShare/​Save

1 thought on “Tarsnap us­age stat­ist­ics”

  1. I found now the reason why some of the sys­tems show a “big” trans­fer, while noth­ing has changed: There was a change (at least the date of a file), and tarsnap uses 64kb blocks for the de­du­plic­a­tion.

Leave a Reply

Your email address will not be published. Required fields are marked *