linux_​base-​c6

Seems I for­got to an­nounce that the linux_​base-​c6 is in the Ports Col­lec­tion now. Well, it is not a re­place­ment for the cur­rent de­fault linux base, the linuxu­lator in­fra­struc­ture ports are miss­ing and we need to check if the ker­nel sup­ports enough of 2.6.18 that noth­ing breaks.

TODO:

  • check for up­dated RPMs for linux_​base-​c6
  • cre­ate linuxu­lator in­fra­struc­ture ports
  • im­prove the ker­nel to sup­port more of linux 2.6.18

To my know­ledge, nobody is work­ing on any­thing of this. Any­one is wel­come to have a look and provide patches.

StumbleUponXINGBalatarinBox.netDiggGoogle GmailNetvouzPlurkSiteJotTypePad PostYahoo BookmarksVKSlashdotPocketHacker NewsDiigoBuddyMarksRedditLinkedInBibSonomyBufferEmailHatenaLiveJournalNewsVinePrintViadeoYahoo MailAIMBitty BrowserCare2 NewsEvernoteMail.RuPrintFriendlyWaneloYahoo MessengerYoolinkWebnewsStumpediaProtopage BookmarksOdnoklassnikiMendeleyInstapaperFarkCiteULikeBlinklistAOL MailTwitterGoogle+PinterestTumblrAmazon Wish ListBlogMarksDZoneDeliciousFlipboardFolkdJamespotMeneameMixiOknotiziePushaSvejoSymbaloo FeedsWhatsAppYouMobdiHITTWordPressRediff MyPageOutlook.comMySpaceDesign FloatBlogger PostApp.netDiary.RuKindle ItNUjijSegnaloTuentiWykopTwiddlaSina WeiboPinboardNetlogLineGoogle BookmarksDiasporaBookmarks.frBaiduFacebookGoogle ClassroomKakaoQzoneSMSTelegramRenrenKnownYummlyShare/​Save

De­bug­ging lang/​mono – 2nd round

Today I had again some en­ergy to look at why mono fails to build on FreeBSD-cur­rent.

I de­cided to do a debug-​build of mono. This did not work ini­tially, I had to pro­duce some patches. 🙁

Does this mean nobody is do­ing de­bug builds of mono on FreeBSD?

I have to say, this ex­per­i­ence with lang/​mono is com­pletely un­sat­is­fy­ing.

Ok, bot­tom line, either the de­bug build seems to pre­vent a race con­di­tion in most cases (I had a lot less lockups for each of the two builds I did).

Whatever it is, I do not care ATM (if the con­fig­ure stuff is look­ing at the ar­chi­tec­ture of the sys­tem, it may be the case that the i386-​portbld-​freebsdX does not en­able some im­port­ant stuff which would be en­abled when run with i486-​portbld-​freebsdX or bet­ter). Here are the patches I used in case someone is in­ter­ested (warn­ing, copy&paste con­ver­ted tabs to spaces, you also have to ap­ply the map.c (a gen­er­ated file… maybe a touch of the right file would al­low to ap­ply this patch in the nor­mal patch stage) re­lated stuff when the build fails, else there is some parser er­ror in mono):

— mcs/​class/​Mono.Posix/Mono.Unix/UnixProcess.cs.orig       2010-​01-​29 11:34:00.592323482 +0100
+++ mcs/class/Mono.Posix/Mono.Unix/UnixProcess.cs    2010-​01-​29 11:34:18.540607357 +0100
@@ –57,7 +57,7 @@ namespace Mono.Unix {
 int r = Nat­ive.Sy­scall.wait­pid (pid, out status,
 Native.WaitOptions.WNOHANG | Native.WaitOptions.WUNTRACED);
 UnixMarshal.ThrowExceptionForLastErrorIf ®; –                       re­turn r;
+                       re­turn status;
 }

 pub­lic int Ex­it­Code {
 — mono/io-layer/processes.c.orig    2010-​01-​29 11:36:08.904331535 +0100
+++ mono/io-layer/processes.c 2010-​01-​29 11:42:21.819159544 +0100
@@ –160,7 +160,7 @@ static gboolean waitfor_​pid (gpointer te
 ret = wait­pid (process->id, &status, WNOHANG);
 } while (er­rno == EINTR);
 –       if (ret <= 0) {
+       if (ret == 0 || (ret < 0 && er­rno != ECHILD)) {
 /​* Pro­cess not ready for wait */​
 #if­def DEBUG
 g_​message (“%s: Pro­cess %d not ready for wait­ing for: %s”,
@@ –169,6 +169,17 @@ static gboolean waitfor_​pid (gpointer te

 re­turn (FALSE);
 }
+
+       if (ret < 0 && er­rno == ECHILD) {
+#if­def DEBUG
+               g_​message (“%s: Pro­cess %d does not ex­ist (any­more)”, _​_​func_​_​,
+                          process->id);
+#en­dif
+               /​* Fak­ing the re­turn status. I do not know if it is cor­rect
+                * to as­sume a suc­cess­ful exit.
+                */​
+               status = 0;
+       }

 #if­def DEBUG
 g_​message (“%s: Pro­cess %d fin­ished”, _​_​func_​_​, ret);
 — mono/metadata/mempool.c.orig      2010-​01-​29 11:58:16.871052861 +0100
+++ mono/metadata/mempool.c   2010-​01-​29 12:30:45.143367454 +0100
@@ –212,12 +212,14 @@ mono_​backtrace (int size)

         En­ter­Crit­ic­alSec­tion (&mempool_​tracing_​lock);
         g_​print (“Al­loc­at­ing %d bytes\n”, size);
+#if defined(HAVE_BACKTRACE_SYMBOLS)
         sym­bols = back­trace (ar­ray, BACKTRACE_​DEPTH);
         names = backtrace_​symbols (ar­ray, sym­bols);
         for (i = 1; i < sym­bols; ++i) {
                 g_​print (“\t%s\n”, names [i]);
         }
         free (names);
+#en­dif
         LeaveCrit­ic­alSec­tion (&mempool_​tracing_​lock);
 }
 — mono/metadata/metadata.c.orig     2010-​01-​29 11:59:38.552316989 +0100
+++ mono/metadata/metadata.c  2010-​01-​29 12:00:43.957337476 +0100
@@ –3673,12 +3673,16 @@ mono_​backtrace (int limit)
         void *array[limit];
         char **names;
         int i;
+#if defined(HAVE_BACKTRACE_SYMBOLS)
         back­trace (ar­ray, limit);
         names = backtrace_​symbols (ar­ray, limit);
         for (i =0; i < limit; ++i) {
                 g_​print (“\t%s\n”, names [i]);
         }
         g_​free (names);
+#else
+       g_​print (“No back­trace available.\n”);
+#en­dif
 }
 #en­dif
 — support/map.c.orig        2010-​01-​29 12:05:22.374653708 +0100
+++ support/map.c 2010-​01-​29 12:10:29.024412452 +0100
@@ –216,7 +216,7 @@
 #define _cnm_dump(to_t, from) do {} while (0)
 #en­dif /​* def _​CNM_​DUMP */​

-#if­def DEBUG
+#if defined(DEBUG) && !defined(__FreeBSD__)
 #define _cnm_return_val_if_overflow(to_t,from,val)  G_​STMT_​START {   \
         int     uns = _​cnm_​integral_​type_​is_​unsigned (to_​t);             \
         gint64  min = (gint64)  _​cnm_​integral_​type_​min (to_​t);           \

I merged a lot of ZFS patches to 7-​stable

Dur­ing the last weeks I iden­ti­fied 64 patches for ZFS which are in 8–stable but not in 7-​stable. For 56 of them I had a deeper look and most of them are com­mited now to 7-​stable. The ones of those 56 which I did not com­mit are not ap­plic­able to 7-​stable (in­fra­struc­ture dif­fer­ences between 8 and 7).

Un­for­tu­nately this did not solve the sta­bil­ity prob­lems I have on a 7-​stable sys­tem.

I also com­mit­ted a diff re­duc­tion (between 8-​stable and 7-​stable) patch which also fixed some not so harm­less mis­merges (mem-​leak and ini­tial­iz­ing the same mu­tex twice at dif­fer­ent places). No idea yet if it helps in my case.

I also want to merge the new arc re­claim lo­gic from head to 8-​stable and 7-​stable. Maybe I can do this to­mor­row.

Cur­rently I run a test with a ker­nel where the shared locks for ZFS are switched to ex­clus­ive locks.

Test­ing tarsnap

I am im­pressed. Yes, really. It seems tarsnap DTRT.

I made a test with tarsnap. I made a backup of some data (a full backup of everything is kept on a ZFS volume on an ex­ternal disk which is only at­tached to make a full backup once in a while) of one of my sys­tems. This data is 1.1 GB in size (most of it is /​usr/​src checked out via sub­ver­sion and ex­ten­ded with some patches – no mu­sic, pic­tures or other such data). This com­presses down to 325 MB. Of this 325 MB of data only 242 MB is stored en­cryp­ted on the tarsnap server (auto­matic de-​duplication on the backup cli­ent). The second backup of the same data in the fol­low­ing night (again 1.1 GB in total, 325 MB com­pressed) caused 216 kB of new data to be stored on the tarsnap server (again, de-​duplication on the cli­ent). What I have now are two full off-​site backups of this data (two archives with 1.1 GB of data after de­com­pres­sion), with the be­ne­fit that the ad­di­tional stor­age space re­quired is the stor­age space of an in­cre­mental backup.

The cost (in dol­lar) of this so far is 0.074603634 for the ini­tial trans­fer of the data, 0.00242067783 for the data stor­age on the first day, plus 0.0019572486 for the trans­fer for the second backup. From the ini­tial 29.93 I still have 29.85 (roun­ded) left. If I factor out the ini­tial trans­fer and as­sum­ing that the rate of change for this sys­tem stays con­stant, this comes down to 0.01 (rounded-​up) per day for this sys­tem (or about 8 years of backup if I do not add more sys­tems and do not add more than the ini­tial 29.93 (= EUR 20) – and the price of this ser­vice does not in­crease, off course). Is this data worth 1 cent per day for me? Yes, for sure! Even more, but hey, you did not read this here. 🙂

That is what you get when a per­son designs a ser­vice which he is will­ing to use him­self for a price he wants to pay him­self (while still not lose money with the ser­vice).

Colin, again, I am im­pressed. Big thumbs up for you!

Now I just have to add some more sys­tems (/​etc and sim­ilar of vari­ous jails… and of course the data of this blog) and pol­ish my tarsnap script for “peri­odic daily” .

P.S.: Yes there are also places to im­prove, I found already some things (the con­fig file parser is a little bit strict what it ac­cepts, and some things should be more doc­u­mented) but Colin is re­spons­ive and open to im­prove­ment sug­ges­tions.

Daily doxy­gen gen­er­ated docs of the FreeBSD ker­nel (head)

I man­aged to get some time to setup an auto­mated gen­er­a­tion of the doxy­gen docs for ker­nel sub­sys­tems of FreeBSD on my web­server.

Every night/​morning (Ger­man timezone) the sources will be up­dated, and the docs get re­gen­er­ated (this takes some time). Cur­rently this de­pends upon some patches to the make­file and doxy­gen con­fig files in tools/​kerneldoc/​subsys. Everything is gen­er­ated dir­ectly in the place where the web­server will look for to de­liver the pages, so if you browse this in the middle of the gen­er­a­tion, the con­tent may not be con­sist­ent (yet).

Please be nice to the web­server and do not mir­ror this. You can gen­er­ate this your­self very easy. As­sum­ing you have the FreeBSD source on a local hard disk, you just need to down­load the patch from http://​www​.Leidinger​.net/​F​r​e​e​B​S​D​/​c​u​r​r​e​n​t​-​p​a​t​c​h​es/ (if you do not find dox.diff, up­date your FreeBSD sources and everything will be OK), ap­ply the patch, cd into tools/​kerneldoc/​subsys and run “make all” (or “make vm” or whatever you are in­ter­ested in). You need doxy­gen in­stalled, off course.

If you want to setup some­thing like this your­self, just down­load the script which is do­ing all the work, change some vari­ables in the be­gin­ning, and cre­ate your own local ver­sion of the com­plete docs.

In case this is us­ing sig­ni­fic­ant traffic, I will ask core/​ad­mins if there is the pos­sib­il­ity to host it on FreeBSD.org re­sources.