Seems I for­got to announce that the linux_base-c6 is in the Ports Col­lec­tion now. Well, it is not a replace­ment for the cur­rent default linux base, the lin­ux­u­la­tor infra­struc­ture ports are miss­ing and we need to check if the ker­nel sup­ports enough of 2.6.18 that noth­ing breaks.

TODO:

• check for updated RPMs for linux_base-c6
• cre­ate lin­ux­u­la­tor infra­struc­ture ports
• improve the ker­nel to sup­port more of linux 2.6.18

To my knowl­edge, nobody is work­ing on any­thing of this. Any­one is wel­come to have a look and pro­vide patches.

Today I had again some energy to look at why mono fails to build on FreeBSD-cur­rent.

I decided to do a debug-build of mono. This did not work ini­tially, I had to pro­duce some patches.

Does this mean nobody is doing debug builds of mono on FreeBSD?

I have to say, this expe­ri­ence with lang/mono is com­pletely unsatisfying.

Ok, bot­tom line, either the debug build seems to pre­vent a race con­di­tion in most cases (I had a lot less lock­ups for each of the two builds I did).

What­ever it is, I do not care ATM (if the con­fig­ure stuff is look­ing at the archi­tec­ture of the sys­tem, it may be the case that the i386-portbld-freebsdX does not enable some impor­tant stuff which would be enabled when run with i486-portbld-freebsdX or bet­ter). Here are the patches I used in case some­one is inter­ested (warn­ing, copy&paste con­verted tabs to spaces, you also have to apply the map.c (a gen­er­ated file… maybe a touch of the right file would allow to apply this patch in the nor­mal patch stage) related stuff when the build fails, else there is some parser error in mono):

--- mcs/class/Mono.Posix/Mono.Unix/UnixProcess.cs.orig       2010-01-29 11:34:00.592323482 +0100
+++ mcs/class/Mono.Posix/Mono.Unix/UnixProcess.cs    2010-01-29 11:34:18.540607357 +0100
@@ -57,7 +57,7 @@ namespace Mono.Unix {
 int r = Native.Syscall.waitpid (pid, out status,
 Native.WaitOptions.WNOHANG | Native.WaitOptions.WUNTRACED);
 UnixMarshal.ThrowExceptionForLastErrorIf (r);
-                       return r;
+                       return status;
 }

 public int ExitCode {

--- mono/io-layer/processes.c.orig    2010-01-29 11:36:08.904331535 +0100
+++ mono/io-layer/processes.c 2010-01-29 11:42:21.819159544 +0100
@@ -160,7 +160,7 @@ static gboolean waitfor_pid (gpointer te
 ret = waitpid (process->id, &status, WNOHANG);
 } while (errno == EINTR);

-       if (ret <= 0) {
+       if (ret == 0 || (ret < 0 && errno != ECHILD)) {
 /* Process not ready for wait */
 #ifdef DEBUG
 g_message ("%s: Process %d not ready for waiting for: %s",
@@ -169,6 +169,17 @@ static gboolean waitfor_pid (gpointer te

 return (FALSE);
 }
+
+       if (ret < 0 && errno == ECHILD) {
+#ifdef DEBUG
+               g_message ("%s: Process %d does not exist (anymore)", __func__,
+                          process->id);
+#endif
+               /* Faking the return status. I do not know if it is correct
+                * to assume a successful exit.
+                */
+               status = 0;
+       }

 #ifdef DEBUG
 g_message ("%s: Process %d finished", __func__, ret);

--- mono/metadata/mempool.c.orig      2010-01-29 11:58:16.871052861 +0100
+++ mono/metadata/mempool.c   2010-01-29 12:30:45.143367454 +0100
@@ -212,12 +212,14 @@ mono_backtrace (int size)

         EnterCriticalSection (&mempool_tracing_lock);
         g_print ("Allocating %d bytesn", size);
+#if defined(HAVE_BACKTRACE_SYMBOLS)
         symbols = backtrace (array, BACKTRACE_DEPTH);
         names = backtrace_symbols (array, symbols);
         for (i = 1; i < symbols; ++i) {
                 g_print ("t%sn", names [i]);
         }
         free (names);
+#endif
         LeaveCriticalSection (&mempool_tracing_lock);
 }

--- mono/metadata/metadata.c.orig     2010-01-29 11:59:38.552316989 +0100
+++ mono/metadata/metadata.c  2010-01-29 12:00:43.957337476 +0100
@@ -3673,12 +3673,16 @@ mono_backtrace (int limit)
         void *array[limit];
         char **names;
         int i;
+#if defined(HAVE_BACKTRACE_SYMBOLS)
         backtrace (array, limit);
         names = backtrace_symbols (array, limit);
         for (i =0; i < limit; ++i) {
                 g_print ("t%sn", names [i]);
         }
         g_free (names);
+#else
+       g_print ("No backtrace available.n");
+#endif
 }
 #endif

--- support/map.c.orig        2010-01-29 12:05:22.374653708 +0100
+++ support/map.c 2010-01-29 12:10:29.024412452 +0100
@@ -216,7 +216,7 @@
 #define _cnm_dump(to_t, from) do {} while (0)
 #endif /* def _CNM_DUMP */

-#ifdef DEBUG
+#if defined(DEBUG) && !defined(__FreeBSD__)
 #define _cnm_return_val_if_overflow(to_t,from,val)  G_STMT_START {   
         int     uns = _cnm_integral_type_is_unsigned (to_t);             
         gint64  min = (gint64)  _cnm_integral_type_min (to_t);           

Dur­ing the last weeks I iden­ti­fied 64 patches for ZFS which are in 8–sta­ble but not in 7-stable. For 56 of them I had a deeper look and most of them are com­mited now to 7-stable. The ones of those 56 which I did not com­mit are not applic­a­ble to 7-stable (infra­struc­ture dif­fer­ences between 8 and 7).

Unfor­tu­nately this did not solve the sta­bil­ity prob­lems I have on a 7–sta­ble sys­tem.

I also com­mit­ted a diff reduc­tion (between 8-stable and 7-stable) patch which also fixed some not so harm­less mis­merges (mem-leak and ini­tial­iz­ing the same mutex twice at dif­fer­ent places). No idea yet if it helps in my case.

I also want to merge the new arc reclaim logic from head to 8-stable and 7-stable. Maybe I can do this tomorrow.

Cur­rently I run a test with a ker­nel where the shared locks for ZFS are switched to exclu­sive locks.

I am impressed. Yes, really. It seems tarsnap DTRT.

I made a test with tarsnap. I made a backup of some data (a full backup of every­thing is kept on a ZFS vol­ume on an exter­nal disk which is only attached to make a full backup once in a while) of one of my sys­tems. This data is 1.1 GB in size (most of it is /usr/src checked out via sub­ver­sion and extended with some patches — no music, pic­tures or other such data). This com­presses down to 325 MB. Of this 325 MB of data only 242 MB is stored encrypted on the tarsnap server (auto­matic de-duplication on the backup client). The sec­ond backup of the same data in the fol­low­ing night (again 1.1 GB in total, 325 MB com­pressed) caused 216 kB of new data to be stored on the tarsnap server (again, de-duplication on the client). What I have now are two full off-site back­ups of this data (two archives with 1.1 GB of data after decom­pres­sion), with the ben­e­fit that the addi­tional stor­age space required is the stor­age space of an incre­men­tal backup.

The cost (in dol­lar) of this so far is 0.074603634 for the ini­tial trans­fer of the data, 0.00242067783 for the data stor­age on the first day, plus 0.0019572486 for the trans­fer for the sec­ond backup. From the ini­tial 29.93 I still have 29.85 (rounded) left. If I fac­tor out the ini­tial trans­fer and assum­ing that the rate of change for this sys­tem stays con­stant, this comes down to 0.01 (rounded-up) per day for this sys­tem (or about 8 years of backup if I do not add more sys­tems and do not add more than the ini­tial 29.93 (= EUR 20) — and the price of this ser­vice does not increase, off course). Is this data worth 1 cent per day for me? Yes, for sure! Even more, but hey, you did not read this here.

That is what you get when a per­son designs a ser­vice which he is will­ing to use him­self for a price he wants to pay him­self (while still not lose money with the service).

Colin, again, I am impressed. Big thumbs up for you!

Now I just have to add some more sys­tems (/etc and sim­i­lar of var­i­ous jails… and of course the data of this blog) and pol­ish my tarsnap script for “peri­odic daily” .

P.S.: Yes there are also places to improve, I found already some things (the con­fig file parser is a lit­tle bit strict what it accepts, and some things should be more doc­u­mented) but Colin is respon­sive and open to improve­ment sug­ges­tions.

I man­aged to get some time to setup an auto­mated gen­er­a­tion of the doxy­gen docs for ker­nel sub­sys­tems of FreeBSD on my web­server.

Every night/morning (Ger­man time­zone) the sources will be updated, and the docs get regen­er­ated (this takes some time). Cur­rently this depends upon some patches to the make­file and doxy­gen con­fig files in tools/kerneldoc/subsys. Every­thing is gen­er­ated directly in the place where the web­server will look for to deliver the pages, so if you browse this in the mid­dle of the gen­er­a­tion, the con­tent may not be con­sis­tent (yet).

Please be nice to the web­server and do not mir­ror this. You can gen­er­ate this your­self very easy. Assum­ing you have the FreeBSD source on a local hard disk, you just need to down­load the patch from http://www.Leidinger.net/FreeBSD/current-patches/ (if you do not find dox.diff, update your FreeBSD sources and every­thing will be OK), apply the patch, cd into tools/kerneldoc/subsys and run “make all” (or “make vm” or what­ever you are inter­ested in). You need doxy­gen installed, off course.

If you want to setup some­thing like this your­self, just down­load the script which is doing all the work, change some vari­ables in the begin­ning, and cre­ate your own local ver­sion of the com­plete docs.

In case this is using sig­nif­i­cant traf­fic, I will ask core/admins if there is the pos­si­bil­ity to host it on FreeBSD.org resources.