linux_base-c6

Seems I for­got to announce that the linux_base-c6 is in the Ports Col­lec­tion now. Well, it is not a replace­ment for the cur­rent default lin­ux base, the lin­ux­u­la­tor infra­struc­ture ports are miss­ing and we need to check if the ker­nel sup­ports enough of 2.6.18 that noth­ing breaks.

TODO:

  • check for updat­ed RPMs for linux_base-c6
  • cre­ate lin­ux­u­la­tor infra­struc­ture ports
  • improve the ker­nel to sup­port more of lin­ux 2.6.18

To my knowl­edge, nobody is work­ing on any­thing of this. Any­one is wel­come to have a look and pro­vide patches.

Debug­ging lang/mono – 2nd round

Today I had again some ener­gy to look at why mono fails to build on FreeBSD-current.

I decid­ed to do a debug-build of mono. This did not work ini­tial­ly, I had to pro­duce some patches. 🙁

Does this mean nobody is doing debug builds of mono on FreeBSD?

I have to say, this expe­ri­ence with lang/mono is com­plete­ly unsatisfying.

Ok, bot­tom line, either the debug build seems to pre­vent a race con­di­tion in most cas­es (I had a lot less lock­ups for each of the two builds I did).

What­ev­er it is, I do not care ATM (if the con­fig­ure stuff is look­ing at the archi­tec­ture of the sys­tem, it may be the case that the i386-portbld-freebsdX does not enable some impor­tant stuff which would be enabled when run with i486-portbld-freebsdX or bet­ter). Here are the patch­es I used in case some­one is inter­est­ed (warn­ing, copy&paste con­vert­ed tabs to spaces, you also have to apply the map.c (a gen­er­at­ed file… maybe a touch of the right file would allow to apply this patch in the nor­mal patch stage) relat­ed stuff when the build fails, else there is some pars­er error in mono):

--- mcs/class/Mono.Posix/Mono.Unix/UnixProcess.cs.orig       2010-01-29 11:34:00.592323482 +0100
+++ mcs/class/Mono.Posix/Mono.Unix/UnixProcess.cs    2010-01-29 11:34:18.540607357 +0100
@@ -57,7 +57,7 @@ namespace Mono.Unix {
 int r = Native.Syscall.waitpid (pid, out status,
 Native.WaitOptions.WNOHANG | Native.WaitOptions.WUNTRACED);
 UnixMarshal.ThrowExceptionForLastErrorIf (r);
-                       return r;
+                       return status;
 }

 public int ExitCode {

--- mono/io-layer/processes.c.orig    2010-01-29 11:36:08.904331535 +0100
+++ mono/io-layer/processes.c 2010-01-29 11:42:21.819159544 +0100
@@ -160,7 +160,7 @@ static gboolean waitfor_pid (gpointer te
 ret = waitpid (process->id, &status, WNOHANG);
 } while (errno == EINTR);

-       if (ret <= 0) {
+       if (ret == 0 || (ret < 0 && errno != ECHILD)) {
 /* Process not ready for wait */
 #ifdef DEBUG
 g_message ("%s: Process %d not ready for waiting for: %s",
@@ -169,6 +169,17 @@ static gboolean waitfor_pid (gpointer te

 return (FALSE);
 }
+
+       if (ret < 0 && errno == ECHILD) {
+#ifdef DEBUG
+               g_message ("%s: Process %d does not exist (anymore)", __func__,
+                          process->id);
+#endif
+               /* Faking the return status. I do not know if it is correct
+                * to assume a successful exit.
+                */
+               status = 0;
+       }

 #ifdef DEBUG
 g_message ("%s: Process %d finished", __func__, ret);

--- mono/metadata/mempool.c.orig      2010-01-29 11:58:16.871052861 +0100
+++ mono/metadata/mempool.c   2010-01-29 12:30:45.143367454 +0100
@@ -212,12 +212,14 @@ mono_backtrace (int size)

         EnterCriticalSection (&mempool_tracing_lock);
         g_print ("Allocating %d bytes\n", size);
+#if defined(HAVE_BACKTRACE_SYMBOLS)
         symbols = backtrace (array, BACKTRACE_DEPTH);
         names = backtrace_symbols (array, symbols);
         for (i = 1; i < symbols; ++i) {
                 g_print ("\t%s\n", names [i]);
         }
         free (names);
+#endif
         LeaveCriticalSection (&mempool_tracing_lock);
 }

--- mono/metadata/metadata.c.orig     2010-01-29 11:59:38.552316989 +0100
+++ mono/metadata/metadata.c  2010-01-29 12:00:43.957337476 +0100
@@ -3673,12 +3673,16 @@ mono_backtrace (int limit)
         void *array[limit];
         char **names;
         int i;
+#if defined(HAVE_BACKTRACE_SYMBOLS)
         backtrace (array, limit);
         names = backtrace_symbols (array, limit);
         for (i =0; i < limit; ++i) {
                 g_print ("\t%s\n", names [i]);
         }
         g_free (names);
+#else
+       g_print ("No backtrace available.\n");
+#endif
 }
 #endif

--- support/map.c.orig        2010-01-29 12:05:22.374653708 +0100
+++ support/map.c 2010-01-29 12:10:29.024412452 +0100
@@ -216,7 +216,7 @@
 #define _cnm_dump(to_t, from) do {} while (0)
 #endif /* def _CNM_DUMP */

-#ifdef DEBUG
+#if defined(DEBUG) && !defined(__FreeBSD__)
 #define _cnm_return_val_if_overflow(to_t,from,val)  G_STMT_START {   \
         int     uns = _cnm_integral_type_is_unsigned (to_t);             \
         gint64  min = (gint64)  _cnm_integral_type_min (to_t);           \

I merged a lot of ZFS patch­es to 7‑stable

Dur­ing the last weeks I iden­ti­fied 64 patch­es for ZFS which are in 8‑stable but not in 7‑stable. For 56 of them I had a deep­er look and most of them are com­mit­ed now to 7‑stable. The ones of those 56 which I did not com­mit are not applic­a­ble to 7‑stable (infra­struc­ture dif­fer­ences between 8 and 7).

Unfor­tu­nate­ly this did not solve the sta­bil­i­ty prob­lems I have on a 7‑stable system.

I also com­mit­ted a diff reduc­tion (between 8‑stable and 7‑stable) patch which also fixed some not so harm­less mis­merges (mem-leak and ini­tial­iz­ing the same mutex twice at dif­fer­ent places). No idea yet if it helps in my case.

I also want to merge the new arc reclaim log­ic from head to 8‑stable and 7‑stable. Maybe I can do this tomorrow.

Cur­rent­ly I run a test with a ker­nel where the shared locks for ZFS are switched to exclu­sive locks.

Test­ing tarsnap

I am impressed. Yes, real­ly. It seems tarsnap DTRT.

I made a test with tarsnap. I made a back­up of some data (a full back­up of every­thing is kept on a ZFS vol­ume on an exter­nal disk which is only attached to make a full back­up once in a while) of one of my sys­tems. This data is 1.1 GB in size (most of it is /usr/src checked out via sub­ver­sion and extend­ed with some patch­es – no music, pic­tures or oth­er such data). This com­press­es down to 325 MB. Of this 325 MB of data only 242 MB is stored encrypt­ed on the tarsnap serv­er (auto­mat­ic de-duplication on the back­up client). The sec­ond back­up of the same data in the fol­low­ing night (again 1.1 GB in total, 325 MB com­pressed) caused 216 kB of new data to be stored on the tarsnap serv­er (again, de-duplication on the client). What I have now are two full off-site back­ups of this data (two archives with 1.1 GB of data after decom­pres­sion), with the ben­e­fit that the addi­tion­al stor­age space required is the stor­age space of an incre­men­tal backup.

The cost (in dol­lar) of this so far is 0.074603634 for the ini­tial trans­fer of the data, 0.00242067783 for the data stor­age on the first day, plus 0.0019572486 for the trans­fer for the sec­ond back­up. From the ini­tial 29.93 I still have 29.85 (round­ed) left. If I fac­tor out the ini­tial trans­fer and assum­ing that the rate of change for this sys­tem stays con­stant, this comes down to 0.01 (rounded-up) per day for this sys­tem (or about 8 years of back­up if I do not add more sys­tems and do not add more than the ini­tial 29.93 (= EUR 20) – and the price of this ser­vice does not increase, off course). Is this data worth 1 cent per day for me? Yes, for sure! Even more, but hey, you did not read this here. 🙂

That is what you get when a per­son designs a ser­vice which he is will­ing to use him­self for a price he wants to pay him­self (while still not lose mon­ey with the service).

Col­in, again, I am impressed. Big thumbs up for you!

Now I just have to add some more sys­tems (/etc and sim­i­lar of var­i­ous jails… and of course the data of this blog) and pol­ish my tarsnap script for “peri­od­ic daily” .

P.S.: Yes there are also places to improve, I found already some things (the con­fig file pars­er is a lit­tle bit strict what it accepts, and some things should be more doc­u­ment­ed) but Col­in is respon­sive and open to improve­ment suggestions.

Dai­ly doxy­gen gen­er­at­ed docs of the FreeB­SD ker­nel (head)

I man­aged to get some time to set­up an auto­mat­ed gen­er­a­tion of the doxy­gen docs for ker­nel sub­sys­tems of FreeB­SD on my webserver.

Every night/morning (Ger­man time­zone) the sources will be updat­ed, and the docs get regen­er­at­ed (this takes some time). Cur­rent­ly this depends upon some patch­es to the make­file and doxy­gen con­fig files in tools/kerneldoc/subsys. Every­thing is gen­er­at­ed direct­ly in the place where the web­serv­er will look for to deliv­er the pages, so if you browse this in the mid­dle of the gen­er­a­tion, the con­tent may not be con­sis­tent (yet).

Please be nice to the web­serv­er and do not mir­ror this. You can gen­er­ate this your­self very easy. Assum­ing you have the FreeB­SD source on a local hard disk, you just need to down­load the patch from http://www.Leidinger.net/FreeBSD/current-patches/ (if you do not find dox.diff, update your FreeB­SD sources and every­thing will be OK), apply the patch, cd into tools/kerneldoc/subsys and run “make all” (or “make vm” or what­ev­er you are inter­est­ed in). You need doxy­gen installed, off course.

If you want to set­up some­thing like this your­self, just down­load the script which is doing all the work, change some vari­ables in the begin­ning, and cre­ate your own local ver­sion of the com­plete docs.

In case this is using sig­nif­i­cant traf­fic, I will ask core/admins if there is the pos­si­bil­i­ty to host it on FreeBSD.org resources.