Peri­odic scrub­bing of ZFS pools

I no­ticed that we do not have some auto­matic way of scrub­bing a ZFS pool peri­od­ic­ally. A quick poll on fs@ re­vealed, that there is in­terest in some­thing like this. So I took a little bit of time to write a peri­odic daily script which checks if the last scrub is X days ago and scrubs a pool ac­cord­ingly. The script has op­tions to scrub all pools, or just a spe­cific sub­set. It also al­lows to spe­cify a time–in­ter­val between scrubs for each pool with dif­fer­ent levels of fall-​back (if no pool-​specific in­ter­val is set, the de­fault in­ter­val is used, which is set to 30 days if no other de­fault in­ter­val is spe­cified).

The dis­cus­sion about this is hap­pen­ing over at fs@, so go there and have a look for the CFT (with a link to the WIP of the script) and the dis­cus­sion if you are in­ter­ested.

So far there are some minor de­tails to sort out (and a little bit of doc­u­ment­a­tion to write) be­fore I can com­mit it… prob­ably next week.

StumbleUponXINGBalatarinBox.netDiggGoogle GmailNetvouzPlurkSiteJotTypePad PostYahoo BookmarksVKSlashdotPocketHacker NewsDiigoBuddyMarksRedditLinkedInBibSonomyBufferEmailHatenaLiveJournalNewsVinePrintViadeoYahoo MailAIMBitty BrowserCare2 NewsEvernoteMail.RuPrintFriendlyWaneloYahoo MessengerYoolinkWebnewsStumpediaProtopage BookmarksOdnoklassnikiMendeleyInstapaperFarkCiteULikeBlinklistAOL MailTwitterGoogle+PinterestTumblrAmazon Wish ListBlogMarksDZoneDeliciousFlipboardFolkdJamespotMeneameMixiOknotiziePushaSvejoSymbaloo FeedsWhatsAppYouMobdiHITTWordPressRediff MyPageOutlook.comMySpaceDesign FloatBlogger PostApp.netDiary.RuKindle ItNUjijSegnaloTuentiWykopTwiddlaSina WeiboPinboardNetlogLineGoogle BookmarksDiasporaBookmarks.frBaiduFacebookGoogle ClassroomKakaoQzoneSMSTelegramRenrenKnownYummlyShare/​Save

Mono build prob­lems on FreeBSD-​current

I try to build mono on FreeBSD-cur­rent (it is a de­pend­ency of some GNOME pro­gram). Un­for­tu­nately this does not work cor­rectly.

What I see are hangs of the build. If I stop the build when it hangs and re­start it, it will con­tinue and suc­ceed to pro­cess the build steps a little bit fur­ther, but then it hangs again.

If I ktrace the hanging pro­cess, I see that there is a call to wait re­turn­ing with the er­ror mes­sage that the child does not ex­ist. Then there is a call to nanosleep.

It looks to me like this pro­cess missed some SIGCLD (or is wait­ing for some­thing which did not ex­ist at all), and a loop is wait­ing for a child to exit. This loop prob­ably has no proper con­di­tion for the fact that there is no such child (any­more). As such it will stay forever in this loop.

So I grepped a litte bit around in mono and found the fol­low­ing code in <mono-src-dir>/mcs/class/Mono.Posix/Mono.Unix/UnixProcess.cs:

pub­lic void Wait­F­orExit ()
{
    int status;
    int r;
    do {
        r = Nat­ive.Sy­scall.wait­pid (pid, out status, (Native.WaitOptions) 0);
    } while (UnixMarshal.ShouldRetrySyscall ®);
    UnixMarshal.ThrowExceptionForLastErrorIf ®;
}

This does look a little bit as it could be re­lated to the prob­lem I see, but ShouldRetrySy­scall only re­turns true if the er­rno is EINTR. So this looks cor­rect. 🙁

I looked a little bit more at this file and it looks like either I do not un­der­stand the se­mantic of this lan­guage, or Get­Pro­cessStatus does re­turn the re­turn­value of the wait­pid call in­stead of the status (which is not what it shall re­turn to my un­der­stand­ing). If I am cor­rect, it can not really de­tect the status of a pro­cess. It would be very bad if such a fun­da­mental thing went un­noticed in mono…  which does not put a good light on the unit-​tests (if any) or the gen­eral test­ing of mono. For this reason I hope I am wrong.

I did not stop there, as this part does not look like it is the prob­lem. I found the fol­low­ing in mono/io-layer/processes.c:

static gboolean waitfor_​pid (gpointer test, gpointer user_​data)
{
…
    do {
        ret = wait­pid (process->id, &status, WNOHANG);
    } while (er­rno == EINTR);

    if (ret <= 0) {
        /​* Pro­cess not ready for wait */​
#if­def DEBUG
        g_​message (“%s: Pro­cess %d not ready for wait­ing for: %s”,
                   _​_​func_​_​, process->id, g_​strerror (er­rno));
#en­dif

        re­turn (FALSE);
    }

#if­def DEBUG
    g_​message (“%s: Pro­cess %d fin­ished”, _​_​func_​_​, ret);
#en­dif

    process->waited = TRUE; … } 

And here we have the prob­lem, I think. I changed the (ret <= 0) to  (ret == 0 || (ret < 0 && er­rno != ECHILD)). This will not really give the cor­rect status, but at least it should not block any­more and I should be able to see the dif­fer­ence dur­ing the build.

And now after test­ing, I see a dif­fer­ence, but the prob­lem is still there. The wait with ECHILD is gone in the loop, but there is still some loop with a sem­a­phore op­er­a­tion:

62960 mono     CALL  clock_gettime(0xd,0xbf9feef8)
62960 mono     RET   clock_​gettime 0
62960 mono     CALL  semop(0x20c0000,0xbf9feef6,0x1)
62960 mono     RET   se­mop 0
62960 mono     CALL  semop(0x20c0000,0xbf9feef6,0x1)
62960 mono     RET   se­mop 0
62960 mono     CALL  semop(0x20c0000,0xbf9feef6,0x1)
62960 mono     RET   se­mop 0
62960 mono     CALL  semop(0x20c0000,0xbf9feef6,0x1)
62960 mono     RET   se­mop 0
62960 mono     CALL  nanosleep(0xbf9fef84,0)
62960 mono     RET   nanosleep 0
62960 mono     CALL  clock_gettime(0xd,0xbf9feef8)
62960 mono     RET   clock_​gettime 0
62960 mono     CALL  semop(0x20c0000,0xbf9feef6,0x1)
62960 mono     RET   se­mop 0
62960 mono     CALL  semop(0x20c0000,0xbf9feef6,0x1)
62960 mono     RET   se­mop 0
62960 mono     CALL  semop(0x20c0000,0xbf9feef6,0x1)
62960 mono     RET   se­mop 0
62960 mono     CALL  semop(0x20c0000,0xbf9feef6,0x1)
62960 mono     RET   se­mop 0
62960 mono     CALL  nanosleep(0xbf9fef84,0)

OK, there is more go­ing on. I think someone with more know­ledge about mono should have a look at this (do not only look at this se­mop thing, but also look why it loses a child).

I do not like be­ing ill

Un­for­tu­nately you can not chose…

So, I am now on the sofa, covered a lot (a flu, I even have no voice any­more; be­fore I left work a fe­male coworker told that her hus­band would prob­ably be happy if this would hap­pen to her…  😀 ) and med­ic­a­tion and wa­ter are not far away on the table.

The good thing with the cur­rent tech­no­logy is, that you can still be a little bit pro­duct­ive (de­pend­ing on the ill­ness).

As you can read this, it means I have my net­book with me, so that I can take care about some simple things.

Video for linux (v4l) emu­la­tion com­ing to the linuxu­lator

I am in the pro­cess of pre­par­ing the im­port of code which makes v4l devices us­able in the linuxu­lator. Ba­sic­ally this means you can use your web­cam in skype (tested by the sub­mit­ter of the patch on amd64).

This is not a “ap­ply patch and com­mit” thing, be­cause the ori­ginal videodev.h (with some modi­fic­a­tions) is used. I was seek­ing the OK from core@ for this. As there is no li­cense in the header, and the ori­ginal au­thor (Alan Cox, the linux one, not our FreeBSD one) gave per­mis­sions to use it, core@ is OK with the im­port.

I in­tent to do a vendor im­port of the linux header (pre­pared today, to­gether with some readme which ex­plains where it comes from and some stuff to show that we are on the safe side re­gard­ing legal stuff), and then I want to copy this over to the linuxu­lator as linux_videodev.h and com­mit the patch (prob­ably a little bit mod­i­fied in a few places). My plan is to com­mit it this week. People which already want to play around with it now can have a look at the emu­la­tion mailing­list, a link to the patch is pos­ted there.

With the header be­ing in a vendor branch, in­ter­ested people could then start to sub­mit new BSD li­censed drivers or modify ex­ist­ing drivers which make use of the v4l in­ter­face, but I let the im­port of the header into the FreeBSD in­clude dir­ect­ory up to the per­son which wants to com­mit the first nat­ive FreeBSD-​v4l sup­port.

When such nat­ive FreeBSD-​v4l sup­port is com­mit­ted, the linuxu­lator code needs to be re­vised.