HOWTO cre­ate a new linux_base port

FreeB­SD is in need of a new linux_base port. It is on my TODO list since a long time, but I do not get the time to cre­ate one. I still do not have the time to work on a new one, but when you read this, I man­aged to get the time to cre­ate a HOWTO which describes what needs to be done to cre­ate a new linux_base port.

I will not describe how to cre­ate a new linux_base port from scratch, I will just describe how you can copy the last one and update it to some­thing new­er based upon the exist­ing infra­struc­ture for RPM packages.

Spe­cif­ic ques­tions which come up dur­ing port­ing a new Lin­ux release should be asked on freebsd-emulation@FreeBSD.org,  there are more peo­ple which can answer ques­tions than here in my blog. I will add use­ful infor­ma­tion to this HOWTO if necessary.

In the easy case most of the work is search­ing the right RPMs and their depen­den­cies to use, and to cre­ate the plist.

Why do we need a new linux_base port?

The cur­rent linux_base port is based upon Fedo­ra 10, which is end of life since Decem­ber 2009. Even Fedo­ra 13 is already end of life. Fedo­ra 16 is sup­posed to be released this year. From a sup­port point of view, Fedo­ra 15 or maybe even Fedo­ra 16 would be a good tar­get for the next linux_base port. Oth­er alter­na­tives would be to use an extend­ed life­time release of anoth­er RPM based dis­tri­b­u­tion, like for exam­ple Cen­tOS 6 (which seems to be based upon Fedo­ra 12 with back­ports from Fedo­ra 13 and 14). Using a Lin­ux release which is told to be sup­port­ed for at least 10 years, sounds nice from a FreeB­SD point of view (only minor changes to the lin­ux ports in such a case, instead of cre­at­ing a com­plete new linux_base each N+2 releas­es like with Fedo­ra), but it also means addi­tion­al work if you want to cre­ate the first linux_base port for it.

The mys­ter­ies you have to con­quer if you want to cre­ate a new linux_base port

What we do not know is, if Fedo­ra 1516, Cen­tOS 6, or any oth­er Lin­ux release will work in a sup­port­ed FreeB­SD release. There are two ways to find this out.

The first one is to take an exist­ing Lin­ux sys­tem, chroot into it (either via NFS or after mak­ing a copy into a direc­to­ry of a FreeB­SD sys­tem), and to run a lot of pro­grams (acrore­ad, skype, shells, scripts, …). The LTP test­suite is not that much use­ful here, as it will test most­ly ker­nel fea­tures, but we do not know which ker­nel fea­tures are manda­to­ry for a giv­en user­land of a Lin­ux release.

The sec­ond way of test­ing if a giv­en Lin­ux release works on FreeB­SD is to actu­al­ly cre­ate a new linux_base port for it and test it with­out chrooting.

The first way is faster, if you are only inter­est­ed in test­ing if some­thing works. The sec­ond way pro­vides an easy to set­up test­bed for FreeB­SD ker­nel devel­op­ers to fix the Lin­ux­u­la­tor so that it works with the new linux_base port. Both ways have their mer­its, but it is up to the per­son doing the work to decide which way to go.

The meat: HOWTO cre­ate a new linux_base port

First off, you need a sys­tem (or a jail) with­out any linux_base port installed. After that you can cre­ate a new linux_base port (= lbN), by just mak­ing a copy of the lat­est one (= lbO). In lbN you need to add lbO as a CONFLICT, and in all oth­er exist­ing linux_base ports, you need to add lbN as a conflict.

Change the PORTNAME, PORTVERSION, reset the PORTREVISION in lbN, and set LINUX_DIST_VER  to the new Linux-release ver­sion in the lbN Make­file (this is used in PORTSDIR/Mk/bsd.linux-rpm.mk and PORTSDIR/Mk/bsd.linux-apps.mk).

If you do not stay with Fedo­ra, there is some more work to do before you can have a look at chos­ing RPMs for instal­la­tion. You need to have a look at PORTSDIR/Mk/bsd.linux-rpm.mk and add some cas­es for the new LINUX_DIST you want to use. Do not for­get to set LINUX_DIST in the lbN Make­file to the name of the dis­tri­b­u­tion you use. You also need to aug­ment the LINUX_DIST_VER check in PORTSDIR/Mk/bsd.linux-rpm.mk with some LINUX_DIST con­di­tion­als. If you are lucky, the direc­to­ry struc­ture for down­loads is sim­i­lar to the Fedo­ra struc­ture, and there is not a lot to do here.

When this is done, you can have a look at the BIN_DISTFILES vari­able in the lbN Make­file. Try to find sim­i­lar RPMs for the new Lin­ux release you want to port. Some may not be avail­able, and it may also be the case that dif­fer­ent ones are need­ed instead. I sug­gest to first work with the ones which are avail­able (make make­sum, test install and cre­ate plist). After that you need to find out what the replace­ment RPMs for non-existing ones are. You are on your own here. Search around the net, and/or have a look at the depen­den­cies in the RPMs of lbO to deter­mine if some­thing was added as a depen­den­cy of some­thing else or not (if not, for­get about it ATM). When you man­aged to find replace­ment RPMs, you can now have a look at the depen­den­cies of the RPMs in lbN. Do not add blind­ly all depen­den­cies, not all are need­ed in FreeB­SD (the linux_base ports are not sup­posed to cre­ate an envi­ron­ment which you can chroot into, they are sup­posed to aug­ment the FreeB­SD sys­tem to be able to run Lin­ux pro­grams in ports like they where FreeB­SD native pro­grams). What you need in the linux_base ports are libraries, con­fig and data files which do not exist in FreeB­SD or have a dif­fer­ent syn­tax than in FreeB­SD (those con­fig or data files which are just in a dif­fer­ent place, can be sym­linked), and basic shell com­mands (which com­mands are need­ed or not… well… good ques­tion, in the past we made deci­sions what to include based upon prob­lem reports from users). Now for the things which are not avail­able and where not added as a depen­den­cy. Those are things which are either used dur­ing install, or where use­ful to have in the past. Find out by what it was replaced and have a look if this replace­ment can eas­i­ly be used instead. If it can be used, add it. If not, well… bad luck, we (the FreeB­SD com­mu­ni­ty) will see how to han­dle this somehow.

If you think that you have all you need in BIN_DISTFILES, please update SRC_DISTFILES accord­ing­ly and gen­er­ate the dis­t­file via  make ‑DPACKAGE_BUILDING make­sum to have the check­sums of the sources (for legal rea­sons we need them on our mirrors).

The next step is to have a look at REMOVE_DIRS, REMOVE_FILES and ADD_DIRS if some­thing needs to be mod­i­fied. Most of them are there to fall back to the cor­re­spond­ing FreeB­SD directories/files, or because they are not need­ed at all (REMOVE_*). Do not remove direc­to­ries from ADD_DIRS, they are cre­at­ed here to fix some edge con­di­tions (I do not remem­ber exact­ly why we had to add them, and I do not take the time ATM to search in the CVS history).

If you are lucky, this is all (make sure the plist is cor­rect). If you are not lucky and you need to make some mod­i­fi­ca­tions to files, have a look at the do-build tar­get in the Make­file, this is the place where some changes are done to cre­ate a nice user experience.

If you arrive here while cre­at­ing a new linux_base port, lean back and feel a bit proud. You man­aged to cre­ate a new linux_base port. It is not very well test­ed at this moment, and it is far from every­thing which needs to be done to have the com­plete Lin­ux infra­struc­ture for a giv­en Lin­ux release, but the most impor­tant part is done. Please noti­fy freebsd-emulation@FreeBSD.org and call for testers.

What is missing?

The full Lin­ux­u­la­tor infra­struc­ture for the FreeB­SD Ports Col­lec­tion has some more ports around a linux_base port. Most of the infra­struc­ture for this is han­dled in Mk/bsd.linux-apps.mk.

UPDATE: I got some time to write how to update the Linux-infrastructure ports.

Non-default lin­ux base ports deprecated

Yes­ter­day I dep­re­cat­ed the non-default Fedo­ra based Lin­ux base ports. This means fc6, f7, f8 and f9 will van­ish soon (I decid­ed for one month of expiry time). This is because all of them are End of Life upstream since a long time (= no secu­ri­ty updates).

The fc4 and f10 ones are still avail­able – even if they are End of Life too – because FreeB­SD 7.x can not use some­thing new­er than the fc4 one, and we have not test­ed yet a more recent Lin­ux distribution.

Prob­a­bly the most easy way to update the Lin­ux base ports to some­thing new­er is to stay with Fedo­ra (we have a lot of ports-infrastructure for it already). Unfor­tu­nate­ly it is not known if some­thing new­er works with­out prob­lems (miss­ing epoll/inotify sup­port could be a road­block here in case it is exten­sive­ly used in a more recent version).

I want to get some time to have a look if a more recent Fedo­ra ver­sion is suit­able for the use as a Lin­ux base in FreeB­SD 8.x+, but I do not have an esti­mate when I can start and how long it may take. In case some­one already test­ed a more recent Fedo­ra ver­sion feel free to share your experience.

A new linux-f10-pango port is ready

In the last days I took (and even had) the time to install a VM with Fedo­ra 10, updat­ed all the pack­ages after instal­la­tion, and cre­at­ed a new linux-f10-pango port (v 1.28.3). I did this because the port has a secu­ri­ty vul­ner­a­bil­i­ty accord­ing to our VuXML DB and there where more and more reports in the last months from users which had a prob­lem with this.

Dur­ing the update of the port I noticed that the port does not con­tain a FORBIDDEN entry, just por­tau­dit com­plains about it because there is an entry in the VuXML. That is not nice. I was told that the ports slush will be lift­ed soon (I need to bump some PORTRE­VI­SIONs), this means that I can com­mit the update prob­a­bly tomor­row, just in time when the new RPM should hit the FreeB­SD dis­tri­b­u­tion infra­struc­ture (MASTER_SITE_LOCAL is updat­ed once a day from a spe­cif­ic fold­er in our home directories).

Thanks to Luchesar V. ILIEV for the nice write­up of what to install in Fedo­ra 10 to be able to build RPMs, and the descrip­tion of how to build your own RPM.

Some updates (kernel/ports)

There was not much to tell in the last months. I was busy with mov­ing and the preg­nan­cy of my wife (ok, she was more busy with this than I was…).

So the recent updates are, that I took some time to com­mit some of my patch­es to SVN. Most of them are in my SVN user area in var­i­ous branch­es. The inter­est­ing ones may be desk­jail and lin­ux­aio. The first one allows to run your desk­top in a jail. The sec­ond one gives async I/O for the linuxulator.

There’s also some oth­er stuff. Feel free to have a look.

It also seems that the we may see the Fedo­ra 8 infra­struc­ture land­ing in the ports col­lec­tion “soon”. I have the impres­sion that Boris just waits for the com­plete unfreeze of the ports col­lec­tion. The last patch I’ve reviewed looked very good. There are some loose ends, like switch­ing it on as the default lin­ux base for FreeBSD-current for exam­ple, but those are things which I pre­fer to do lat­er than in the same com­mit. First let it be there for a while and let curi­ous users test it a lit­tle bit more. If every­thing is ok, we can switch the default lin­ux base to F8 in ‑cur­rent.

Progress in the linuxulator

On of the major show­stop­per bugs in the lin­ux 2.6 emu­la­tion is that acrore­ad does not work. Now we have patch­es (proof of con­cept by Intron, refined patch by Kib) for it. I did­n’t had time to test it yet (mind you, every­one else is not able to run acrore­ad with 2.6, I’m able to run it at least with some files or no file at all), but I want to do an exten­sive test (I know sev­er­al ways of killing it with 2.6).

If every­thing goes well and no oth­er show­stop­per bug appears, we may be able to request more exten­sive test­ing of the 2.6 emu­la­tion, at least on i386. First this should be done by ask­ing peo­ple to switch, and maybe after a week by switch­ing the default emu­la­tion to 2.6 in ‑cur­rent (at least for a while).

This is spe­cial­ly impor­tant as the Fedo­ra Lega­cy project announced that they will aban­don sup­port for FC4. FC5+ is not able to run on a 2.4 kernel.

And while I’m at it: I sub­mit­ted the sta­tus report for the lin­ux­u­la­tor. It con­tains some nice sta­tis­tic about the num­ber of fixed bugs (com­par­ing 6.2 and ‑cur­rent). No, I will not tell you in advance, you have to wait some days until the report shows up. 😛