For­cing a route in Sol­ar­is?

I have a little prob­lem find­ing a clean solu­tion to the fol­low­ing prob­lem.

A ma­chine with two net­work in­ter­faces and no de­fault route. The first in­ter­face gets an IP at boot time and the cor­res­pond­ing stat­ic route is in­ser­ted dur­ing boot in­to the rout­ing table without prob­lems. The second in­ter­face only gets an IP ad­dress when the shared-​IP zones on the ma­chine are star­ted, dur­ing boot the in­ter­face is plumbed but without any ad­dress. The net­works on those in­ter­faces are not con­nec­ted and the ma­chine is not a gate­way (this means we have a machine-​administration net­work and a production-​network). The stat­ic routes we want to have for the ad­dresses of the zones are not ad­ded to the rout­ing table, be­cause the next hop is not reach­able at the time the routing-​setup is done. As soon as the zones are up (and the in­ter­face gets an IP), a re-​run of the routing-​setup adds the miss­ing stat­ic routes.

Un­for­tu­nately I can not tell Sol­ar­is to keep the stat­ic route even if the next hop is not reach­able ATM (at least I have not found an op­tion to the route com­mand which does this).

One solu­tion to this prob­lem would be to add an ad­dress at boot to the in­ter­face which does not have an ad­dress at boot-​time ATM (prob­ably with the de­prec­ated flag set). The prob­lem is, that this sub­net (/​28) has not enough free ad­dresses any­more, so this is not an op­tion.

An­oth­er solu­tion is to use a script which re-​runs the routing-​setup after the zones are star­ted. This is a prag­mat­ic solu­tion, but not a clean solu­tion.

As I un­der­stand the in.routed man-​page in.routed is not an op­tion with the de­fault con­fig, be­cause the ma­chine shall not route between the net­works, and shall not change the rout­ing based upon RIP mes­sages from oth­er ma­chines. Un­for­tu­nately I do not know enough about it to be sure, and I do not get the time to play around with this. I have seen some in­ter­st­ing op­tions re­gard­ing this in the man-​page, but play­ing around with this and sniff­ing the net­work to see what hap­pens, is not an op­tion ATM. Any­one with a config/​tutorial for this “do not broad­cast any­thing, do not ac­cept any­thing from outside”-case (if pos­sible)?

Sony BRAVIA TV & DLNA formats

As I wrote earli­er, I try to get some in­fos which formats my Sony BRAVIA 5800 TV is able to play over the net­work. Sony is not really help­ful (they tell only names someone with a DLNA spec could cor­rectly in­ter­pret). Now I took the time to move my TV in­to a dif­fer­ent sub­net (the same where my NAS is in, not like be­fore in a DMZ), and I in­stalled min­idlna. After some net­work sniff­ing, the use of the In­tel UP­nP Device Spy and some min­idlna–source read­ing I have now a bet­ter idea what my Sony TV ex­pects.

The DLNA-​specification seems to man­date a MIME-​type and some DLNA-​specific iden­ti­fi­er which de­scribes the con­tent a play­er (a DLNA-​Renderer) is able to dis­play. In the fol­low­ing I will present the MIME-​type, the DLNA-​identifier, and prob­ably a Sony-​specific iden­ti­fi­er.

Re­gard­ing pic­tures the TV only ac­cepts JPEGs, bit in small, me­di­um and large sizes. I did not both­er to look up what this means in real val­ues, so far this is not of high in­terest for me. For au­dio the TV ac­cepts MP3s and LPCM (raw PCM samples). The raw sniffed data from the TV looks like this:

image/jpeg:DLNA.ORG_PN=JPEG_SM
image/jpeg:DLNA.ORG_PN=JPEG_MED
image/jpeg:DLNA.ORG_PN=JPEG_LRG
audio/mpeg:DLNA.ORG_PN=MP3
audio/L16:DLNA.ORG_PN=LPCM

The more in­ter­est­ing part for me is the video part. The TV sup­ports MPEG2 Video (the MPEG_​ part in the DLNA.ORG_PN) and H.264 (the AVC_​ part in the DLNA.ORG_PN). For MPEG2 it sup­ports pro­gram streams (PS in DLNA.ORG_PN) and trans­port streams (TS in DLNA.ORG_PN). For PS it sup­ports PAL and NTSC res­ol­u­tions (720×576 is PAL, HD res­ol­u­tions like 720p or 1080i or 1080p are not sup­por­ted). The packet-​length of a trans­port steam can be 188 bytes or 192 bytes. If the width is >= 1288 or the height is >= 720, min­idlna adds HD in DLNA.ORG_PN, else it will add SD. The EU in DLNA.ORG_PN is for SD video with a height of 576 or 288 pixels. De­pend­ing of the com­bin­a­tion of the packet-​length and if there is a timestamp in use or not, the DLNA.ORG_PN will have a _​ISO or a _​T ap­pen­ded.

It also sup­ports H.264. The DLNA.ORG_PN starts with a AVC in this case. Only trans­port streams (TS  in DLNA.ORG_PN) is sup­por­ted. As with MPEG2, the packet-​length of the TS can be 188 or 192 bytes. De­pend­ing of the com­bin­a­tion of the packet-​length and if there is a timestamp in use or not, the DLNA.ORG_PN will have a _​ISO or a _​T ap­pen­ded. De­pend­ing on the pro­file used, min­idlna adds some more in­fos to the DLNA.ORG_PN, BL if it is a baseline-​profile, MP if it is a main-​profile, and HP if it is a high-​profile. I do not see this in the val­id video formats my TV re­ques­ted over the wire. As with the MPEG2 format, SD or HD is ad­ded (in min­idlna) de­pend­ing on the width and height, but also on the bitrate of the video. For the main-​profile the width has to be <= 720, the height <= 576 and the bitrate <= 10M (base 10, not base 2) for SD, and the width has to be <=1920, the height <= 1152 and the bitrate <= 20M (base 10, not base 2) for HD. For the high-​profile the width has to be <=1920, the height <=1152, the bitrate <= 30M (base 10, not base 2) and the au­dio has to be AC3 to get the HD ad­ded in DLNA.ORG_PN. The au­dio is spe­cified in DLNA.ORG_PN as MPEG1_​L3 for MP3, AC3 for AC3, and AAC or AAC_​MULT5 for AAC (ste­reo or 5-​channel). As can be seen be­low, the TV seems only to sup­port AC3 au­dio for AVC. The TV also has _​24_​, _​50_​ and _​60_​ in DLNA.ORG_PN. I did not find those things in the min­idlna source (but I have not really searched for this). I could ima­gine that _​24_​ stands for 24 pic­tures per second, and the _​50_​ and _​60_​ for pro­gress­ive videos (with 50 re­spect­ively 60 pic­tures per second), but this is pure spec­u­la­tion from my side. Here is the raw sniffed data:

video/mpeg:DLNA.ORG_PN=AVC_TS_HD_24_AC3_ISO;SONY.COM_PN=AVC_TS_HD_24_AC3_ISO
video/vnd.dlna.mpeg-tts:DLNA.ORG_PN=AVC_TS_HD_24_AC3;SONY.COM_PN=AVC_TS_HD_24_AC3
video/vnd.dlna.mpeg-tts:DLNA.ORG_PN=AVC_TS_HD_24_AC3_T;SONY.COM_PN=AVC_TS_HD_24_AC3_T

video/vnd.dlna.mpeg-tts:DLNA.ORG_PN=MPEG_PS_PAL
video/vnd.dlna.mpeg-tts:DLNA.ORG_PN=MPEG_PS_NTSC

video/vnd.dlna.mpeg-tts:DLNA.ORG_PN=MPEG_TS_SD_50_L2_T
video/vnd.dlna.mpeg-tts:DLNA.ORG_PN=MPEG_TS_SD_60_L2_T
video/mpeg:DLNA.ORG_PN=MPEG_TS_SD_EU_ISO
video/vnd.dlna.mpeg-tts:DLNA.ORG_PN=MPEG_TS_SD_EU
video/vnd.dlna.mpeg-tts:DLNA.ORG_PN=MPEG_TS_SD_EU_T
video/vnd.dlna.mpeg-tts:DLNA.ORG_PN=MPEG_TS_SD_50_AC3_T
video/vnd.dlna.mpeg-tts:DLNA.ORG_PN=MPEG_TS_SD_60_AC3_T
video/mpeg:DLNA.ORG_PN=MPEG_TS_HD_50_L2_ISO;SONY.COM_PN=HD2_50_ISO
video/mpeg:DLNA.ORG_PN=MPEG_TS_HD_60_L2_ISO;SONY.COM_PN=HD2_60_ISO
video/vnd.dlna.mpeg-tts:DLNA.ORG_PN=MPEG_TS_HD_50_L2_T;SONY.COM_PN=HD2_50_T
video/vnd.dlna.mpeg-tts:DLNA.ORG_PN=MPEG_TS_HD_60_L2_T;SONY.COM_PN=HD2_60_T

video/mpeg:DLNA.ORG_PN=AVC_TS_HD_50_AC3_ISO;SONY.COM_PN=AVC_TS_HD_50_AC3_ISO
video/mpeg:DLNA.ORG_PN=AVC_TS_HD_60_AC3_ISO;SONY.COM_PN=AVC_TS_HD_60_AC3_ISO
video/vnd.dlna.mpeg-tts:DLNA.ORG_PN=AVC_TS_HD_50_AC3;SONY.COM_PN=AVC_TS_HD_50_AC3
video/vnd.dlna.mpeg-tts:DLNA.ORG_PN=AVC_TS_HD_60_AC3;SONY.COM_PN=AVC_TS_HD_60_AC3
video/vnd.dlna.mpeg-tts:DLNA.ORG_PN=AVC_TS_HD_50_AC3_T;SONY.COM_PN=AVC_TS_HD_50_AC3_T
video/vnd.dlna.mpeg-tts:DLNA.ORG_PN=AVC_TS_HD_60_AC3_T;SONY.COM_PN=AVC_TS_HD_60_AC3_T

video/​x-​mp2t-​mphl-​188

So far I did not get the time to ex­per­i­ment with this. I also have the im­pres­sion that min­idlna has still some rough edges (the sin­tel video I used to test be­fore with a dif­fer­ent me­dia serv­er, does not show up in the list with min­idlna).

Linuxu­lat­or ex­plained (for de­velopers): adding ioctls dir­ectly to the ker­nel

After giv­ing an over­view of the in-​kernel ba­sics of the Linuxu­lat­or, I want now to de­scribe how to add sup­port for new ioctls to the Linuxu­lat­or.

Where are the files to modi­fy?

The plat­form in­de­pend­ent code for the ioctls is in SRC/​sys/​compat/​linux/linux_ioctl.c. The defines to have names for the ioctl val­ues are in SRC/​sys/​compat/​linux/linux_ioctl.h.

How to modi­fy them?

First of all cre­ate a new head­er which will con­tain all the struc­tures, named val­ues and mac­ros for those new ioctls. As writ­ten above, the ioctl val­ues (e.g. #define LINUX_​VIDIOC_​ENCODER_​CMD 0x564d /​* 0xc028564d */​) do not be­long there, they shall be ad­ded to linux_ioctl.h. Dur­ing the course of adding sup­port for ioctls, you will need this new head­er. Add it in the SRC/​sys/​compat/​linux/​ dir­ect­ory, and pre­fix the name with a linux_​. It would be good to de­cide on a com­mon tag here (ref­er­enced as your­tag in the fol­low­ing), and stay with it. Use it wherever you need to have some spe­cif­ic name for the ioctl-​set you want to add. In this case it would res­ult in linux_​your­tag.h (or even linux_​ioctl_​your­tag.h, de­pend­ing if this is used for some­thing very spe­cif­ic to the ioctls, or some gen­er­ic linux fea­ture) as the name of the head­er file. This was not done in the past, so do not ex­pect that the names in­side the linux_ioctl.c file will be con­sist­ent to this nam­ing scheme, but it is nev­er too late to cor­rect mis­takes of the past (at least in Open Source soft­ware de­vel­op­ment).

Now add this head­er to linux_ioctl.c (you need to in­clude compat/linux/linux_yourtag.h). After that add the ioctl val­ues to linux_ioctl.h. As can be seen above, the defines should be named the same as on linux, but with a LINUX_​ pre­fix (make sure they where not defined be­fore some­where else). The ioctl val­ues need to be the same hex val­ues as in Linux, off course. Sort them ac­cord­ing to their hex value. When you ad­ded all, you need to add two more defines. The LINUX_​IOCTL_​your­tag_​MIN and LINUX_​IOCTL_​your­tag_​MAX ones. The MIN-​one needs to be an ali­as for the first (sor­ted ac­cord­ing to the hex value) ioctl you ad­ded, and MAX needs to be an ali­as for the last (again, sor­ted ac­cord­ing to the hex value) ioctl you ad­ded.

The next step is to let the Linuxu­lat­or know that it is able to handle the ioctls in the LINUX_​IOCTL_​your­tag_​MIN to LINUX_​IOCTL_​your­tag_​MAX range. Search the stat­ic linux_​ioctl_​function_​t sec­tion of linux_ioctl.c and add such a vari­able for your ioctl set. The name of the vari­able should be some­thing like linux_​ioctl_​your­tag.

Sim­il­ar for the handler-​definition for this. Search the stat­ic struct linux_​ioctl_​handler sec­tion and add a yourtag_​handler. Set it to { linux_​ioctl_​your­tag, LINUX_​IOCTL_​your­tag_​MIN, LINUX_​IOCTL_​your­tag_​MAX }. To make this hand­ler known to the Linuxu­lat­or, you need to add it to the DATA_​SET sec­tion. Add DATA_SET(linux_ioctl_handler_set, your­tag_​handler) there.

Now the meat, the func­tion which handles the ioctls. You already defined it as linux_​ioctl_​function_​t, but now you need to write it. The out­line of it looks like this:

stat­ic int
linux_​ioctl_​your­tag(struct thread *td, struct linux_​ioctl_​args *args)
{
        struct file *fp;
        int er­ror;
        switch (args->cmd & 0xffff) {
        case LINUX_​an_​easy_​ioctl:
                break;
        case LINUX_​a_​not_​so_​easy_​ioctl:
                /​* your hand­ling of the ioctl */
                fdrop(fp, td);
                re­turn (er­ror);
        /​* some more hand­ling of your ioctls */
        de­fault:
       re­turn (ENOIOCTL);
        }
        er­ror = ioctl(td, (struct ioctl_​args *)args);
        re­turn (er­ror);
}

An easy ioctl in the switch above is an ioctl where you do not have to do some­thing but can pass the ioctl through to FreeBSD it­self. The not so easy ioctl case is an ioctl where you need to do e.g. a fget(td, args->fd, &fp). This is just an ex­ample, there are also oth­er pos­sib­il­it­ies where you need to do ad­di­tion­al stuff be­fore the re­turn, or where you do not pass the ioctl to FreeBSD. A typ­ic­al ex­ample of what needs to be done here is to copy val­ues from linux struc­tures to FreeBSD struc­tures (and the oth­er way too), or to trans­late between 64bit and 32bit. Linux pro­grams on amd64 are 32bit ex­ecut­ables and 32bit structures/​pointers. To make this work on amd64, you need to find a way to map between the two. There are ex­amples in the ker­nel where this is already the case. The more prom­in­ent ex­amples in the 64bit<->32bit re­gard are the v4l and v4l2 ioctls.

The te­di­ous part is to re­search if a trans­la­tion has to be done and if yes what needs to be trans­lated how. When this is done, most of the work is not so hard. The linux_yourtag.h should con­tain the struc­tures you need for this trans­la­tion work.

It is also pos­sible to add ioctls in a ker­nel mod­ule, but this is not sub­ject to this de­scrip­tion (I will up­date this post­ing with a link to it when I get time to write about it).

(Free)BSD in­side An­droid

Today I was look­ing in­to the Open­Source li­censes which are dis­played for An­droid (2.3.4). There are sev­er­al files which come with a BSD li­cense.

Dur­ing look­ing at it, I no­ticed that the libm has the copy­right of sev­er­al FreeBSD people. I did not had an in-​deep look if this is be­cause they took the FreeBSD libm, or if this is be­cause parts of the FreeBSD libm where ad­op­ted by oth­er BSD pro­jects.

What I no­ticed is, that some spe­cial char­ac­ters are not dis­played cor­rectly. For ex­ample the name Dag-​Erling Smør­grav looks mangled in the dis­play of the li­cense in­side the phone (I hope it is dis­played bet­ter in my blog). His name is not the only prob­lem case, there are also oth­er char­ac­ters which are not rendered as ex­pec­ted.

This does not really look pro­fes­sion­al.

HOWTO add linux-​infrastructure ports for a new linux_​base port

In my last blog-​post I de­scribed how to cre­ate a new linux_​base port. This blog-​post is about the oth­er Linux-ports which make up the Linux-in­fra­struc­ture in the FreeBSD Ports Col­lec­tion for a giv­en Linux-​release.

What are linux-​infrastructure ports?

A linux_​base port con­tains as much as pos­sible and at the same time as little as pos­sible to make up a use­ful Linux-​compatibility-​experience in FreeBSD. I know, this is not a de­script­ive ex­plan­a­tion. And it is not on pur­pose. There are no fixed rules what has to be in­side or what not. It “ma­tured” in­to the cur­rent shape. A prac­tic­al ex­ample is, that there is no GUI–stuff in the linux_​base. While you need the GUI parts like GTK or QT for soft­ware like Skype and acror­ead, you do not need them for head­less game serv­ers. While you may need vari­ous lib­rar­ies for game serv­ers, you may not need those for Skype or acror­ead. As such some stand­ard parts are in sep­ar­ate ports which are named linux–LINUX_​DIST_​SUFFIX-NAME. For GTK and the Fe­dora 10 re­lease this res­ults in linux-​f10-​gtk2. Such gen­er­ic ports which de­pend upon a spe­cif­ic Linux-​release make up the Linux-​infrastructure in the FreeBSD Ports Col­lec­tion. Those ports are ref­er­enced in port-​Makefiles via the USE_​LINUX_​APPS vari­able, e.g. USE_LINUX_APPS=gtk2.

If you cre­ated a new linux_​base port, you need most stand­ard in­fra­struc­ture ports in a ver­sion for the Linux-​release used in the linux_​base port, to have the Linux-​application ports in the FreeBSD Ports Col­lec­tion work­ing (if you are un­lucky, some ports do not play well with the Linux-​release you have chosen, but this is out of the scope of this HOWTO).

Up­dat­ing Mk/bsd.linux-apps.mk

 First we need to set the LINUX_​DIST_​SUFFIX vari­able to a value suit­able to the new Linux-​release. This is done in the con­di­tion­al which checks the OVERRIDE_​LINUX_​NONBASE_​PORTS vari­able for val­id val­ues. Add an ap­pro­pri­ate con­di­tion­al, and do not for­get to add the new val­id value to the IGNORE line in the last else branch of the con­di­tion­al.

The next step is to check the _​LINUX_​APPS_​ALL and _​LINUX_​26_​APPS vari­ables. If there are some in­fra­struc­ture ports which are not avail­able for the new Linux-​release, the con­di­tion­al which checks the avail­ab­il­ity of a giv­en in­fra­struc­ture port for a giv­en Linux-​release needs to be mod­i­fied. If at a later step you no­tice that there are some ad­di­tion­al in­fra­struc­ture ports ne­ces­sary for the new Linux-​release, _​LINUX_​APPS_​ALL and the check-​logic needs to be mod­i­fied too (e.g. add a new vari­able for your Linux-​release, add the con­tent of the vari­able to _​LINUX_​APPS_​ALL, and change the check to do the right thing).

After that two te­di­ous parts need to be done.

For each in­fra­struc­ture port there is a set of vari­ables. The name_​PORT vari­able con­tains the loc­a­tion of the port in the Ports Col­lec­tion. Typ­ic­ally you do not have to change it (if you really want to change it, do not do it, fix the nam­ing of the in­fra­struc­ture port in­stead), be­cause we use a nam­ing con­ven­tion here which in­cludes the LINUX_​DIST_​SUFFIX. The name_​DETECT vari­able is an in­tern­al vari­able, do not change it (if you cre­ate a new in­fra­struc­ture port, copy it from some­where else and make sure the name in value of the vari­able matches the port name in the name of the vari­able). Then there are sev­er­al name_​suf­fix_​FILE vari­ables. Leave the ex­ist­ing ones alone, and add a new one with the cor­rect suf­fix for your new Linux-​release. The value of the vari­able needs to be an im­port­ant file which is in­stalled by the in­fra­struc­ture port in ques­tion. FYI: The con­tent of the name_​suf­fix_​FILE vari­ables are used to set the name_​DETECT vari­ables, de­pend­ing on the Linux-​relase the name_​DETECT vari­ables are used to check if the port is already in­stalled. Ideally the name_​suf­fix_​FILE vari­able points to a lib­rary in the port. The name_​DEPENDS vari­able lists de­pend­en­cies of this in­fra­struc­ture port. If the de­pend­en­cies changed in your Linux-​release, you need to add a con­di­tion­al to change the de­pend­ency if LINUX_​DIST_​SUFFIX is set to your Linux-​release.

Nor­mally this is all what needs to be done in PORTSDIR/Mk/bsd.linux-apps.mk, the rest of the file is code to check de­pend­en­cies and some cor­rect­ness checks.

The second te­di­ous part is to ac­tu­ally cre­ate all those in­fra­struc­ture ports. Nor­mally you can copy an ex­ist­ing in­fra­struc­ture port, re­name it, ad­just the PORTNAME, PORTVERSION, PORTREVISION, MASTER_​SITES, PKGNAMEPREFIX, DISTFILES, CONFLICTS (also in all oth­er Linux-​release ver­sions of this in­fra­struc­ture port), LINUX_​DIST_​VER, RPMVERSION (if set/​neccesary) and SRC_​DISTFILE vari­ables, gen­er­ate the dist­file check­sums (make make­sum), and fix the plist. I sug­gest to script parts of this work (as of this writ­ing Fresh­ports counts 68 ports where the port­name starts with linux-​f10-).

Adding new in­fra­struc­ture ports, or re­mov­ing in­fra­struc­ture ports for a giv­en Linux-​release

If your Linux-​release does not come with a pack­age for an ex­ist­ing in­fra­struc­ture port, just do not cre­ate a cor­res­pond­ing name_​suf­fix_​FILE line. You still need to do the right thing re­gard­ing de­pend­en­cies of ports which de­pend upon this non-​existing in­fra­struc­ture port (if your Linux-​release comes with pack­ages for them).

To add a new in­fra­struc­ture port, copy an ex­ist­ing block, re­name the vari­ables, set them cor­rectly, add a new vari­able for your Linux-​release in the first _​LINUX_​APPS_​ALL sec­tion, add the con­tent of this vari­able to _​LINUX_​APPS_​ALL, and change the check-​logic as de­scribed above.

Fi­nal words

If you have some­thing which in­stalls and dein­stalls cor­rectly, feel free to provide it on freebsd-​emulation@​FreeBSD.​org for re­view/​testing. If you have ques­tions dur­ing the port­ing, feel also free to send a mail there.