Book review: FreeB­SD Device Drivers

In mid-April a woman from the mar­ket­ing depart­ment of No Starch Press con­tact­ed me and asked if I am inter­est­ed to do a pub­lic review of the FreeB­SD Device Dri­vers book by Joseph Kong (no link to a book shop, go and have a look in your pre­ferred one). Just this sim­ple ques­tion, no strings attached.

I had my nose in some device dri­vers in the past, but I nev­er wrote one, and nev­er had a look at the big pic­ture. I was inter­est­ed to know how every­thing fits togeth­er, so this made me a good vic­tim for a review (novice enough to learn some­thing new and to have a look if enough is explained, and expe­ri­enced enough to under­stand what is going on in the FreeB­SD kernel).

Some min­utes after I agreed to review it (but with a lit­tle notice that I do not know how long I need to review it), I had the PDF ver­sion of the book. That was faster than I expect­ed (maybe I am too old-school and used to have paper ver­sions of books in my hands).

Let the review begin… but bear with me, this is the first time I do a real pub­lic review of a book (instead of a tech­ni­cal review for an author). And as this is my very own per­son­al opin­ion, I will not allow com­ments here. This page is all about my opin­ion while read­ing the book, ques­tions I have while read­ing the book shall serve as a hint about the qual­i­ty of the book and they should be answered in the book, not here.

In short, the book is not per­fect, but it is a good book. There is room for improve­ment, but on a very high lev­el. If you want to write a device dri­ver for FreeB­SD, this book is a must. I sug­gest to read it com­plete­ly, even chap­ters which do not belong to the type of dri­ver you want to write (spe­cial­ly the case stud­ies of real dri­vers). The rea­son is that each chap­ter has some notes which may not only apply to the chap­ter in ques­tion, but to all kinds of device dri­vers. The long review fol­lows now.

The first chap­ter is titled “Build­ing and run­ning mod­ules”. The author begins with descrip­tion of the usu­al device dri­ver types (NIC dri­ver, pseudo-device, …) and how they can be added to the ker­nel (sta­t­i­cal­ly linked in or as a mod­ule). The first code exam­ple is a small and easy ker­nel mod­ule, so that we do not have to reboot the sys­tem we use to devel­op a dri­ver (except we make a fault dur­ing dri­ver devel­op­ment which caus­es the machine to pan­ic or hang). Every part of the exam­ple is well explained. This is fol­lowed by an overview about char­ac­ter devices (e.g. disks) and a sim­ple character-device dri­ver (so far a pseudo-device, as we do not have real hard­ware we access) which is not only as-well explained as the module-example, but there is also a note where the code was sim­pli­fied and what should be done instead.

After read­ing this chap­ter you should be able to write your own ker­nel mod­ule in 5 min­utes (well, after 5 min­utes it will not be able to do a lot – just a “hel­lo world” – but at least you can already load/unload/execute some code into/from/in the kernel).

I have not tried any exam­ple myself, but I com­piled a lot of mod­ules and dri­vers I mod­i­fied in the past and remem­ber to have seen the described parts.

The sec­ond chap­ter explains how to allo­cate and free mem­o­ry in the ker­nel. There is the pos­si­bil­i­ty to allo­cate maybe-contiguous mem­o­ry (the nor­mal case, when your hard­ware does not do DMA or does not have the require­ment that the mem­o­ry region it makes DMA from/too needs to be con­tigu­ous), and real­ly con­tigu­ous. For the size argu­ment of the free­ing of the the con­tigu­ous mem­o­ry there is the sen­tence “Gen­er­al­ly, size should be equal the amount allo­cat­ed.”. Imme­di­ate­ly I want­ed to know what hap­pens if you spec­i­fy a dif­fer­ent size (as a non-native eng­lish speak­er I under­stand this sen­tence in a way that I am allowed to spec­i­fy a dif­fer­ent size and as such are able to free only parts of the allo­cat­ed mem­o­ry). Unfor­tu­nate­ly this is not answered. I had a look into the source, the ker­nel frees mem­o­ry pages, so the size argu­ment (and addr argu­ment) will be round­ed to include a full page. This means the­o­ret­i­cal­ly I am able to free parts of the allo­cat­ed mem­o­ry, but this is a source-maintenance night­mare (needs knowl­edge about the machine spe­cif­ic page bound­aries and you need to make sure that you do the absolute­ly cor­rect size cal­cu­la­tions).  To me this looks more like as long as nobody is point­ing a gun at my head and tells me to use a dif­fer­ent size, spec­i­fy­ing the same size as made dur­ing the allo­ca­tion of this mem­o­ry region is the way to go.

After read­ing this chap­ter you should know how to kill the sys­tem by allo­cat­ing all the RAM in the kernel.

Again, I did not try to com­pile the exam­ples in this chap­ter, but the dif­fer­ence of the mem­o­ry allo­ca­tion in the ker­nel com­pared with mem­o­ry allo­ca­tion in the user­land is not that big.

The third chap­ter explains the device com­mu­ni­ca­tion and con­trol inter­faces (ioctl/sysctl) of a dri­ver. The ioctl part teached me some parts I always want­ed to know when I touched some ioctls, but nev­er both­ered to find out before. Unfor­tu­nate­ly this makes me a lit­tle bit ner­vous about the way ioctls are han­dled in the FreeB­SD lin­ux­u­la­tor, but this is not urgent ATM (and can prob­a­bly be han­dled by a com­mend in the right place). The sysctl part takes a lit­tle bit longer to fol­low through, but there is also more to learn about it. If you just mod­i­fy an exist­ing dri­ver with an exist­ing sysctl inter­face, it prob­a­bly just comes down to copy&paste with lit­tle mod­i­fi­ca­tions, but if you need to make more com­plex changes or want to add a sysctl inter­face to a dri­ver, this part of the book is a good way to under­stand what is pos­si­ble and how every­thing fits togeth­er. Per­son­al­ly I would have wished for a more detailed guide when to pick the ioctl inter­face and when the sysctl inter­face than what was writ­ten in the con­clu­sion of the chap­ter, but it is prob­a­bly not that easy to come up with a good list which fits most drivers.

After read­ing this chap­ter you should be able to get data in and out of the ker­nel in 10 minutes.

As before, I did not com­pile the exam­ples in this chap­ter. I already added ioctls and sysctls in var­i­ous places in the FreeB­SD kernel.

Chap­ter 4 is about thread syn­chro­niza­tion – mutex­es, shared/exclusive locks, reader/writer locks and con­di­tion vari­ables. For me this chap­ter is not as good as the pre­vi­ous ones. While I got a good expla­na­tion of every­thing, I missed a nice overview table which com­pares the var­i­ous meth­ods of thread syn­chro­niza­tion. Bren­dan Gregg did a nice table to give an overview of DTrace vari­able types and when to use them. Some­thing like this would have been nice in this chap­ter too. Apart from this I got all the info I need (but hey, I already wrote a NFS client for an exper­i­men­tal com­put­er with more than 200000 CPUs in 1998, so I’m famil­iar with such syn­chro­niza­tion primitives).

Delayed exe­cu­tion is explained in chap­ter 5. Most of the infor­ma­tion pre­sent­ed there was new to me. While there where not much exam­ples pre­sent­ed (there will be some in a lat­er chap­ter), I got a good overview about what exists. This time there was even an overview when to use which type of delayed exe­cu­tion infra­struc­ture. I would have pre­ferred to have this overview in the begin­ning of the chap­ter, but that is maybe some kind of per­son­al preference.

In chap­ter 6 a com­plete device dri­ver is dis­sect­ed. It is the vir­tu­al null modem ter­mi­nal dri­ver. The chap­ter pro­vides real-world exam­ples of event-handlers, call­outs and taskqueues which where not demon­strat­ed in chap­ter five. At the same time the chap­ter serves as a descrip­tion of the func­tions a TTY dri­ver needs to have.

Auto­mat­ed device detec­tion with New­bus and the cor­re­spond­ing resource allo­ca­tion (I/O ports, device mem­o­ry and inter­rupts) are explained in chap­ter 7. It is easy… if you have a real device to play with. Unfor­tu­nate­ly the chap­ter missed a para­graph or two about the sus­pend and resume meth­ods. If you think about it, it is not hard to come up with what they are sup­posed to do, but a lit­tle explic­it descrip­tion of what they shall do, in what state the hard­ware should be put and what to assume when being called would have been nice.

Chap­ter 8 is about inter­rupts. It is easy to add an inter­rupt han­dler (or to remove one), the hard part is to gen­er­ate an inter­rupt. The exam­ple code uses the par­al­lel port, and the chap­ter also con­tains a lit­tle expla­na­tion how to gen­er­ate an inter­rupt… if you are not afraid to touch real hard­ware (the par­al­lel port) with a resistor.

In chap­ter 9 the lpt(4) dri­ver is explained, as most of the top­ics dis­cussed so far are used inside. The expla­na­tion how every­thing is used is good, but what I miss some­times is why they are used. The most promi­nent (and only) exam­ple here for me is why are call­outs used to catch stray inter­rupts? That call­outs are a good way of han­dling this is clear to me, the big ques­tion is why can there be stray inter­rupts. Can this hap­pen only for the par­al­lel port (respec­tive­ly a lim­it­ed amount of devices), or does every dri­ver for real inter­rupt dri­ven hard­ware need to come with some­thing like this? I assume this is some­thing spe­cif­ic to the device, but a lit­tle expla­na­tion regard­ing this would have been nice.

Access­ing I/O ports and I/O mem­o­ry for devices are explained in chap­ter 10 based upon a dri­ver for a LED device (turn on and off 2 LEDs on an ISA bus). All the func­tions to read and write data are well explained, just the part about the mem­o­ry bar­ri­er is a lit­tle bit short. It is not clear why the CPU reorder­ing of mem­o­ry access­es mat­ter to what looks like func­tion calls. Those func­tion calls may be macros, but this is not explained in the text. Some lit­tle exam­ples when to use the bar­ri­ers instead of an abstract descrip­tion would also have been nice at this point.

Chap­ter 11 is sim­i­lar to chap­ter 10, just that a PCI bus dri­ver is dis­cussed instead of an ISA bus dri­ver. The dif­fer­ences are not that big, but important.

In chap­ter 12 it is explained how to do DMA in a dri­ver. This part is not easy to under­stand. I would have want­ed to have more exam­ples and expla­na­tions of the DMA tag and DMA map parts. I am also sur­prised to see dif­fer­ent sup­port­ed archi­tec­tures for the flags BUS_DMA_COHERENT and BUS_DMA_NOCACHE for dif­fer­ent func­tions. Either this means FreeB­SD is not coher­ent in those parts, or it is a bug in the book, or it is sup­posed to be like this and the rea­sons are not explained in the book. As there is no explic­it note about this, it prob­a­bly leads to con­fu­sion of read­ers which pay enough atten­tion here. It would also have been nice to have an expla­na­tion when to use those flags which are only imple­ment­ed on a sub­set of the archi­tec­tures FreeB­SD sup­ports. Any­way, the expla­na­tions give enough infor­ma­tion to under­stand what is going on and to be able to have a look at oth­er device dri­vers for real-live exam­ples and to get a deep­er under­stand­ing of this topic.

Disk dri­vers and block I/O (bio) requests are described in chap­ter 13. With this chap­ter I have a lit­tle prob­lem. The author used the word “unde­fined” in sev­er­al places where I as a non-native speak­er would have used “not set” or “set to 0”. The word “unde­fined” implies for me that there may be garbage inside, where­as from a tech­ni­cal point of view I can not imag­ine that some ran­dom val­ue in those places would have the desired result. In my opin­ion each such place is obvi­ous, so I do not expect that an expe­ri­enced pro­gram­mer would lose time/hairs/sanity over it, but inex­pe­ri­enced pro­gram­mers which try to assem­ble the cor­re­spond­ing struc­tures on the (unini­tial­ized) heap (for what­ev­er rea­son), may strug­gle with this.

Chap­ter 14 is about the CAM lay­er. While the pre­vi­ous chap­ter showed how to write a dri­ver for a disk device, chap­ter 14 gave an overview about how to an HBA to the CAM lay­er. It is just an overview, it looks like CAM needs a book on its own to be ful­ly described. The sim­ple (and most impor­tant) cas­es are described, with the hardware-specific parts being an exer­cise for the per­son writ­ing the device dri­ver. I have the impres­sion it gives enough details to let some­one with hard­ware (or pro­to­col), and more impor­tant­ly doc­u­men­ta­tion for this device, start writ­ing a driver.

It would have been nice if chap­ter 13 and 14 would have had a lit­tle schemat­ic which describes at which lev­el of the kernel-subsystems the cor­re­spond­ing dri­ver sits. And while I am at it, a schemat­ic with all the dri­ver com­po­nents dis­cussed in this book at the begin­ning as an overview, or in the end as an annex, would be great too.

An overview of USB dri­vers is giv­en in chap­ter 15 with the USB print­er dri­ver as an exam­ple for the expla­na­tion of the USB dri­ver inter­faces. If USB would not be as com­plex as it is, it would be a nice chap­ter to start driver-writing exper­i­ments (due to the avail­abil­i­ty of var­i­ous USB devices). Well… bad luck for curi­ous peo­ple. BTW, the author gives point­ers to the offi­cial USB docs, so if you are real­ly curi­ous, feel free to go ahead. 🙂

Chap­ter 16 is the first part about net­work dri­vers. It deals with ifnet (e.g. stuff need­ed for ifcon­fig), ifme­dia (sim­pli­fied: which kind of cable and speed is sup­port­ed), mbufs and MSI(-X). As in oth­er chap­ters before, a lit­tle overview and a lit­tle pic­ture in the begin­ning would have been nice.

Final­ly, in chap­ter 17, the pack­et recep­tion and trans­mis­sion of net­work dri­vers is described. Large exam­ple code is bro­ken up into sev­er­al pieces here, for more easy dis­cus­sion of relat­ed information.

One thing I miss after reach­ing the end of the book is a dis­cus­sion of sound dri­vers. And this is sure­ly not the only type of dri­vers which is not dis­cussed, I can come up with cryp­to, firewire, gpio, watch­dog, smb and iic devices with­in a few sec­onds. While I think that it is much more easy to under­stand all those dri­vers now after read­ing the book, it would have been nice to have at least a lit­tle overview of oth­er dri­ver types and maybe even a short descrip­tion of their dri­ver methods.

Con­clu­sion: As I wrote already in the begin­ning, the book is not per­fect, but it is good. While I have not writ­ten a device dri­ver for FreeB­SD, the book pro­vid­ed enough insight to be able to write one and to under­stand exist­ing dri­vers. I real­ly hope there will be a sec­ond edi­tion which address­es the minor issues I had while read­ing it to make it a per­fect book.

Email app from Android 3.1 in Android 3.2?

As pre­vi­ous­ly report­ed, I tried the update to Android 3.2 on my Tab and was not hap­py about the new EMail app. At the week­end I had a lit­tle bit of time, so I tried to get the Email.apk from Android 3.1 into Android 3.2.

Long sto­ry short, I failed.

Tita­ni­um­Back­up PRO was restor­ing or hours (the option to migrate from a dif­fer­ent ROM ver­sion was enabled) until I killed the app, and it did not get any­where (I just emailed their sup­port if I did some­thing com­plete­ly stu­pid, or of this is a bug in TB). And a copy by hand into /system/apps did not work (app fails to start).

Ideas wel­come.

Web­Sphere 7: solu­tion to “pass­word is not set” while there is a pass­word set

I googled a lot regard­ing the error mes­sage “pass­word is not set” when test­ing a data­source in Web­Sphere (7.0.0.21), but I did not find a solu­tion. A co-worker final­ly found a solu­tion (by accident?).

Prob­lem case

While hav­ing the appli­ca­tion JVMs run­ning, I cre­at­ed a new JAAS-J2C authen­ti­ca­tor (in my case the same login but a dif­fer­ent pass­word), and changed the data­source to use the new authen­ti­ca­tor. I saved the con­fig and syn­chro­nized it. The files config/cells/cell­name/nodes/node­name/resources.xml and config/cells/cell­name/security.xml showed that the changes arrived on the node. Test­ing the data­source con­nec­tiv­i­ty fails now with:

DSRA8201W: Data­Source Con­fig­u­ra­tion: DSRA8040I: Failed to con­nect to the Data­Source.  Encoun­tered java.sql.SQLException: The appli­ca­tion serv­er reject­ed the con­nec­tion. (Pass­word is not set.)DSRA0010E: SQL State = 08004, Error Code = ‑99,999.

Restart­ing the appli­ca­tion JVMs does not help.

Solu­tion

After stop­ping every­thing (appli­ca­tion JVMs, nodeagent and deploy­ment man­ag­er) and start­ing every­thing again, the con­nec­tion test of the data­source works direct­ly as expected.

I have not test­ed if it is enough to just stop all appli­ca­tion JVMs on one node and the cor­re­spding nodeagent, or if I real­ly have to stop the deploy­ment man­ag­er too.