Stat­ic DTrace probes for the linuxu­lat­or up­dated

I got a little bit of time to up­date my 3 year old work of adding stat­ic DTrace probes to the linuxu­lat­or.

The changes are not in HEAD, but in my linuxulator-​dtrace branch. The re­vi­sion to have a look at is r230910. In­cluded are some DTrace scripts:

  • script to check in­tern­al locks
  • script to trace fu­texes
  • script to gen­er­ate stats for DTra­ci­fied linuxu­lat­or parts
  • script to check for er­rors:
    • emu­la­tion er­rors (un­sup­por­ted stuff, un­known stuff, …)
    • ker­nel er­rors (re­source short­age, …)
    • pro­gram­ming er­rors (er­rors which can hap­pen if someone made a mis­take, but should not hap­pen)

The programming-​error checks give hints about user­land pro­gram­ming er­rors re­spect­ively a hint about the reas­on of er­ror re­turn val­ues due to re­source short­age or maybe a wrong com­bin­a­tion of para­met­ers. An ex­ample er­ror mes­sage for this case is “Ap­plic­a­tion %s is­sued a sy­sctl which failed the length restrictions.\nThe length passed is %d, the min length sup­por­ted is 1 and the max length sup­por­ted is %d.\n”.

The stats-​script (tailored spe­cially to the linuxu­lat­or, but this can eas­ily be ex­ten­ded to the rest of the ker­nel) can re­port about:

  • num­ber of calls to a ker­nel func­tion per ex­ecut­able bin­ary (not per PID!): al­lows to see where an op­tim­iz­a­tion would be be­ne­fi­cial for a giv­en ap­plic­a­tion
  • graph of CPU time spend in ker­nel func­tions per ex­ecut­able bin­ary: to­geth­er with the num­ber of calls to this func­tion this al­lows to de­term­ine if a ker­nel op­tim­iz­a­tion would be be­ne­fi­cial /​ is pos­sible for a giv­en ap­plic­a­tion
  • graph of longest run­ning (CPU-​time!) ker­nel func­tion in total
  • tim­ing stat­ist­ics for the emul_​lock
  • graph of longest held (CPU-​time!) locks

Un­for­tu­nately this can not be com­mit­ted to HEAD as-​is. The DTrace SDT pro­vider can not handle probes which are ad­ded to the ker­nel after the SDT pro­vider is already loaded. This means that you either have to com­pile the linuxu­lat­or stat­ic­ally in­to the ker­nel, or you have to load the SDT ker­nel mod­ule after the linuxu­lat­or mod­ule is loaded. If you do not re­spect this, you get a ker­nel pan­ic on first ac­cess of one of the pro­viders in the linuxu­lat­or (AFAIR this in­cludes list­ing the probes avail­able in the ker­nel).