Alexander Leidinger

Just another weblog

Feb
05

Show­ing off some num­bers…

At work we have some per­form­ance prob­lems.

One ap­plic­a­tion (not off-the-shelf soft­ware) is not per­form­ing good. The prob­lem is that the design of the ap­plic­a­tion is far from good (auto-commit is used, and the Or­acle DB is do­ing too much writes for what the ap­plic­a­tion is sup­posed to do be­cause of this). Dur­ing help­ing our DBAs in their per­form­ance ana­lysis (the vendor of the ap­plic­a­tion is telling our hard­ware is not fast enough and I had to provide some num­bers to show that this is not the case and they need to im­prove the soft­ware as it does not com­ply to the per­form­ance re­quire­ments they got be­fore de­vel­op­ing the ap­plic­a­tion) I no­ticed that the filesys­tem where the DB and the ap­plic­a­tion are loc­ated (a ZFS if someone is in­ter­ested) is do­ing some­times 1.200 IO (write) op­er­a­tions per second (to write about 100 MB). Yeah, that is a lot of IOops our SAN is able to do! Un­for­tu­nately too ex­pens­ive to buy for use at home. :(

An­other ap­plic­a­tion (nagios 3.0) was gen­er­at­ing a lot of ma­jor faults (caused by a lot of fork()s for the checks). It is a Sun­Fire V890, and the highest num­ber of MF per second I have seen on this ma­chine was about 27.000. It never went be­low 10.000. On av­er­age maybe some­where between 15.000 and 20.000. My Sol­aris–Desktop (an Ul­tra 20) is gen­er­at­ing maybe sev­eral hun­dred MF if a lot is go­ing on (most of the time is does not gen­er­ate much). Nobody can say the V890 is not used… :) Oh, yes, I sug­ges­ted to en­able the nagios con­fig set­ting for large sites, now the ma­jor faults are around 0 – 10.000 and the ma­chine is not that stressed any­more. The next step is prob­ably to have a look at the an­cient probes (mi­grated from the big brother setup which was there sev­eral years be­fore) and re­duce the num­ber of forks they do.

Share/Save

Tags: , , , , , , , , ,