ADSL RAM … final­ly aban­doned (but with good news)

As I already wrote, the­o­ret­i­cal­ly ADSL RAM is avail­able at my place. The analy­sis of the sit­u­a­tion revealed first that the ISP side of my line uses out­dat­ed hard­ware. After the tech­ni­cian I know unof­fi­cial­ly took care about it (remote­ly switch­ing me to a dif­fer­ent port), I have seen an imme­di­ate improve­ment of the sig­nal to noise ratio. It is about 20 dB better.

Unfor­tu­nate­ly this was not enough to be able to switch to the rate adap­tive mode. Accord­ing to their data­base the line length allows to give me 1.5 MBit. My line is run­ning already at 2 MBit and my ADSL modem tells me it could do 8 MBit, so I dis­agree a bit with their database.

As the tech­ni­cian agrees with me, the next step would be to tem­po­rary move my house by some hun­dred meters towards the ISP end­point of the line, unfor­tu­nate­ly the high­er man­age­ment seems to be hav­ing some busi­ness ideas with our region (FTTT, Fiber To The Town (which means we will prob­a­bly get 16 MBit via ADSL) … but maybe even FTTH), so they are now mon­i­tor­ing the data­base for such changes since a while.

I have the impres­sion they seem to pre­vent such changes to the data­base because they think that if peo­ple get 2 MBit (instead of noth­ing, large parts of a town near­by does not even have the slow­est ADSL con­nec­tion) or 8 MBit (instead of 2 MBit), they are not inter­est­ed in get­ting FTTH (or 16 MBit). Togeth­er with their IPTV ini­tia­tive I do not real­ly under­stand it. To get their IPTV, you need to have at least a 8 MBit line. With 8 MBit you can only cov­er one TV at SD res­o­lu­tion (at least with their IPTV offer), if you want HD res­o­lu­tion, you need to switch to their VDSL stuff (which is not avail­able in our town). What peo­ple are doing cur­rent­ly is to switch to a cable provider where they can get about 32 MBit (I do not switch, switch­ing is a risky action here, I rather stay with a slow con­nec­tion that to have no con­nec­tion at all for some months). With 32 MBit (and TV) peo­ple have less a need to switch to fiber (and pay 150 EUR for the work to get fiber into the house) than with 2 MBit or nothing.

The final out­come is, that the tech­ni­cian I know does not want to ask some­one to play with the data­base to move my house tem­po­rary (which I can under­stand). The good part of those news is, that I may get more than 8 MBit in the not so dis­tant future (the cur­rent plan­ning is to fin­ish the FTTT work until autumn).

Solaris UFS full while df shows plen­ty of free space/inodes

At work we have a Solaris 8 with a UFS which told the appli­ca­tion that it can not cre­ate new files. The df com­mand showed plen­ty if free inodes, and there was also enough space free in the FS. The rea­son that the appli­ca­tion got the error was that while there where still plen­ty of frag­ments free, no free block was avail­able any­more. You can not cre­ate a new file only with frag­ments, you need to have at least one free block for each new file.

To see the num­ber of free blocks of a UFS you can call “fstyp ‑v | head ‑18” and look at the val­ue behind “nbfree”.

To get this work­ing again we cleaned up the FS a lit­tle bit (compressing/deleting log files), but this is only a tem­po­rary solu­tion. Unluck­i­ly we can not move this appli­ca­tion to a Solaris 10 with ZFS, so I was play­ing around a lit­tle bit to see what we can do.

First I made a his­togram of the file sizes. The back­up of the FS I was play­ing with had a lit­tle bit more than 4 mil­lion files in this FS. 28.5% of them where small­er than or equal 512 bytes, 31.7% where small­er than or equal 1k (frag­ment size), 36% small­er than or equal 8k (block size) and 74% small­er than or equal 16k. The fol­low­ing graph shows in red the crit­i­cal part, files which need a block and pro­duce frag­ments, but can not life with only fragments.

chart

Then I played around with newfs options for this one spe­cif­ic FS with this spe­cif­ic data mix. Chang­ing the num­ber of inodes did not change much the out­come for our prob­lem (as expect­ed). Chang­ing the opti­miza­tion from “time” to “space” (and restor­ing all the data from back­up into the emp­ty FS) gave us 1000 more free blocks. On a FS which had 10 Mio free blocks when emp­ty this is not much, but we expect that the restore con­sumes less frag­ments and more full blocks than the live-FS of the appli­ca­tion (we can not com­pare, as the con­tent of the live-FS changed a lot since we had the prob­lem). We assume that e.g. the logs of the appli­ca­tion are split over a lot of frag­ments instead of full blocks, due to small writes to the logs by the appli­ca­tion. The restore should write all the data in big chunks, so our expec­ta­tion is that the FS will use more full blocks and less frag­ments. Because of this we expect that the live-FS with this spe­cif­ic data mix could ben­e­fit from chang­ing the optimization.

I also played around with the frag­ment size. The expec­ta­tion was that it will only change what is report­ed in the out­put of df (reduc­ing the report­ed avail­able space for the same amount of data). Here is the result:

chart

The dif­fer­ence between 1k (default) and 2k is not much. For 8k we would have to much unused space lost. The frag­ment size of 4k looks like it is accept­able to get a bet­ter mon­i­tor­ing sta­tus of this par­tic­u­lar data mix.

Based upon this we will prob­a­bly cre­ate a new FS with a frag­ment size of 4k and we will prob­a­bly switch the opti­miza­tion direct­ly to “space”. This way we will have a bet­ter report­ing on the fill lev­el of the FS for our data mix (but we will not be able to ful­ly use the real space of the FS) and as such our mon­i­tor­ing should alert us in time to do a cleanup of the FS or to increase the size of the FS.

DTrace probes for the Lin­ux­u­la­tor updated

If some­one had a look at the ear­li­er post about DTrace probes for the Lin­ux­u­la­tor: I updat­ed the patch at the same place. The dif­fer­ence between the pre­vi­ous one is that some D‑scripts are fixed now to do what I meant, spe­cial­ly the ones which pro­vide sta­tis­tics output.

Com­pil­ing Sam­ba 3.5.8 with AD sup­port on Solaris 10 u8

If some­one needs a sam­ba which is able to com­mu­ni­cate with an AD 2008 serv­er on a Solaris 10 sys­tem… here is how I did it.

Pre­req­ui­sites

  • /opt/SUNWspro con­tains the Stu­dio 12 compiler
  • tar­balls of openldap-stable-20100719 (2.4.23), heimdal‑1.4, samba‑3.5.8
  • export PATH=/opt/SUNWspro/bin:/usr/xpg6/bin:/usr/xpg4/bin:/usr/perl5/bin:/usr/bin:/usr/openwin/bin:/bin:/usr/sfw/bin:/usr/sfw/sbin:/sbin:/usr/sbin:/usr/sadm/admin/bin:/usr/sadm/bin:/usr/java/jre/bin:/usr/ccs/bin:/usr/ucb CC=cc CXX=CC
  • DEST=/path/to/final/location

Com­pil­ing everything

openldap-stable-20100719 (2.4.23)

export CPPFLAGS=”-I/usr/sfw/include” LDFLAGS=”-L/usr/sfw/lib ‑R/usr/sfw/lib”
./configure –prefix=$DEST/openldap‑2.4.23 –disable-slapd
make depend
make install

heimdal‑1.4

./configure –prefix=$DEST/heimdal‑1.4 –with-openldap=$DEST/openldap‑2.4.23 –with-hdbdir=$DEST/heimdal-instance/var/heimdal –sysconfdir=$DEST/heimdal-instance/etc
cd lib/hcrypto/libtommath

Unfor­tu­nate­ly heimdal‑1.4 does not con­tain all the files you need. As of this writ­ing (if you try to do this a lot lat­er, you may get more recent ver­sions which may or may not work with heim­dal 1.4) I was able to down­load them from

cd ../../..
make
make install
mkdir ‑p $DEST/heimdal-instance/var/heimdal $DEST/heimdal-instance/etc

samba‑3.5.8

export CPPFLAGS=”-I$DEST/openldap‑2.4.23/include” LDFLAGS=”-L$DEST/openldap‑2.4.23/lib ‑R$DEST/openldap‑2.4.23/lib ‑R$DEST/samba‑3.5.8/lib ‑R$DEST/heimdal‑1.4/lib”
./configure –prefix=$DEST/samba‑3.5.8 –sysconfdir=$DEST/samba-instance/etc –localstatedir=$DEST/samba-instance/var –with-privatedir=$DEST/samba-instance/private –with-lockdir=$DEST/samba-instance/var/locks –with-statedir=$DEST/samba-instance/var/locks –with-cachedir=$DEST/samba-instance/var/locks –with-piddir=$DEST/samba-instance/var/locks –with-ncalrpcdir=$DEST/samba-instance/var/ncalrpc –with-configdir=$DEST/samba-instance/config –with-ldap –with-krb5=$DEST/heimdal‑1.4 –with-ads –with-quotas –with-aio-support –with-shared-modules=vfs_zfsacl
gmake
gmake install

After that you have a sam­ba in $DEST/samba‑3.5.8, the con­fig for it should be put into $DEST/samba-instance/config and if you need to have a cus­tom krb4.conf you can put it int $DEST/heimdal-instance/etc/.