What you should know about SSH

Mi­chael W. Lu­cas pub­lished his new book “SSH Mas­tery” (no link to an on­line store, get it from your pre­ferred on­line or off­line one in your part of the world).

Do you think you know a lot about SSH? I thought I did when Mi­chael searched tech­nical proof-​readers for this book. I offered to have a look at his work in pro­gress and he gently ac­cep­ted (while I do not get money for this, I am one of the per­sons he thanks for  the tech­nical re­view in the be­gin­ning, so I am in­volved some­how and as such you should take the fol­low­ing with a grain of salt).

I already had user re­stric­tions in place be­fore the re­view, but now I nar­rowed down some re­stric­tions based upon some con­di­tion­als. I already used SSH tun­nels for vari­ous things be­fore (where leg­ally ap­plic­able), but I learned some ad­di­tional VPN tech­niques with SSH. I already used mul­tiple ssh-​keys for vari­ous things, but Mi­chael provides some in­ter­est­ing ways of hand­ling a large-​volume of ssh-​keys over mul­tiple ma­chines. … I really hope that my re­view was as valu­able for Mi­chael, as it was for me to do the re­view.

He ends the book with “You now know more about SSH, OpenSSH and Putty than the vast ma­jor­ity of IT pro­fes­sion­als! Con­grat­u­la­tions”, and this is true, and all that in his writ­ing style where you can come with a prob­lem, read about it, and leave with a solu­tion (nor­mally with a little bit of en­ter­tain­ment in between).

I know a lot of people which work daily with SSH, and they know only a small part of what is presen­ted in this book. In my opin­ion this book is a must-​have for every System/​Database/​Application/​Whatever Ad­min­is­trator in charge of some­thing on an UNIX-​like sys­tem, and even “nor­mal users” of SSH (no mat­ter if they use PuTTY, or a ssh com­mand line pro­gram on an UNIX-​like sys­tem (most prob­ably it will be OpenSSH or a clone of it)) will get some help­ful in­form­a­tion from this book.

I can only re­com­mend it.

StumbleUponXINGBalatarinBox.netDiggGoogle GmailNetvouzPlurkSiteJotTypePad PostYahoo BookmarksVKSlashdotPocketHacker NewsDiigoBuddyMarksRedditLinkedInBibSonomyBufferEmailHatenaLiveJournalNewsVinePrintViadeoYahoo MailAIMBitty BrowserCare2 NewsEvernoteMail.RuPrintFriendlyWaneloYahoo MessengerYoolinkWebnewsStumpediaProtopage BookmarksOdnoklassnikiMendeleyInstapaperFarkCiteULikeBlinklistAOL MailTwitterGoogle+PinterestTumblrAmazon Wish ListBlogMarksDZoneDeliciousFlipboardFolkdJamespotMeneameMixiOknotiziePushaSvejoSymbaloo FeedsWhatsAppYouMobdiHITTWordPressRediff MyPageOutlook.comMySpaceDesign FloatBlogger PostApp.netDiary.RuKindle ItNUjijSegnaloTuentiWykopTwiddlaSina WeiboPinboardNetlogLineGoogle BookmarksDiasporaBookmarks.frBaiduFacebookGoogle ClassroomKakaoQzoneSMSTelegramRenrenKnownYummlyShare/​Save

Tun­ing guide in the wiki

In the light of the re­cent bench­mark dis­cus­sion, a vo­lun­teer im­por­ted the tun­ing man-​page into the wiki. Some com­ments at some places for pos­sible im­prove­ments are already made. Please go over there, have a look, and par­ti­cip­ate please (testing/​verification/​discussion/​improvements/​…).

As al­ways, feel free to re­gister with First­nameLast­name and tell a FreeBSD com­mit­ter to add you to the con­trib­ut­ors group for write ac­cess (you also get the be­ne­fit to be able to re­gister for an email no­ti­fic­a­tion for spe­cific pages).

HOWTO add linux-​infrastructure ports for a new linux_​base port

In my last blog-​post I de­scribed how to cre­ate a new linux_​base port. This blog-​post is about the other Linux–ports which make up the Linux–in­fra­struc­ture in the FreeBSD Ports Col­lec­tion for a given Linux-​release.

What are linux-​infrastructure ports?

A linux_​base port con­tains as much as pos­sible and at the same time as little as pos­sible to make up a use­ful Linux-​compatibility-​experience in FreeBSD. I know, this is not a de­script­ive ex­plan­a­tion. And it is not on pur­pose. There are no fixed rules what has to be in­side or what not. It “ma­tured” into the cur­rent shape. A prac­tical ex­ample is, that there is no GUI–stuff in the linux_​base. While you need the GUI parts like GTK or QT for soft­ware like Skype and acror­ead, you do not need them for head­less game serv­ers. While you may need vari­ous lib­rar­ies for game serv­ers, you may not need those for Skype or acror­ead. As such some stand­ard parts are in sep­ar­ate ports which are named linux–LINUX_​DIST_​SUFFIX-NAME. For GTK and the Fe­dora 10 re­lease this res­ults in linux-​f10-​gtk2. Such gen­eric ports which de­pend upon a spe­cific Linux-​release make up the Linux-​infrastructure in the FreeBSD Ports Col­lec­tion. Those ports are ref­er­enced in port-​Makefiles via the USE_​LINUX_​APPS vari­able, e.g. USE_LINUX_APPS=gtk2.

If you cre­ated a new linux_​base port, you need most stand­ard in­fra­struc­ture ports in a ver­sion for the Linux-​release used in the linux_​base port, to have the Linux-​application ports in the FreeBSD Ports Col­lec­tion work­ing (if you are un­lucky, some ports do not play well with the Linux-​release you have chosen, but this is out of the scope of this HOWTO).

Up­dat­ing Mk/bsd.linux-apps.mk

 First we need to set the LINUX_​DIST_​SUFFIX vari­able to a value suit­able to the new Linux-​release. This is done in the con­di­tional which checks the OVERRIDE_​LINUX_​NONBASE_​PORTS vari­able for valid val­ues. Add an ap­pro­pri­ate con­di­tional, and do not for­get to add the new valid value to the IGNORE line in the last else branch of the con­di­tional.

The next step is to check the _​LINUX_​APPS_​ALL and _​LINUX_​26_​APPS vari­ables. If there are some in­fra­struc­ture ports which are not avail­able for the new Linux-​release, the con­di­tional which checks the avail­ab­il­ity of a given in­fra­struc­ture port for a given Linux-​release needs to be mod­i­fied. If at a later step you no­tice that there are some ad­di­tional in­fra­struc­ture ports ne­ces­sary for the new Linux-​release, _​LINUX_​APPS_​ALL and the check-​logic needs to be mod­i­fied too (e.g. add a new vari­able for your Linux-​release, add the con­tent of the vari­able to _​LINUX_​APPS_​ALL, and change the check to do the right thing).

After that two te­di­ous parts need to be done.

For each in­fra­struc­ture port there is a set of vari­ables. The name_​PORT vari­able con­tains the loc­a­tion of the port in the Ports Col­lec­tion. Typ­ic­ally you do not have to change it (if you really want to change it, do not do it, fix the nam­ing of the in­fra­struc­ture port in­stead), be­cause we use a nam­ing con­ven­tion here which in­cludes the LINUX_​DIST_​SUFFIX. The name_​DETECT vari­able is an in­ternal vari­able, do not change it (if you cre­ate a new in­fra­struc­ture port, copy it from some­where else and make sure the name in value of the vari­able matches the port name in the name of the vari­able). Then there are sev­eral name_​suf­fix_​FILE vari­ables. Leave the ex­ist­ing ones alone, and add a new one with the cor­rect suf­fix for your new Linux-​release. The value of the vari­able needs to be an im­port­ant file which is in­stalled by the in­fra­struc­ture port in ques­tion. FYI: The con­tent of the name_​suf­fix_​FILE vari­ables are used to set the name_​DETECT vari­ables, de­pend­ing on the Linux-​relase the name_​DETECT vari­ables are used to check if the port is already in­stalled. Ideally the name_​suf­fix_​FILE vari­able points to a lib­rary in the port. The name_​DEPENDS vari­able lists de­pend­en­cies of this in­fra­struc­ture port. If the de­pend­en­cies changed in your Linux-​release, you need to add a con­di­tional to change the de­pend­ency if LINUX_​DIST_​SUFFIX is set to your Linux-​release.

Nor­mally this is all what needs to be done in PORTSDIR/Mk/bsd.linux-apps.mk, the rest of the file is code to check de­pend­en­cies and some cor­rect­ness checks.

The second te­di­ous part is to ac­tu­ally cre­ate all those in­fra­struc­ture ports. Nor­mally you can copy an ex­ist­ing in­fra­struc­ture port, re­name it, ad­just the PORTNAME, PORTVERSION, PORTREVISION, MASTER_​SITES, PKGNAMEPREFIX, DISTFILES, CONFLICTS (also in all other Linux-​release ver­sions of this in­fra­struc­ture port), LINUX_​DIST_​VER, RPMVERSION (if set/​neccesary) and SRC_​DISTFILE vari­ables, gen­er­ate the dist­file check­sums (make make­sum), and fix the plist. I sug­gest to script parts of this work (as of this writ­ing Fresh­ports counts 68 ports where the port­name starts with linux-​f10-).

Adding new in­fra­struc­ture ports, or re­mov­ing in­fra­struc­ture ports for a given Linux-​release

If your Linux-​release does not come with a pack­age for an ex­ist­ing in­fra­struc­ture port, just do not cre­ate a cor­res­pond­ing name_​suf­fix_​FILE line. You still need to do the right thing re­gard­ing de­pend­en­cies of ports which de­pend upon this non-​existing in­fra­struc­ture port (if your Linux-​release comes with pack­ages for them).

To add a new in­fra­struc­ture port, copy an ex­ist­ing block, re­name the vari­ables, set them cor­rectly, add a new vari­able for your Linux-​release in the first _​LINUX_​APPS_​ALL sec­tion, add the con­tent of this vari­able to _​LINUX_​APPS_​ALL, and change the check-​logic as de­scribed above.

Fi­nal words

If you have some­thing which in­stalls and dein­stalls cor­rectly, feel free to provide it on freebsd-​emulation@​FreeBSD.​org for re­view/​testing. If you have ques­tions dur­ing the port­ing, feel also free to send a mail there.

HOWTO cre­ate a new linux_​base port

FreeBSD is in need of a new linux_​base port. It is on my TODO list since a long time, but I do not get the time to cre­ate one. I still do not have the time to work on a new one, but when you read this, I man­aged to get the time to cre­ate a HOWTO which de­scribes what needs to be done to cre­ate a new linux_​base port.

I will not de­scribe how to cre­ate a new linux_​base port from scratch, I will just de­scribe how you can copy the last one and up­date it to some­thing newer based upon the ex­ist­ing in­fra­struc­ture for RPM pack­ages.

Spe­cific ques­tions which come up dur­ing port­ing a new Linux re­lease should be asked on freebsd-​emulation@​FreeBSD.​org,  there are more people which can an­swer ques­tions than here in my blog. I will add use­ful in­form­a­tion to this HOWTO if ne­ces­sary.

In the easy case most of the work is search­ing the right RPMs and their de­pend­en­cies to use, and to cre­ate the plist.

Why do we need a new linux_​base port?

The cur­rent linux_​base port is based upon Fe­dora 10, which is end of life since Decem­ber 2009. Even Fe­dora 13 is already end of life. Fe­dora 16 is sup­posed to be re­leased this year. From a sup­port point of view, Fe­dora 15 or maybe even Fe­dora 16 would be a good tar­get for the next linux_​base port. Other al­tern­at­ives would be to use an ex­ten­ded life­time re­lease of an­other RPM based dis­tri­bu­tion, like for ex­ample Cen­tOS 6 (which seems to be based upon Fe­dora 12 with back­ports from Fe­dora 13 and 14). Us­ing a Linux re­lease which is told to be sup­por­ted for at least 10 years, sounds nice from a FreeBSD point of view (only minor changes to the linux ports in such a case, in­stead of cre­at­ing a com­plete new linux_​base each N+2 re­leases like with Fe­dora), but it also means ad­di­tional work if you want to cre­ate the first linux_​base port for it.

The mys­ter­ies you have to con­quer if you want to cre­ate a new linux_​base port

What we do not know is, if Fe­dora 15/​16, Cen­tOS 6, or any other Linux re­lease will work in a sup­por­ted FreeBSD re­lease. There are two ways to find this out.

The first one is to take an ex­ist­ing Linux sys­tem, ch­root into it (either via NFS or after mak­ing a copy into a dir­ect­ory of a FreeBSD sys­tem), and to run a lot of pro­grams (acror­ead, skype, shells, scripts, …). The LTP test­suite is not that much use­ful here, as it will test mostly ker­nel fea­tures, but we do not know which ker­nel fea­tures are man­dat­ory for a given user­land of a Linux re­lease.

The second way of test­ing if a given Linux re­lease works on FreeBSD is to ac­tu­ally cre­ate a new linux_​base port for it and test it without ch­root­ing.

The first way is faster, if you are only in­ter­ested in test­ing if some­thing works. The second way provides an easy to setup test­bed for FreeBSD ker­nel de­velopers to fix the Linuxu­lator so that it works with the new linux_​base port. Both ways have their mer­its, but it is up to the per­son do­ing the work to de­cide which way to go.

The meat: HOWTO cre­ate a new linux_​base port

First off, you need a sys­tem (or a jail) without any linux_​base port in­stalled. After that you can cre­ate a new linux_​base port (= lbN), by just mak­ing a copy of the latest one (= lbO). In lbN you need to add lbO as a CONFLICT, and in all other ex­ist­ing linux_​base ports, you need to add lbN as a con­flict.

Change the PORTNAME, PORTVERSION, re­set the PORTREVISION in lbN, and set LINUX_​DIST_​VER  to the new Linux-​release ver­sion in the lbN Make­file (this is used in PORTSDIR/Mk/bsd.linux-rpm.mk and PORTSDIR/Mk/bsd.linux-apps.mk).

If you do not stay with Fe­dora, there is some more work to do be­fore you can have a look at chos­ing RPMs for in­stall­a­tion. You need to have a look at PORTSDIR/Mk/bsd.linux-rpm.mk and add some cases for the new LINUX_​DIST you want to use. Do not for­get to set LINUX_​DIST in the lbN Make­file to the name of the dis­tri­bu­tion you use. You also need to aug­ment the LINUX_​DIST_​VER check in PORTSDIR/Mk/bsd.linux-rpm.mk with some LINUX_​DIST con­di­tion­als. If you are lucky, the dir­ect­ory struc­ture for down­loads is sim­ilar to the Fe­dora struc­ture, and there is not a lot to do here.

When this is done, you can have a look at the BIN_​DISTFILES vari­able in the lbN Make­file. Try to find sim­ilar RPMs for the new Linux re­lease you want to port. Some may not be avail­able, and it may also be the case that dif­fer­ent ones are needed in­stead. I sug­gest to first work with the ones which are avail­able (make make­sum, test in­stall and cre­ate plist). After that you need to find out what the re­place­ment RPMs for non-​existing ones are. You are on your own here. Search around the net, and/​or have a look at the de­pend­en­cies in the RPMs of lbO to de­term­ine if some­thing was ad­ded as a de­pend­ency of some­thing else or not (if not, for­get about it ATM). When you man­aged to find re­place­ment RPMs, you can now have a look at the de­pend­en­cies of the RPMs in lbN. Do not add blindly all de­pend­en­cies, not all are needed in FreeBSD (the linux_​base ports are not sup­posed to cre­ate an en­vir­on­ment which you can ch­root into, they are sup­posed to aug­ment the FreeBSD sys­tem to be able to run Linux pro­grams in ports like they where FreeBSD nat­ive pro­grams). What you need in the linux_​base ports are lib­rar­ies, con­fig and data files which do not ex­ist in FreeBSD or have a dif­fer­ent syn­tax than in FreeBSD (those con­fig or data files which are just in a dif­fer­ent place, can be sym­linked), and ba­sic shell com­mands (which com­mands are needed or not… well… good ques­tion, in the past we made de­cisions what to in­clude based upon prob­lem re­ports from users). Now for the things which are not avail­able and where not ad­ded as a de­pend­ency. Those are things which are either used dur­ing in­stall, or where use­ful to have in the past. Find out by what it was re­placed and have a look if this re­place­ment can eas­ily be used in­stead. If it can be used, add it. If not, well… bad luck, we (the FreeBSD com­munity) will see how to handle this some­how.

If you think that you have all you need in BIN_​DISTFILES, please up­date SRC_​DISTFILES ac­cord­ingly and gen­er­ate the dist­file via  make –DPACKAGE_​BUILDING make­sum to have the check­sums of the sources (for legal reas­ons we need them on our mir­rors).

The next step is to have a look at REMOVE_​DIRS, REMOVE_​FILES and ADD_​DIRS if some­thing needs to be mod­i­fied. Most of them are there to fall back to the cor­res­pond­ing FreeBSD directories/​files, or be­cause they are not needed at all (REMOVE_​*). Do not re­move dir­ect­or­ies from ADD_​DIRS, they are cre­ated here to fix some edge con­di­tions (I do not re­mem­ber ex­actly why we had to add them, and I do not take the time ATM to search in the CVS his­tory).

If you are lucky, this is all (make sure the plist is cor­rect). If you are not lucky and you need to make some modi­fic­a­tions to files, have a look at the do-​build tar­get in the Make­file, this is the place where some changes are done to cre­ate a nice user ex­per­i­ence.

If you ar­rive here while cre­at­ing a new linux_​base port, lean back and feel a bit proud. You man­aged to cre­ate a new linux_​base port. It is not very well tested at this mo­ment, and it is far from everything which needs to be done to have the com­plete Linux in­fra­struc­ture for a given Linux re­lease, but the most im­port­ant part is done. Please no­tify freebsd-​emulation@​FreeBSD.​org and call for test­ers.

What is miss­ing?

The full Linuxu­lator in­fra­struc­ture for the FreeBSD Ports Col­lec­tion has some more ports around a linux_​base port. Most of the in­fra­struc­ture for this is handled in Mk/bsd.linux-apps.mk.

UPDATE: I got some time to write how to up­date the Linux-​infrastructure ports.