Michael W. Lucas published his new book “SSH Mastery” (no link to an online store, get it from your preferred online or offline one in your part of the world).
Do you think you know a lot about SSH? I thought I did when Michael searched technical proof-readers for this book. I offered to have a look at his work in progress and he gently accepted (while I do not get money for this, I am one of the persons he thanks for the technical review in the beginning, so I am involved somehow and as such you should take the following with a grain of salt).
I already had user restrictions in place before the review, but now I narrowed down some restrictions based upon some conditionals. I already used SSH tunnels for various things before (where legally applicable), but I learned some additional VPN techniques with SSH. I already used multiple ssh-keys for various things, but Michael provides some interesting ways of handling a large-volume of ssh-keys over multiple machines. … I really hope that my review was as valuable for Michael, as it was for me to do the review.
He ends the book with “You now know more about SSH, OpenSSH and Putty than the vast majority of IT professionals! Congratulations”, and this is true, and all that in his writing style where you can come with a problem, read about it, and leave with a solution (normally with a little bit of entertainment in between).
I know a lot of people which work daily with SSH, and they know only a small part of what is presented in this book. In my opinion this book is a must-have for every System/Database/Application/Whatever Administrator in charge of something on an UNIX-like system, and even “normal users” of SSH (no matter if they use PuTTY, or a ssh command line program on an UNIX-like system (most probably it will be OpenSSH or a clone of it)) will get some helpful information from this book.
I can only recommend it.
GD Star Rating
GD Star Rating
, database application
, grain of salt
, michael w lucas
, proof readers
, putty ssh
, ssh command
, ssh putty
, system database
In the light of the recent benchmark discussion, a volunteer imported the tuning man-page into the wiki. Some comments at some places for possible improvements are already made. Please go over there, have a look, and participate please (testing/verification/discussion/improvements/…).
As always, feel free to register with FirstnameLastname and tell a FreeBSD committer to add you to the contributors group for write access (you also get the benefit to be able to register for an email notification for specific pages).
GD Star Rating
GD Star Rating
, email notification
, man page
, tuning guide
I have the habit to chmod with the relative notation (e.g. g+w or a+r or go-w or similar) instead of the absolute one (e.g. 0640 or u=rw,g=r,o=). Recently I had to chmod a lot of files. As usual I was using the relative notation. With a lot of files, this took a lot of time. Time was not really an issue, so I did not stop it to restart with a better performing command (e.g. find /path –type f –print0 | xargs –0 chmod 0644; find /path –type d –print0 | xargs –0 chmod 0755), but I thought a little tips&tricks posting may be in order, as not everyone knows the difference.
The relative notation
When you specify g+w, it means to remove the write access for the group, but keep everything else like it is. Naturally this means that chmod first has to lookup the current access rights. So for each async write request, there has to be a read-request first.
The absolute notation
The absolute notation is what most people are used to (at least the numeric one). It does not need to read the access rights before changing them, so there is less I/O to be done to get what you want. The drawback is that it is not so nice for recursive changes. You do not want to have the x-bit for data files, but you need it for directories. If you only have a tree with data files where you want to have an uniform access, the example above via find is probably faster (for sure if the directory meta-data is still in RAM).
If you have a mix of binaries and data, it is a little bit more tricky to come up with a way which is faster. If the data has a name-pattern, you could use it in the find.
And if you have a non-uniform access for the group bits and want to make sure the owner has write access to everything, it may be faster to use the relative notation than to find a replacement command-sequence with the absolute notation.
GD Star Rating
GD Star Rating
, command sequence
, little bit
, meta data
, path type
, speed traps
, time time
In my last blog-post I described how to create a new linux_base port. This blog-post is about the other Linux–ports which make up the Linux–infrastructure in the FreeBSD Ports Collection for a given Linux-release.
What are linux-infrastructure ports?
A linux_base port contains as much as possible and at the same time as little as possible to make up a useful Linux-compatibility-experience in FreeBSD. I know, this is not a descriptive explanation. And it is not on purpose. There are no fixed rules what has to be inside or what not. It “matured” into the current shape. A practical example is, that there is no GUI–stuff in the linux_base. While you need the GUI parts like GTK or QT for software like Skype and acroread, you do not need them for headless game servers. While you may need various libraries for game servers, you may not need those for Skype or acroread. As such some standard parts are in separate ports which are named linux–LINUX_DIST_SUFFIX-NAME. For GTK and the Fedora 10 release this results in linux-f10-gtk2. Such generic ports which depend upon a specific Linux-release make up the Linux-infrastructure in the FreeBSD Ports Collection. Those ports are referenced in port-Makefiles via the USE_LINUX_APPS variable, e.g. USE_LINUX_APPS=gtk2.
If you created a new linux_base port, you need most standard infrastructure ports in a version for the Linux-release used in the linux_base port, to have the Linux-application ports in the FreeBSD Ports Collection working (if you are unlucky, some ports do not play well with the Linux-release you have chosen, but this is out of the scope of this HOWTO).
First we need to set the LINUX_DIST_SUFFIX variable to a value suitable to the new Linux-release. This is done in the conditional which checks the OVERRIDE_LINUX_NONBASE_PORTS variable for valid values. Add an appropriate conditional, and do not forget to add the new valid value to the IGNORE line in the last else branch of the conditional.
The next step is to check the _LINUX_APPS_ALL and _LINUX_26_APPS variables. If there are some infrastructure ports which are not available for the new Linux-release, the conditional which checks the availability of a given infrastructure port for a given Linux-release needs to be modified. If at a later step you notice that there are some additional infrastructure ports necessary for the new Linux-release, _LINUX_APPS_ALL and the check-logic needs to be modified too (e.g. add a new variable for your Linux-release, add the content of the variable to _LINUX_APPS_ALL, and change the check to do the right thing).
After that two tedious parts need to be done.
For each infrastructure port there is a set of variables. The name_PORT variable contains the location of the port in the Ports Collection. Typically you do not have to change it (if you really want to change it, do not do it, fix the naming of the infrastructure port instead), because we use a naming convention here which includes the LINUX_DIST_SUFFIX. The name_DETECT variable is an internal variable, do not change it (if you create a new infrastructure port, copy it from somewhere else and make sure the name in value of the variable matches the port name in the name of the variable). Then there are several name_suffix_FILE variables. Leave the existing ones alone, and add a new one with the correct suffix for your new Linux-release. The value of the variable needs to be an important file which is installed by the infrastructure port in question. FYI: The content of the name_suffix_FILE variables are used to set the name_DETECT variables, depending on the Linux-relase the name_DETECT variables are used to check if the port is already installed. Ideally the name_suffix_FILE variable points to a library in the port. The name_DEPENDS variable lists dependencies of this infrastructure port. If the dependencies changed in your Linux-release, you need to add a conditional to change the dependency if LINUX_DIST_SUFFIX is set to your Linux-release.
Normally this is all what needs to be done in PORTSDIR/Mk/bsd.linux-apps.mk, the rest of the file is code to check dependencies and some correctness checks.
The second tedious part is to actually create all those infrastructure ports. Normally you can copy an existing infrastructure port, rename it, adjust the PORTNAME, PORTVERSION, PORTREVISION, MASTER_SITES, PKGNAMEPREFIX, DISTFILES, CONFLICTS (also in all other Linux-release versions of this infrastructure port), LINUX_DIST_VER, RPMVERSION (if set/neccesary) and SRC_DISTFILE variables, generate the distfile checksums (make makesum), and fix the plist. I suggest to script parts of this work (as of this writing Freshports counts 68 ports where the portname starts with linux-f10-).
Adding new infrastructure ports, or removing infrastructure ports for a given Linux-release
If your Linux-release does not come with a package for an existing infrastructure port, just do not create a corresponding name_suffix_FILE line. You still need to do the right thing regarding dependencies of ports which depend upon this non-existing infrastructure port (if your Linux-release comes with packages for them).
To add a new infrastructure port, copy an existing block, rename the variables, set them correctly, add a new variable for your Linux-release in the first _LINUX_APPS_ALL section, add the content of this variable to _LINUX_APPS_ALL, and change the check-logic as described above.
If you have something which installs and deinstalls correctly, feel free to provide it on freebsd-emulation@FreeBSD.org for review/testing. If you have questions during the porting, feel also free to send a mail there.
GD Star Rating
GD Star Rating
Tags: descriptive explanation
, freebsd ports collection
, game servers
, headless game
, linux application
, linux base
, linux compatibility
, linux dist
, linux ports
, linux release
FreeBSD is in need of a new linux_base port. It is on my TODO list since a long time, but I do not get the time to create one. I still do not have the time to work on a new one, but when you read this, I managed to get the time to create a HOWTO which describes what needs to be done to create a new linux_base port.
I will not describe how to create a new linux_base port from scratch, I will just describe how you can copy the last one and update it to something newer based upon the existing infrastructure for RPM packages.
Specific questions which come up during porting a new Linux release should be asked on freebsd-emulation@FreeBSD.org, there are more people which can answer questions than here in my blog. I will add useful information to this HOWTO if necessary.
In the easy case most of the work is searching the right RPMs and their dependencies to use, and to create the plist.
Why do we need a new linux_base port?
The current linux_base port is based upon Fedora 10, which is end of life since December 2009. Even Fedora 13 is already end of life. Fedora 16 is supposed to be released this year. From a support point of view, Fedora 15 or maybe even Fedora 16 would be a good target for the next linux_base port. Other alternatives would be to use an extended lifetime release of another RPM based distribution, like for example CentOS 6 (which seems to be based upon Fedora 12 with backports from Fedora 13 and 14). Using a Linux release which is told to be supported for at least 10 years, sounds nice from a FreeBSD point of view (only minor changes to the linux ports in such a case, instead of creating a complete new linux_base each N+2 releases like with Fedora), but it also means additional work if you want to create the first linux_base port for it.
The mysteries you have to conquer if you want to create a new linux_base port
What we do not know is, if Fedora 15/16, CentOS 6, or any other Linux release will work in a supported FreeBSD release. There are two ways to find this out.
The first one is to take an existing Linux system, chroot into it (either via NFS or after making a copy into a directory of a FreeBSD system), and to run a lot of programs (acroread, skype, shells, scripts, …). The LTP testsuite is not that much useful here, as it will test mostly kernel features, but we do not know which kernel features are mandatory for a given userland of a Linux release.
The second way of testing if a given Linux release works on FreeBSD is to actually create a new linux_base port for it and test it without chrooting.
The first way is faster, if you are only interested in testing if something works. The second way provides an easy to setup testbed for FreeBSD kernel developers to fix the Linuxulator so that it works with the new linux_base port. Both ways have their merits, but it is up to the person doing the work to decide which way to go.
The meat: HOWTO create a new linux_base port
First off, you need a system (or a jail) without any linux_base port installed. After that you can create a new linux_base port (= lbN), by just making a copy of the latest one (= lbO). In lbN you need to add lbO as a CONFLICT, and in all other existing linux_base ports, you need to add lbN as a conflict.
Change the PORTNAME, PORTVERSION, reset the PORTREVISION in lbN, and set LINUX_DIST_VER to the new Linux-release version in the lbN Makefile (this is used in PORTSDIR/Mk/bsd.linux-rpm.mk and PORTSDIR/Mk/bsd.linux-apps.mk).
If you do not stay with Fedora, there is some more work to do before you can have a look at chosing RPMs for installation. You need to have a look at PORTSDIR/Mk/bsd.linux-rpm.mk and add some cases for the new LINUX_DIST you want to use. Do not forget to set LINUX_DIST in the lbN Makefile to the name of the distribution you use. You also need to augment the LINUX_DIST_VER check in PORTSDIR/Mk/bsd.linux-rpm.mk with some LINUX_DIST conditionals. If you are lucky, the directory structure for downloads is similar to the Fedora structure, and there is not a lot to do here.
When this is done, you can have a look at the BIN_DISTFILES variable in the lbN Makefile. Try to find similar RPMs for the new Linux release you want to port. Some may not be available, and it may also be the case that different ones are needed instead. I suggest to first work with the ones which are available (make makesum, test install and create plist). After that you need to find out what the replacement RPMs for non-existing ones are. You are on your own here. Search around the net, and/or have a look at the dependencies in the RPMs of lbO to determine if something was added as a dependency of something else or not (if not, forget about it ATM). When you managed to find replacement RPMs, you can now have a look at the dependencies of the RPMs in lbN. Do not add blindly all dependencies, not all are needed in FreeBSD (the linux_base ports are not supposed to create an environment which you can chroot into, they are supposed to augment the FreeBSD system to be able to run Linux programs in ports like they where FreeBSD native programs). What you need in the linux_base ports are libraries, config and data files which do not exist in FreeBSD or have a different syntax than in FreeBSD (those config or data files which are just in a different place, can be symlinked), and basic shell commands (which commands are needed or not… well… good question, in the past we made decisions what to include based upon problem reports from users). Now for the things which are not available and where not added as a dependency. Those are things which are either used during install, or where useful to have in the past. Find out by what it was replaced and have a look if this replacement can easily be used instead. If it can be used, add it. If not, well… bad luck, we (the FreeBSD community) will see how to handle this somehow.
If you think that you have all you need in BIN_DISTFILES, please update SRC_DISTFILES accordingly and generate the distfile via make –DPACKAGE_BUILDING makesum to have the checksums of the sources (for legal reasons we need them on our mirrors).
The next step is to have a look at REMOVE_DIRS, REMOVE_FILES and ADD_DIRS if something needs to be modified. Most of them are there to fall back to the corresponding FreeBSD directories/files, or because they are not needed at all (REMOVE_*). Do not remove directories from ADD_DIRS, they are created here to fix some edge conditions (I do not remember exactly why we had to add them, and I do not take the time ATM to search in the CVS history).
If you are lucky, this is all (make sure the plist is correct). If you are not lucky and you need to make some modifications to files, have a look at the do-build target in the Makefile, this is the place where some changes are done to create a nice user experience.
If you arrive here while creating a new linux_base port, lean back and feel a bit proud. You managed to create a new linux_base port. It is not very well tested at this moment, and it is far from everything which needs to be done to have the complete Linux infrastructure for a given Linux release, but the most important part is done. Please notify freebsd-emulation@FreeBSD.org and call for testers.
What is missing?
The full Linuxulator infrastructure for the FreeBSD Ports Collection has some more ports around a linux_base port. Most of the infrastructure for this is handled in Mk/bsd.linux-apps.mk.
UPDATE: I got some time to write how to update the Linux-infrastructure ports.
GD Star Rating
GD Star Rating
Tags: easy case
, linux base
, linux ports
, linux release
, minor changes