|
|
|
How to Build Linux Kernels |
|
|
|
This is a sizeable project, so let's start with an overview of the steps:
Now that you see where we're going with this, let's get started!
The kernel source itself is available from kernel.org via both http and perhaps more efficiently through FTP.
You do know how to use anonymous FTP, right? If not, see my how-to guide.
You will need the kernel tar archive file,
a rather large file named something like:
linux-release.tar.bz2
and the corresponding very small digital signature file:
linux-release.tar.bz2.sign
where release is either three or four
numbers separated by dots, something like 3.1
or 3.1.6
You need to understand the Linux release numbering scheme. Some kernel releases are two numbers: 3.X to be general, or for example: 3.0, 3.1, 3.2, and so on. The difference between these is that 3.(X+1) contains new features not found in 3.X. New device drivers, for recent hardware (maybe a network interface based on a new chipset) or for recently added software features (maybe a new networking protocol, or a new type of file system). Plus, of course, all the bugs found and fixed in the 3.X. The two-number 3.X kernel release include new features and bug fixes as that last number X increases.
But some kernel releases are three numbers: 3.X.Y. These are bug fixes only — 3.X.1 makes a small bug fix to the 3.X.0 kernel, 3.X.2 fixes a small bug in 3.X.1, and so on. The kernel organization has strict rules: these must be bug fixes only (no sneaking new features into the kernel!) and they must be small (100 lines or less of C code). The three-number 3.X.Y kernel releases are cumulative bug fixes to 3.X.
Which version do you need? That depends on your hardware — the latest 3.X.Y should be fine, but you may not really need the very latest if the recent bug fixes were in drivers for hardware you don't own.
Reason to use a later release — Your hardware includes a super-fast network interface chipset that was only supported starting with the 3.X release. You need to use at least 3.X!
Reason to use an even later release — You know that you need 3.X, and let's say it's the last stable major release. But look at the archive, is there a more recent 3.X.Y bug-fix release? If you are about to build a kernel, it only makes sense to include the latest tested bug fixes.
Reason not to upgrade to a later release — Your hardware is supported by the 3.X kernel you are already using, and there are no 3.X.Y bug fixes that involve drivers that you use.
You do not want to run some hacker's operating system instead of the real kernel!
You must verify both the integrity of the data you just downloaded and the identity of its source. In other words, is it really the kernel source, and did it really come from the Linux kernel organization?
To be safe, you need to check the digital signature with GNU Privacy Guard (GnuPG) or similar.
See the digital signature section of my "Just Enough Cryptography" page for an explanation of what digital signatures are.
See the page on verifying digital signatures to learn how to apply this technology. Verification is vital, but an explanation right here would make this page at least as much about digital signatures as about building Linux kernels! Really, go look at that other page, this page will be right here waiting for you when you get back....
From here forward, I will use oldrelease and release to stand for, well, whatever your old and new releases will be! You will have to think just a little.
AlphaServer DS10 and AlphaStation 200 4/233 running CentOS Linux.
You need to do some steps as root here, so become root with the su command.
Your distribution or earlier work by yourself or other administrators may have already installed kernel source under a directory named /usr/src/linux-oldrelease with a symbolic link pointing to it. Find out, and if necessary, remove that link:
# cd /usr/src # ls -l # rm linux
Extract the kernel source with the following. You must be in the right directory, /usr/src, and I leave it up to you to figure out where you put that downloaded archive file. I mean, really, if you can't handle that then you have no business trying to build a kernel. But my experience tells me that I need to put two commands here to keep some people from making horrible messes of things:
# cd /usr/src # tar xvfj /path/to/linux-release.tar.bz2
Recreate the symbolic link, so that other software can be compiled on your system. If you're really curious, software making calls to system libraries need to find the include files under /usr/src/linux/include and this will support that. Or just take my word for it, this is important:
# ln -s linux-release linux # ls -l
The tar archive may have created files owned by some random UID other than 0, meaning root. This is not good! Fix it and make sure that everything has worked so far:
# chown -R root:root linux-release # ls -laF
You should see something like the following:
total 16 drwxr-xr-x 4 root root 4096 May 28 19:02 ./ drwxr-xr-x 15 root root 4096 May 28 21:28 ../ lrwxrwxrwx 1 root root 17 May 28 19:02 linux -> linux-release@ drwxr-xr-x 19 root root 4096 May 28 19:02 linux-release/ drwxr-xr-x 19 root root 4096 Apr 06 08:29 linux-oldrelease/
Now, just to be safe, let's remove any stale object files inappropriately cluttering your source tree, so we can build a fresh clean kernel:
# cd /usr/src/linux # make clean
From here on, we will just rely on /usr/src/linux to be set up and pointing to the new source tree, making the pathnames a little simpler if we need to mention them. However, almost everything should be done in the directory /usr/src/linux and so there is little need to mention full path names!
There are thousands of choices to be made when configuring a kernel build, and if you just dive in and start answering them you probably will not get enough of them right to produce a kernel that boots.
However, you are running some kernel that boots and runs on your hardware. I would presume that it is probably of an earlier release if you are like most people who would read this page, but its configuration would be a good starting point.
The very best solution would be to ask the running kernel how it is configured. That way you would get the truth. So, try that:
# zcat /proc/config.gz > .config
If that didn't work, maybe your kernel supports this but only as a loadable module. So, let's try loading that module and asking again:
# modprobe ikconfig # zcat /proc/config.gz > .config
If that worked, great, move on to the next step. If it did not work, you will need to find a configuration file that your distribution has probably stashed away somewhere on the system. Try this while paying attention to the available file name or names. Pick the latest if you have a choice:
# ls /boot # cp /boot/config-latest-release .config
Red Hat has been known to hide the configuration files in other places:
# ls /usr/src/linux*/.config # ls /usr/src/linux*/configs
If all else fails:
# find / -name '.config' -o -name 'config-*'
Or, worse yet, try the following, where CONFIG_EXPERIMENTAL is a configuration variable set in the file regardless of architecture. You will probably find that string in other files so you will need to think a little about what you find.
# find / -type f -exec grep 'CONFIG_EXPERIMENTAL=[yn]' {} /dev/null \;
You need to somehow install something appropriate as /usr/src/linux/.config, otherwise the rest of this project will fail.
Let's say that you would like your new kernel to describe
itself as version
3.X.Y-whatever
instead of plain old
3.X.Y
when someone runs the command
$ uname -r
or
$ uname -a
The trick is to edit the file Makefile and change its opening to the following. Do not change the PATCHLEVEL and SUBLEVEL values! Leave those numbers, shown here as X and Y, just as they already are.
VERSION = 3 PATCHLEVEL = X SUBLEVEL = Y EXTRAVERSION = -whatever
Be careful — you must be very conservative in what you put here! The string needs to be fairly short and contain nothing but letters, numbers, "." and "-". Any other punctuation marks or white space will cause horrible problems later. Also be careful to not add any white space to the ends of the lines, some very literal-minded scripts will process the Makefile.
Best practice: Leave this alone unless you know what you're doing! But if you do change EXTRAVERSION:
Now you are ready for the fun part! You are finally ready to run the kernel build configuration tool. I am being a little fussy by saying "kernel build configuration" instead of "kernel configuration", but that is what you are doing. You are specifying how to build a kernel, as we do with Linux and BSD. You are not configuring pre-existing kernel modules as you might do with Solaris.
The more friendy interface is started like this:
# make gconfig &
However, that may require some Gnome or GTK packages that you haven't installed, and either you haven't set up YUM or urpmi to make installation easy or you just don't want to add those packages. In that case try the following. If that doesn't work either, then it's time to install those packages....
# make xconfig &
The terminal emulator where you ran that command will have lots of output, including some error messages about setting kernel variables that were not defined and perhaps about specified settings for kernel variables that do not exist. Don't panic! Remember what you did — you started with the configuration of an older kernel to build a newer one. The kernel feature set changes from version to version, and so we expect to see these rather mild warning messages.
I believe that it only makes sense to build the kernel
configuration into the kernel itself.
Do this:
Under General Setup select:
Kernel .config support
and then select:
Enable access to .config through /proc/config.gz
If you build it into the monolithic kernel (shown as a check
mark in the configuration tool) it will always be there.
If you build it as a loadable module then it will only appear
when the appropriate module is loaded, you will need to
first do something like this:
# modprobe ikconfig
and then the kernel data structure will appear as
/proc/config.gz.
I would suggest simply building this into the monolithic
kernel so the configuration is always available.
Your configuration should be pretty close to what you want, and in many or even most cases it would be reasonable to just save the settings at that point and exit the tool.
However, you probably should explore the kernel configuration just to learn about how many things the Linux kernel can support.
Just try not to do anything terribly silly like disabling support for your hard drive controller. Unless you really do want a kernel that will not boot...
Save and exit when you have finished exploring.
This is much easier than many people expect. Start by seeing the list of available build targets:
# make help | less
There are two main ways of doing this. The traditional method builds a monolithic kernel and a huge collection of load modules, and a later step easily puts them all into their proper locations. Alternatively, you could build an RPM package.
Start building the monolithic kernel and loadable modules:
# make all
If you have multiple CPU cores, you might use the -jN option to allow multiple simultaneous processes. For example, if you have four cores, this may finish almost four times as quickly:
# make -j4 all
However, wouldn't it be better to include your kernel in your configuration management? This is easier when you build the kernel as a package. Let's build both source and binary RPM packages. Add -jN as appropriate:
# make rpm-pkg
Now be patient... Go get some coffee or make a pot of tea. Or maybe get lunch. This will take a while. Probably an hour or more on a single core. On a system with an AMD Phenom II X4 965 processor with 4 cores, below is the CPU core utilization for using the -j4 option, at right, and without it, at left.
xosview -mem -page -swap -ints +net & |
xosview -mem -page -swap -ints +net & |
| Platform | Command | Time |
| Intel Core 2 Duo 2 cores (using 1) |
make rpm-pkg |
87m 10s real 60m 41s user 20m 25s system |
| Intel Core 2 Duo 2 cores (using 2) |
make -j2 rpm-pkg |
53m 59s real 69m 10s user 23m 38s system |
| Intel i7 4 cores (using 1) |
make rpm-pkg |
42m 41s real 26m 00s user 04m 17s system |
| Intel i7 4 cores (using 2) |
make -j2 rpm-pkg |
25m 04s real 35m 38s user 06m 02s system |
| Intel i7 4 cores (using 4) |
make -j4 rpm-pkg |
15m 56s real 38m 09s user 06m 42s system |
Here are the timing results of kernel builds with different numbers of cores in use.
| real | Total elapsed time, what the clock on the wall would show. |
| user | Time spent running user processes. This is the total time the CPUs spent doing the actual computation. |
| system | Time spent running kernel processes. For this task, this value is going to be disk I/O. |
The i7 CPU goes faster than the Core 2 CPU, the single-core build takes less than half as much user time (that is, actual computation) on the i7.
The Intel i7 platform has much faster disks, with an 8 GB solid-state disk in front of a faster mechanical disk. Look at how much less system time (file system I/O) is needed! Only about 20-25% of the required system time, so the i7 platform's disks are four to fives times as fast. So, the system and real (wall-clock) times can't be directly compared between the Intel Core 2 Duo and the Intel i7.
Yes, the total user and system times increase slightly with an increase in the number of cores in use. I believe this is because some things are needlessly re-compiled: a compilation has started on one core, it isn't finished yet, and so it is also started on another core.
This is much easier than it used to be. It hasn't been difficult for some time, but some people are still scared off by criticisms that haven't been appropriate for ages now.
# make modules_install # make install
The resulting collection of modules will be quite large, around 2 GB, because the modules include debugging symbols. You can strip those out by using the following sequence instead. This results in the same set of modules occupying just 130-150 MB:
# make INSTALL_MOD_STRIP=1 modules_install # make install
First, let's see what got built:
# tree ~/rpmbuild/*RPMS
Let's be cautiously confident — our new kernel is almost certainly going to work! But just in case it doesn't, we don't want to remove our existing and functioning kernel. So, we need to simply install the new kernel, leaving the old one in place, as opposed to doing it as an upgrade and removing the old one. Something like this, modified as needed to reflect your architecture and kernel release:
# rpm -ivh /usr/src/whatever/RPMS/arch/kernel-release.arch.rpm
We still need to build and install an "initrd", an initial RAM disk image, and then modify the boot loader. That sounds awfully complicated, and it used to be. Now it's just one long command.
# new-kernel-pkg --mkinitrd --install --make-default --depmod release
What do you specify for "release" in the above? Look at Makefile. If yours looks like the below, then you would use 3.6.9-something.
VERSION = 3 PATCHLEVEL = 6 SUBLEVEL = 9 EXTRAVERSION = -something
Edit the GRUB configuration file, either /boot/grub/menu.lst or grub.conf. If you built the kernel in the traditional way, move the stanza for your new kernel to the top of the list and change the default to 0 (GRUB starts counting at zero). Notice that if you built it as a package, one of the options to our new-kernel-pkg command has already made this the default! It is going to work, right?
Do comment out any hiddenmenu line that your distribution might have installed initially because they thought that you shouldn't see what's going on.
default=0
timeout=10
splashimage=(hd0,0)/grub/splash.xpm.gz
## hiddenmenu
title linux-3.X.Y-whatever
root (hd0,0)
kernel /vmlinuz-3.X.Y-whatever ro root=LABEL=/ quiet
initrd /initrd-3.X.Y-whatever.img
title Mandriva Linux 2.6.33.6-desktop
root (hd0,0)
kernel /vmlinuz-2.6.33.6-desktop ro root=LABEL=/ quiet
initrd /initrd-2.6.33.6-desktop.img
If you're still using LILO and you don't want to upgrade, then you will need to edit /etc/lilo.conf and make the appropriate changes before running lilo -v to recreate the boot block. But seriously, LILO? Today?
If you're doing this on an Alpha, ignore all this about GRUB and LILO and modify /etc/aboot.conf and then go see my page specifically about building kernels on Alphas for some further details.
# aboot default configurations # NOTICE: You have a /boot partition. This means that # all kernel paths are relative to /boot/ 0:2/vmlinuz-3.X.Y-whatever initrd=/initrd-3.X.Y-whatever root=/dev/sda4 quiet 1:2/vmlinuz-current initrd=/initrd-current root=/dev/sda4 quiet 2:2/vmlinuz-previous initrd=/initrd-previous root=/dev/sda4 quiet
You didn't do anything silly like get rid of your functional old kernel, right? Or configure your boot loader in such a way that it isn't available? Did you??
Pretty large! This command sequence will show that the collection of new modules is much larger than what you installed from Red Hat or CentOS:
# du -sh /lib/modules/*
Why is there such a big difference in sizes? And, I suspect, in the kernel load time during the initial stages of booting?
The kernel modules were stripped of their symbols. See the manual page for the strip command for what is going on. I have not yet found an easy way to ask the kernel build process to do this for me.
You should have made the very reasonable choice of asking for the kernel's configuration, the .config file, to be built into the kernel and made available as /proc/config.gz. That means that all you have to keep track of is your kernel release and a copy of your .config file, a file of a little over 100 kbytes, about a quarter of that if compressed with gzip.
Click here to explore Linux kernel details.
|
|
|||||||||
|
|||||||||
|
|||||||||
|
| © Bob Cromwell May 2013. Created with /bin/vi and ImageMagick, hosted on OpenBSD with Apache. Root password available here, privacy policy here. | |||||
