Compiling the Linux Kernel

This post will serve as a primer to people who are new to the world of Linux hacking, and are attempting to compile the Linux kernel from source. The various steps from downloading the kernel source to booting from the new kernel image are explained. Also given are tips on cleaning up the source code, doing verbose compilation etc.

1. Downloading the kernel source code

In order to compile a new kernel we have to download the source code of the Linux kernel. We can download the source from www.kernel.org. Here we can find all versions of the Linux kernel source code. Let's take an example. Suppose we want to compile the 2.6.9 version of the linux kernel. We have to download the 2.6.9 source code from:

http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.9.tar.bz2

It's better to download the bzipped version, as that will be more compressed than its gzipped counterpart; hence will take less time to download. A wget from the command line will look like:

wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.9.tar.bz2

Once we download the required kernel version source, we need to bunzip and untar it. We can do the following:

tar xvjf linux-2.6.9.tar.bz2

The 'x' option is to denote the untarring (e'x'traction), 'v' for verbose, 'j' for specifying that we need to bunzip the file before untarring and 'f' for stating the name of the input file.

The file will untar into the directory linux-2.6.9. Once it's untarred 'cd' to linux-2.6.9.

2. Configuring the kernel

We have to configure the kernel before we start compiling it. During the configuration phase, we will select the components which we want to be part of the kernel. For example: suppose we are using the ext3 filesystem. Then we need to select the ext3 filesystem support while configuring the kernel. Typically we have to run a

make menuconfig

This will bring up the ncurses interface for configuring the kernel. There are other options such as 'make xconfig' and 'make config'. The former will bring up the configuration menu in graphical mode and the latter in text mode.

Once we select the different components we want for our kernel, we can exit the configuration interface. We should select the option to save the configuration from the configuration menu, before exiting.

After we have configured the kernel as mentioned above, we can find a file named '.config' in the top level directory of the source. This file is the configuration file. It contains various options and their states (whether they are selected or not). For example, if we choose to have the PCI support in our kernel we can find an entry of the form:

CONFIG_PCI=y

in the .config file. Similarly, options which are selected as not required will appear as not set. Suppose we have not selected the XFS filesystem support in our kernel we will find the following in the .config

# CONFIG_XFS_FS is not set

A great feature of 2.6 kernels is that if we are running make menuconfig (or xconfig or config) for the first time, then the configuration menu we are presented with is based on our current kernel configuration. In my case, I have a Fedora Core 1 system. The kernel which I run is '2.4.22-1.2115.nptl'. Hence when I run a 'make menuconfig' for the first time on the source then the configuration menu presented will contain the options as given in '/boot/config-2.4.22-1.2115.nptl'.

3. Building Dependencies

This step is required in kernels prior to 2.6 series (here I am only referring to the stable series kernels). For example if we are using a 2.4 kernel then we have to build the dependencies explicitly. We have to run the following:

make dep

This will build the dependencies. But for a 2.6 kernel we can skip this step. The dependencies are automatically created when making the final image with a 2.6 kernel.

4. Creating the final image

We can build various types of kernel binary images. We can build a plain kernel image, or a compressed version of it; the usual choice is compressed, or the 'bzImage'. We can create the bzImage by running

make bzImage

In 2.6 kernels this step will also resolve the dependencies and proceed to create a bzImage image.

After the compilation is over we can find the kernel image at the path arch/i386/boot/bzImage in case of an image for a 386 based processor (Pentium, AMD etc.).

5. Compiling and Installing the modules

In the configuring section if we have selected some components to be built as kernel modules then we need to compile those modules. To compile the modules we should run the command:

make modules

This command will compile the components (which are selected for module compilation) to modules. In a 2.4 kernel the result will be .o files of the corresponding components. But in a 2.6 kernel the output file will be a .ko module. For example if we have given the option for the Network driver of Realtek cards to be built as modules then after giving a 'make modules' we can find in 'driver/net/' a file named 8139too.o in the case of a 2.4 kernel and 8139too.ko in the case of a 2.6 kernel.

After we have compiled the modules, it's time now to install the modules. To install the modules run:

make modules_install

as root. This will install the modules and other necessary files into the /lib/modules/2.6.9 directory.

6. Booting from the new kernel

Once we are done with the installation of modules, we can go for an automatic installation procedure for the kernel binary. We just have to run

make install

This will update the kernel image on to the /boot area, update the configuration file of the bootloader (lilo.conf or grub.conf) and then do the necessary actions to make the new kernel bootable.

After this we need to reboot the machine. When the machine boots next time the boot menu will present us with the option to boot from the new kernel we built. We choose that option and voila!! boot into a kernel we built all by ourselves!

7. Manual installation of the kernel

In case 'make install' does not work, or if we cannot perform an automatic installation due to some other reason, we can go for a manual installation of the kernel. For example, if we are using the grub boot loader then we have to copy the bzImage into the boot partition and then change the '/etc/grub.conf' to reflect the presence of the new image. If we are having lilo boot loader then we have to copy the bzImage to the boot location and then modify the lilo.conf and then run the 'lilo' command to make sure that next time we boot we will have our new image as a choice to boot from. The following are the steps we should perform as root user if we are using lilo boot loader:

 cp -a arch/i386/boot/bzImage /boot/bzImage-2.6.9 

After this we add the following entry to /etc/lilo.conf

image=/boot/bzImage-2.6.9
    label=2.6.9-kernel
    root=your_root_disk

We should run lilo after this

lilo -v

We will reboot the machine after this. When we are prompted at the lilo prompt enter '2.6.9-kernel' as the boot option and we will be booting to the new custom built kernel.

8. Verbose compilation

We find that the compilation of the kernel is very quiet. Much less information on what is getting compiled is shown on the screen while the compilation proceeds.

#make bzImage
CHK     include/linux/version.h
UPD     include/linux/version.h
SPLIT   include/linux/autoconf.h -> include/config/*
CC      scripts/mod/empty.o
HOSTCC  scripts/mod/mk_elfconfig
MKELF   scripts/mod/elfconfig.h
HOSTCC  scripts/mod/file2alias.o
HOSTCC  scripts/mod/modpost.o
HOSTCC  scripts/mod/sumversion.o
....
.... 

If we need to know what commands are used for compilation, then we need to give the verbose compilation option while compiling. That is:

make bzImage V=1

This will output the commands which are executed while compiling. Here is a snippet from the compilation output:

<..snip..>
make -f scripts/Makefile.build obj=init
gcc -Wp,-MD,init/.main.o.d -nostdinc -iwithprefix include -D__KERNEL__ -Iinclude  -Wall
-Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -pipe -msoft-float
-mpreferred-stack-boundary=2  -march=i686 -Iinclude/asm-i386/mach-default -O2
-fomit-frame-pointer     -DKBUILD_BASENAME=main -DKBUILD_MODNAME=main -c -o init/main.o
init/main.c
CHK     include/linux/compile.h
UPD     include/linux/compile.h
gcc -Wp,-MD,init/.version.o.d -nostdinc -iwithprefix include -D__KERNEL__ -Iinclude  -Wall
-Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -pipe -msoft-float
-mpreferred-stack-boundary=2  -march=i686 -Iinclude/asm-i386/mach-default -O2
-fomit-frame-pointer     -DKBUILD_BASENAME=version -DKBUILD_MODNAME=version -c -o
init/version.o init/version.c
<..snip..>

9. Cleaning the kernel source

After we have initiated compilation once on the source if we want to clean the object files and other temporary files then we have to run the following:

make clean

This will remove most generated files but will keep the configuration file.

If we need an absolute cleaning, i.e. if we want to return the source to the state in which it was before we started the compilation, then do a

make mrproper

This command will delete all generated files, the configuration file as well as various backup files. This will in effect unwind all the changes we made to the source. The source after this step will be as good as it was just after the download and untar.

10. Conclusion

We have seen how to obtain the linux kernel source, how to configure it, how to build the kernel image and modules, how to boot from the newly compiled kernel and how to do a verbose compilation. Also we have seen how to clean up the temporary files and configuration files which were created during the compilation. The next step for a budding kernel hacker would be to modify the kernel source and try experimenting with it.

What is a service in linux ?

1. A Linux service is an application (or set of applications) that runs in the background waiting to be used, or carrying out essential tasks.
2. Each service has its corresponding service script, these services being managed by shell scripts so called init scripts or service scripts.
3. Normally the name of the scirpt is same as the name of the daemon which script starts when it is executed, for example smb is service if you start this service it will start the daemons smbd and nmbd.

As i told above when you start a service it starts a daemon with the name of its "script name ", and these services are started by executing there corresponding scripts present in the /etc/ininit.d/ directory.

What is initrd why is it required ?

1. The kernel almost certainly will have been passed an initial RAM disk image (usually called "initrd") by the boot loader.
   
2. This is needed to provide needed device-special files in /dev, as devices are now dymanically created by the udev daemon which is only started during the middle of the boot sequence.
3. /dev will be an empty mount point, missing the needed stdin, stdout, stderr, null, random, and other devices.
4. The initrd may also contain some needed device drivers (e.g., for SCSI controller or SATA chipsets)
5. The initrd is uncompressed into RAM and mounted temporarily as the root of the filesystem (/). Then the file /linuxrc is run, possibly loading those device drivers.

What is the role of an Administrator ?( Linux in perticular )

• With the power of administrator access comes responsibility. In other words, if something goes wrong, it's your butt on the line.
• Administrator tasks: managing users; installing, configuring, and upgrading hardware; installing, configuring, and upgrading software; backups; monitoring system activity (e.g. who's doing what, how is the OS performing); security configuration and monitoring; handling emergencies; contingency planning
• It's an all or nothing job. Don't accept an administrator role unless you are the only person in possession of the root password, or the others with the password are trusted administrators.
• As the administrator, you must always consider the ethical implications of your actions or inaction.
• The number one rule for backups is to test the restore operation frequently.
• Linux has no one recommended backup strategy, but tar does a fine job in most cases.
• Some complex applications, notably databases, require special handling of the backup process because they will lock files, or may have to be shut down to make a proper snapshot of the data.

What is an Operating System?

An operating system (OS) is a resource manager. It takes the form of a set of software routines that allow users and application programs to access system resources (e.g. the CPU, memory, disks, modems, printers network cards etc.) in a safe, efficient and abstract way.

For example, an OS ensures safe access to a printer by allowing only one application program to send data directly to the printer at any one time. An OS encourages efficient use of the CPU by suspending programs that are waiting for I/O operations to complete to make way for programs that can use the CPU more productively. An OS also provides convenient abstractions (such as files rather than disk locations) which isolate application programmers and users from the details of the underlying hardware.

Fig. 1.1: General operating system architecture

Fig. 1.1 presents the architecture of a typical operating system and shows how an OS succeeds in presenting users and application programs with a uniform interface without regard to the details of the underlying hardware. We see that:

• The operating system kernel is in direct control of the underlying hardware. The kernel provides low-level device, memory and processor management functions (e.g. dealing with interrupts from hardware devices, sharing the processor among multiple programs, allocating memory for programs etc.)
• Basic hardware-independent kernel services are exposed to higher-level programs through a library of system calls (e.g. services to create a file, begin execution of a program, or open a logical network connection to another computer).
• Application programs (e.g. word processors, spreadsheets) and system utility programs (simple but useful application programs that come with the operating system, e.g. programs which find text inside a group of files) make use of system calls. Applications and system utilities are launched using a shell (a textual command line interface) or a graphical user interface that provides direct user interaction.

Operating systems (and different flavours of the same operating system) can be distinguished from one another by the system calls, system utilities and user interface they provide, as well as by the resource scheduling policies implemented by the kernel.