What is syslog ?, Why use a log file to record happenings on systems?

  1. Basically syslog is a system V initialization script present in /etc/rc.d/init.d/
  2. When this script is executed it starts and manages two daemons syslogd and klogd
  3. Linux applications use the syslog utility to export all their errors and status messages to files located in the /var/log directory, these files are termed as log files.
  4. log files give the information about system activities, will be most help full during troubleshooting the system problems.
  5. By looking atthe log files one can know the system state just before the problem occured to diagnose the problem and come to about intrusion activities.
Log file examples :
/var/log/dmesg: kernel boot messages
/var/log/messages:standard system error messages
/var/log/maillog:Mail system mesages
/var/log/secure:Security, authentication, and xinetd messages
/var/log/audit/audit.log:kernel auditing messages

There are various other log files that store info from other appln like ( Apache, Squid etc)

Why use a log file to record happenings on systems?


If you are being notified of login attempts and failures, system errors and possible security problems then you will find that:

  • A log will keep track of what is happening on your system at all times.

  • It will alert you to problems before they arise, for example; if your partition is becoming full or if there is an impending attack.

  • The problem would be seen prior to your "healthy" backups being cycled out.

Compiling the Linux Kernel

This post will serve as a primer to people who are new to the world of Linux hacking, and are attempting to compile the Linux kernel from source. The various steps from downloading the kernel source to booting from the new kernel image are explained. Also given are tips on cleaning up the source code, doing verbose compilation etc.


1. Downloading the kernel source code

In order to compile a new kernel we have to download the source code of the Linux kernel. We can download the source from www.kernel.org. Here we can find all versions of the Linux kernel source code. Let's take an example. Suppose we want to compile the 2.6.9 version of the linux kernel. We have to download the 2.6.9 source code from:

http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.9.tar.bz2

It's better to download the bzipped version, as that will be more compressed than its gzipped counterpart; hence will take less time to download. A wget from the command line will look like:

wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.9.tar.bz2

Once we download the required kernel version source, we need to bunzip and untar it. We can do the following:

tar xvjf linux-2.6.9.tar.bz2

The 'x' option is to denote the untarring (e'x'traction), 'v' for verbose, 'j' for specifying that we need to bunzip the file before untarring and 'f' for stating the name of the input file.

The file will untar into the directory linux-2.6.9. Once it's untarred 'cd' to linux-2.6.9.

2. Configuring the kernel

We have to configure the kernel before we start compiling it. During the configuration phase, we will select the components which we want to be part of the kernel. For example: suppose we are using the ext3 filesystem. Then we need to select the ext3 filesystem support while configuring the kernel. Typically we have to run a
make menuconfig
This will bring up the ncurses interface for configuring the kernel. There are other options such as 'make xconfig' and 'make config'. The former will bring up the configuration menu in graphical mode and the latter in text mode.

Once we select the different components we want for our kernel, we can exit the configuration interface. We should select the option to save the configuration from the configuration menu, before exiting.

After we have configured the kernel as mentioned above, we can find a file named '.config' in the top level directory of the source. This file is the configuration file. It contains various options and their states (whether they are selected or not). For example, if we choose to have the PCI support in our kernel we can find an entry of the form:

CONFIG_PCI=y
in the .config file. Similarly, options which are selected as not required will appear as not set. Suppose we have not selected the XFS filesystem support in our kernel we will find the following in the .config
# CONFIG_XFS_FS is not set

A great feature of 2.6 kernels is that if we are running make menuconfig (or xconfig or config) for the first time, then the configuration menu we are presented with is based on our current kernel configuration. In my case, I have a Fedora Core 1 system. The kernel which I run is '2.4.22-1.2115.nptl'. Hence when I run a 'make menuconfig' for the first time on the source then the configuration menu presented will contain the options as given in '/boot/config-2.4.22-1.2115.nptl'.

3. Building Dependencies

This step is required in kernels prior to 2.6 series (here I am only referring to the stable series kernels). For example if we are using a 2.4 kernel then we have to build the dependencies explicitly. We have to run the following:
make dep
This will build the dependencies. But for a 2.6 kernel we can skip this step. The dependencies are automatically created when making the final image with a 2.6 kernel.

4. Creating the final image

We can build various types of kernel binary images. We can build a plain kernel image, or a compressed version of it; the usual choice is compressed, or the 'bzImage'. We can create the bzImage by running
make bzImage
In 2.6 kernels this step will also resolve the dependencies and proceed to create a bzImage image.

After the compilation is over we can find the kernel image at the path arch/i386/boot/bzImage in case of an image for a 386 based processor (Pentium, AMD etc.).

5. Compiling and Installing the modules

In the configuring section if we have selected some components to be built as kernel modules then we need to compile those modules. To compile the modules we should run the command:
make modules
This command will compile the components (which are selected for module compilation) to modules. In a 2.4 kernel the result will be .o files of the corresponding components. But in a 2.6 kernel the output file will be a .ko module. For example if we have given the option for the Network driver of Realtek cards to be built as modules then after giving a 'make modules' we can find in 'driver/net/' a file named 8139too.o in the case of a 2.4 kernel and 8139too.ko in the case of a 2.6 kernel.

After we have compiled the modules, it's time now to install the modules. To install the modules run:

make modules_install
as root. This will install the modules and other necessary files into the /lib/modules/2.6.9 directory.

6. Booting from the new kernel

Once we are done with the installation of modules, we can go for an automatic installation procedure for the kernel binary. We just have to run
make install
This will update the kernel image on to the /boot area, update the configuration file of the bootloader (lilo.conf or grub.conf) and then do the necessary actions to make the new kernel bootable.

After this we need to reboot the machine. When the machine boots next time the boot menu will present us with the option to boot from the new kernel we built. We choose that option and voila!! boot into a kernel we built all by ourselves!

7. Manual installation of the kernel

In case 'make install' does not work, or if we cannot perform an automatic installation due to some other reason, we can go for a manual installation of the kernel. For example, if we are using the grub boot loader then we have to copy the bzImage into the boot partition and then change the '/etc/grub.conf' to reflect the presence of the new image. If we are having lilo boot loader then we have to copy the bzImage to the boot location and then modify the lilo.conf and then run the 'lilo' command to make sure that next time we boot we will have our new image as a choice to boot from. The following are the steps we should perform as root user if we are using lilo boot loader:
 cp -a arch/i386/boot/bzImage /boot/bzImage-2.6.9 
After this we add the following entry to /etc/lilo.conf
image=/boot/bzImage-2.6.9
label=2.6.9-kernel
root=your_root_disk
We should run lilo after this
lilo -v
We will reboot the machine after this. When we are prompted at the lilo prompt enter '2.6.9-kernel' as the boot option and we will be booting to the new custom built kernel.

8. Verbose compilation

We find that the compilation of the kernel is very quiet. Much less information on what is getting compiled is shown on the screen while the compilation proceeds.
#make bzImage
CHK include/linux/version.h
UPD include/linux/version.h
SPLIT include/linux/autoconf.h -> include/config/*
CC scripts/mod/empty.o
HOSTCC scripts/mod/mk_elfconfig
MKELF scripts/mod/elfconfig.h
HOSTCC scripts/mod/file2alias.o
HOSTCC scripts/mod/modpost.o
HOSTCC scripts/mod/sumversion.o
....
....

If we need to know what commands are used for compilation, then we need to give the verbose compilation option while compiling. That is:
make bzImage V=1
This will output the commands which are executed while compiling. Here is a snippet from the compilation output:
<..snip..>
make -f scripts/Makefile.build obj=init
gcc -Wp,-MD,init/.main.o.d -nostdinc -iwithprefix include -D__KERNEL__ -Iinclude -Wall
-Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -pipe -msoft-float
-mpreferred-stack-boundary=2 -march=i686 -Iinclude/asm-i386/mach-default -O2
-fomit-frame-pointer -DKBUILD_BASENAME=main -DKBUILD_MODNAME=main -c -o init/main.o
init/main.c
CHK include/linux/compile.h
UPD include/linux/compile.h
gcc -Wp,-MD,init/.version.o.d -nostdinc -iwithprefix include -D__KERNEL__ -Iinclude -Wall
-Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -pipe -msoft-float
-mpreferred-stack-boundary=2 -march=i686 -Iinclude/asm-i386/mach-default -O2
-fomit-frame-pointer -DKBUILD_BASENAME=version -DKBUILD_MODNAME=version -c -o
init/version.o init/version.c
<..snip..>

9. Cleaning the kernel source

After we have initiated compilation once on the source if we want to clean the object files and other temporary files then we have to run the following:
make clean
This will remove most generated files but will keep the configuration file.

If we need an absolute cleaning, i.e. if we want to return the source to the state in which it was before we started the compilation, then do a

make mrproper
This command will delete all generated files, the configuration file as well as various backup files. This will in effect unwind all the changes we made to the source. The source after this step will be as good as it was just after the download and untar.

10. Conclusion

We have seen how to obtain the linux kernel source, how to configure it, how to build the kernel image and modules, how to boot from the newly compiled kernel and how to do a verbose compilation. Also we have seen how to clean up the temporary files and configuration files which were created during the compilation. The next step for a budding kernel hacker would be to modify the kernel source and try experimenting with it.

What is a service in linux ?

  1. A Linux service is an application (or set of applications) that runs in the background waiting to be used, or carrying out essential tasks.
  2. Each service has its corresponding service script, these services being managed by shell scripts so called init scripts or service scripts.
  3. Normally the name of the scirpt is same as the name of the daemon which script starts when it is executed, for example smb is service if you start this service it will start the daemons smbd and nmbd.
As i told above when you start a service it starts a daemon with the name of its "script name ", and these services are started by executing there corresponding scripts present in the /etc/ininit.d/ directory.

What is initrd why is it required ?

  1. The kernel almost certainly will have been passed an initial RAM disk image (usually called "initrd") by the boot loader.
  2. This is needed to provide needed device-special files in /dev, as devices are now dymanically created by the udev daemon which is only started during the middle of the boot sequence.
  3. /dev will be an empty mount point, missing the needed stdin, stdout, stderr, null, random, and other devices.
  4. The initrd may also contain some needed device drivers (e.g., for SCSI controller or SATA chipsets)
  5. The initrd is uncompressed into RAM and mounted temporarily as the root of the filesystem (/). Then the file /linuxrc is run, possibly loading those device drivers.

What is the role of an Administrator ?( Linux in perticular )

  • With the power of administrator access comes responsibility. In other words, if something goes wrong, it's your butt on the line.
  • Administrator tasks: managing users; installing, configuring, and upgrading hardware; installing, configuring, and upgrading software; backups; monitoring system activity (e.g. who's doing what, how is the OS performing); security configuration and monitoring; handling emergencies; contingency planning
  • It's an all or nothing job. Don't accept an administrator role unless you are the only person in possession of the root password, or the others with the password are trusted administrators.
  • As the administrator, you must always consider the ethical implications of your actions or inaction.
  • The number one rule for backups is to test the restore operation frequently.
  • Linux has no one recommended backup strategy, but tar does a fine job in most cases.
  • Some complex applications, notably databases, require special handling of the backup process because they will lock files, or may have to be shut down to make a proper snapshot of the data.

What the kernel does when it starts up ?

  1. The kernel un-compresses

  2. The kernel claims a certain amount of memory for its working tables and buffers (kernel memory).

  3. All the driver that are built into the kernel initialize by detecting their respective hardware

  4. The kernel mounts the root file system The kernel mounts the root directory of the root file system to the kernels' idea of a system (superstructure) root directory.

  5. The kernel executes /sbin/init

The kernel now waits for asynchronous events to occur; i.e. It is now ready to serve the system by servicing requests from processes and hardware.

Everything that happens subsequently on the system has to either be requested by a binary through a system call to the kernel, or an asynchronous event triggered by hardware

Basic Yum Commands and how to use them

This is not an exhaustive list of all yum commands but it is a list of the basic/common/important ones. For a complete list see the yum man page.

yum list [available|installed|extras|updates|obsoletes|all|recent] [pkgspec]

This command lets you list packages in any repository enabled on your system or installed. It also lets you list specific types of packages as well as refine your list with a package specification of any of the package's name, arch, version, release, epoch.

yum list

By default 'yum list' without any options will list all packages in all the repositories and all the packages installed on your system. Note: 'yum list all' and 'yum list' give the same output.

yum list available

Lists all the packages available to be installed in any enabled repository on your system.

yum list installed

This is equivalent to rpm -qa. It lists all the packages installed on the system.

yum list extras

This command lists any installed package which no longer appears in any of your enabled repositories. Useful for finding packages which linger between upgrades or things installed not from a repo.

yum list obsoletes

This command lists any obsoleting relationships between any available package and any installed package.

yum list updates

This command lists any package in an enabled repository which is an update for any installed package.

yum list recent

This command lists any package added to any enabled repository in the last seven(7) days.

yum list pkgspec

This command allows you to refine your listing for particular packages.

Examples of pkgspecs:

yum list zsh
yum list joe\*
yum list \*.i386
yum list dovecot-1.0.15

yum install/remove/update

....

yum check-update

Exactly like yum list updates but returns an exit code of 100 if there are updates available. Handy for shell scripting.

yum grouplist
yum groupinfo
yum groupinstall
yum groupupdate
yum groupremove

Please see the YumGroups page on this wiki for information about the above commands.

yum info

This displays more information about any package installed or available. It takes the same arguments as yum list but it is best run with a specific package name or glob. Example:

$ yum info yum
Installed Packages
Name : yum
Arch : noarch
Version : 3.2.20
Release : 3.fc10
Size : 2.5 M
Repo : installed
Summary : RPM installer/updater
URL : http://yum.baseurl.org/
License : GPLv2+
Description: Yum is a utility that can check for and automatically download and
: install updated RPM packages. Dependencies are obtained and downloaded
: automatically prompting the user as necessary.

yum search

This allows you to search for information from the various metadata available about packages. It can accept multiple arguments. It will output the packages which match the most terms first followed by the next highest number of matches, etc. Specifically yum search looks at the following fields: name, summary, description, url. If you're searching for what package provides a certain command try yum provides instead.

Search example:

$ yum search python rsync ssh
========================= Matched: python, rsync, ssh ==========================
rdiff-backup.i386 : Convenient and transparent local/remote incremental
: mirror/backup

============================ Matched: python, rsync ============================
cobbler.noarch : Boot server configurator

============================= Matched: python, ssh =============================
denyhosts.noarch : A script to help thwart ssh server attacks
pexpect.noarch : Pure Python Expect-like module
python-paramiko.noarch : A SSH2 protocol library for python
python-twisted-conch.i386 : Twisted SSHv2 implementation

============================= Matched: rsync, ssh ==============================
duplicity.i386 : Encrypted bandwidth-efficient backup using rsync algorithm
pssh.noarch : Parallel SSH tools

yum provides/yum whatprovides

This command searches for which packages provide the requested dependency of file. This also takes wildcards for files. Examples:

$ yum provides MTA
2:postfix-2.5.5-1.fc10.i386 : Postfix Mail Transport Agent
Matched from:
Other : MTA

exim-4.69-7.fc10.i386 : The exim mail transfer agent
Matched from:
Other : MTA

sendmail-8.14.3-1.fc10.i386 : A widely used Mail Transport Agent (MTA)
Matched from:
Other : Provides-match: MTA


$ yum provides \*bin/ls
coreutils-6.12-17.fc10.i386 : The GNU core utilities: a set of tools commonly
: used in shell scripts
Matched from:
Filename : /bin/ls


yum shell

....

yum makecache

Is used to download and make usable all the metadata for the currently enabled yum repos. This is useful if you want to make sure the cache is fully current with all metadata before continuing.

yum clean

During its normal use yum creates a cache of metadata and packages. This cache can take up a lot of space. The yum clean command allows you to clean up these files. All the files yum clean will act on are normally stored in /var/cache/yum.

Example commands and what they do:

yum clean packages

This cleans up any cached packages in any enabled repository cache directory.

yum clean metadata

This cleans up any xml metadata that may have been cached from any enabled repository.

yum clean dbcache

Yum will create or download some sqlite database files as part of its normal operation. This command clean up the cached copies of those from any enabled repository cache.

yum clean all

Clean all cached files from any enabled repository. Useful to run from time to time to make sure there is nothing using unnecessary space.

What is "Kernel memory" and "User memory".?

Memory is divided into two areas, kernel memory and user memory

  1. Kernel memory is also known as kmem, kernel space and kernel land

    1. This contains the kernel binary itself, working tables to keep track of status on the system and buffers.

    2. The kernel binary is typically 3 meg big and the working tables consume another 3 meg (only an example; may vary)

    3. Examples of working tables that the kernel keeps in kernel memory for the operation of the system are the Global Open File Table, the Process Table and the Mount Table.

    4. Traditionally once the size of kernel memory has been set on bootup (as determined by the finite set sizes of all the tables) it cannot be resized (System V). Linux has a clever way of overcoming this limitation by allowing kernel tables to grow into user memory as required!

  2. User memory is also known as umem, user space and user land.

    1. This is for the use of user programs.

Figure 1.11. Kernel memory and User Memory

Kernel memory and User Memory

Figure 1.12. Kernel Memory, table and buffer cache allocations

Kernel Memory, table and buffer cache allocations



See How kernel sits in the memory

The boot loader loads the kernel binary into memory from the hard disk, and places it at the beginning of memory.

Figure 1.10. Kernel loaded into memory

Kernel loaded into memory

Once the kernel has been read in the boot loader tells the CPU to execute it by issuing a JMP (Jump) instruction. The kernel now begins to execute


What are the purposes of the C- Libraries:?

Libraries fulfill two purposes:

  1. They save duplication of the same code that would otherwise be contained duplicated inside many binaries individually; this saves disk space.

  2. Very often library calls provide an abstraction interface to similar system calls to simplify programming tasks for a programmer.

An example for number 1 above:

The three binary programs ping, traceroute and telnet all support a command line argument that is a hostname of a computer on the Internet to connect to. However these programs must format packets sent to IP addresses not hostnames.

Somehow these programs must convert a hostname to an IP address and instead of all three containing the same code they all call the same library call in the standard C library called gethostbyname(). Where they supply the requested hostname that was read from the command line ion the parenthesis of the call.

The C library now does the hard work of issuing multiple system calls to connect to a DNS server on the Internet, send a namelookup query, interpret a response and return the resolved IP address to the calling program (ping, traceroute or telnet).

What is an Operating System?

An operating system (OS) is a resource manager. It takes the form of a set of software routines that allow users and application programs to access system resources (e.g. the CPU, memory, disks, modems, printers network cards etc.) in a safe, efficient and abstract way.

For example, an OS ensures safe access to a printer by allowing only one application program to send data directly to the printer at any one time. An OS encourages efficient use of the CPU by suspending programs that are waiting for I/O operations to complete to make way for programs that can use the CPU more productively. An OS also provides convenient abstractions (such as files rather than disk locations) which isolate application programmers and users from the details of the underlying hardware.


Fig. 1.1: General operating system architecture

Fig. 1.1 presents the architecture of a typical operating system and shows how an OS succeeds in presenting users and application programs with a uniform interface without regard to the details of the underlying hardware. We see that:

  • The operating system kernel is in direct control of the underlying hardware. The kernel provides low-level device, memory and processor management functions (e.g. dealing with interrupts from hardware devices, sharing the processor among multiple programs, allocating memory for programs etc.)
  • Basic hardware-independent kernel services are exposed to higher-level programs through a library of system calls (e.g. services to create a file, begin execution of a program, or open a logical network connection to another computer).
  • Application programs (e.g. word processors, spreadsheets) and system utility programs (simple but useful application programs that come with the operating system, e.g. programs which find text inside a group of files) make use of system calls. Applications and system utilities are launched using a shell (a textual command line interface) or a graphical user interface that provides direct user interaction.
Operating systems (and different flavours of the same operating system) can be distinguished from one another by the system calls, system utilities and user interface they provide, as well as by the resource scheduling policies implemented by the kernel.

How to break the GRUB password?

1.Reboot your machine with "linux rescue CD " in it.
2.Typr "linux rscue ask method "
3.vim /etc/grub.conf
4.comment a line which is beginning with the word "password"
5.reboot, your GRUB pasword is removed now
Just follow the steps below
#grub
grub>md5crypt
Password: ******
Encrypted: $1$jxcdN0$hVHViq1aiPf8FziuGJGZp0

/*Copy down encrypted password: */
$1$jxcdN0$hVHViq1aiPf8FziuGJGZp0
/*quit grub now by entering following command*/
grub> quit
#vim /etc/grub.conf
Insert encrypted password in between "splashimage..." and "title...": 

Save edited fileand quit
:wq
#reboot
---done------
Now u cant edit the grub menu with to enter and modify u need to giive the grub password by pressing P