My blog has moved!

You should be automatically redirected. If not, visit
http://benohead.com
and update your bookmarks.

Showing posts with label Linux. Show all posts
Showing posts with label Linux. Show all posts

Friday, April 27, 2012

Debian: Configure a static IP address

In order to configure a network device statically, you have to edit /etc/network/interfaces:

# vi /etc/network/interfaces

There you can configure a static IP address and the subnet mask:

auto eth0
iface eth0 inet static
address 192.168.128.128
netmask 255.255.255.0

In order to have the changes enabled, you need to restart the network interface either with:

# /etc/init.d/networking restart
or
# /etc/init.d/networking stop
# /etc/init.d/networking start
or
# ifdown eth0
# ifup eth0

You can then check the new eth0 settings with:

# ifconfig -a

Debian: change the keyboard layout from the console

I've installed a new test system to play around a little bit (a Debian Squeeze install). Since I was too lazy to install it from scratch and I'm using VMWare anyway, I thought I'd just download a pre-installed virtual machine.

Now this virtual machine had a US keyboard configured and I'm using a German keyboard. After googling for how to change the keyboard layout from the console, I found a few hints:

dpkg-reconfigure console-data

Unfortunately the package console-data is not installed on my system.

Then I found the following:

dpkg-reconfigure keyboard-configuration

This package is installed and you can choose a keyboard layout. Great !

After saving, I pressed "Z" and got a "Y" on the console. So still having a US keyboard configured.

Since keyboard-configuration modifies the file /etc/default/keyboard, I checked this file:

# cat /etc/default/keyboard
# Check /usr/share/doc/keyboard-configuration/README.Debian for
# documentation on what to do after having modified this file.
 
# The following variables describe your keyboard and can have the same
# values as the XkbModel, XkbLayout, XkbVariant and XkbOptions options
# in /etc/X11/xorg.conf.

XKBMODEL="pc105"
XKBLAYOUT="de"
XKBVARIANT="nodeadkeys"
XKBOPTIONS=""

# If you don't want to use the XKB layout on the console, you can
# specify an alternative keymap. Make sure it will be accessible
# before /usr is mounted.
# KMAP=/etc/console-setup/defkeymap.kmap.gz

Everything looks fine... But the hint in the beginning of the file might be pointing to the solution. So let's check this README.Debian file:

# cat /usr/share/doc/keyboard-configuration/README.Debian
After modifying /etc/default/keyboard, you can apply the changes to the linux console by running setupcon. If X is configured to use that file too, then the changes will become visible to X only if

udevadm trigger --subsystem-match=input --action=change

There is the solution:

#setupcon

And the keyboard layout is German !!

Linux: get info regarding a process ID

With the following, you can get some information regarding a given process ID. It displays the following:
  • working directory
  • environment variables
  • cpu usage
  • memory usage
  • TCP connections
  • UDP connections
  • List of open file handles

Here an example of the ouput for the rpcbin process:

# processinfo.sh 3822
Information about process 3822

----- Working Directory -----

/

----- Command line -----

/sbin/rpcbind CONSOLE=/dev/console SELINUX_INIT=YES ROOTFS_FSTYPE=ext3 SHELL=/bin/sh TERM=linux ROOTFS_FSCK=0 crashkernel=128M-:64M LC_ALL=POSIX instmode=cd INIT_VERSION=sysvinit-2.86 REDIRECT=/dev/tty1 COLU

----- Environment -----

CONSOLE=/dev/console
SELINUX_INIT=YES
ROOTFS_FSTYPE=ext3
SHELL=/bin/sh
TERM=linux
ROOTFS_FSCK=0
crashkernel=128M-:64M
LC_ALL=POSIX
instmode=cd
INIT_VERSION=sysvinit-2.86
REDIRECT=/dev/tty1
COLUMNS=96
PATH=/bin:/sbin:/usr/bin:/usr/sbin
vga=0x314
DO_CONFIRM=
RUNLEVEL=3
PWD=/
SPLASHCFG=/etc/bootsplash/themes/SLES/config/bootsplash-800x600.cfg
PREVLEVEL=N
LINES=33
HOME=/
SHLVL=2
splash=silent
SPLASH=yes
ROOTFS_BLKDEV=/dev/sda2
_=/sbin/startproc
DAEMON=/sbin/rpcbind

----- Resource usage -----

CPU : 0.0%
MEMORY: 0.0%

----- TCP connections -----

Status: LISTEN local: 0.0.0.0:111 remote: 0.0.0.0:*
Status: LISTEN local: :::111 remote: :::*

----- UDP connections -----

Local: 0.0.0.0:605 remote: 0.0.0.0:*
Local: 0.0.0.0:111 remote: 0.0.0.0:*
Local: :::605 remote: :::*
Local: :::111 remote: :::*

----- Open files -----

COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
rpcbind 3822 root cwd DIR 8,2 4096 2 /
rpcbind 3822 root rtd DIR 8,2 4096 2 /
rpcbind 3822 root txt REG 8,2 56536 123002 /sbin/rpcbind
rpcbind 3822 root mem REG 8,2 61467 466970 /lib64/libnss_files-2.11.1.so
rpcbind 3822 root mem REG 8,2 19114 466959 /lib64/libdl-2.11.1.so
rpcbind 3822 root mem REG 8,2 39712 466992 /lib64/libgssglue.so.1.0.0
rpcbind 3822 root mem REG 8,2 108213 466964 /lib64/libnsl-2.11.1.so
rpcbind 3822 root mem REG 8,2 1661454 466953 /lib64/libc-2.11.1.so
rpcbind 3822 root mem REG 8,2 135646 466979 /lib64/libpthread-2.11.1.so
rpcbind 3822 root mem REG 8,2 160248 467049 /lib64/libtirpc.so.1.0.10
rpcbind 3822 root mem REG 8,2 42016 467021 /lib64/libwrap.so.0.7.6
rpcbind 3822 root mem REG 8,2 149797 466946 /lib64/ld-2.11.1.so
rpcbind 3822 root 0u CHR 1,3 0t0 2372 /dev/null
rpcbind 3822 root 1u CHR 1,3 0t0 2372 /dev/null
rpcbind 3822 root 2u CHR 1,3 0t0 2372 /dev/null
rpcbind 3822 root 3r REG 8,2 0 361286 /var/run/rpcbind.lock
rpcbind 3822 root 4u sock 0,6 0t0 8357 can't identify protocol
rpcbind 3822 root 5u unix 0xffff880233c4a3c0 0t0 8319 /var/run/rpcbind.sock
rpcbind 3822 root 6u IPv4 8321 0t0 UDP *:sunrpc
rpcbind 3822 root 7u IPv4 8325 0t0 UDP *:soap-beep
rpcbind 3822 root 8u IPv4 8326 0t0 TCP *:sunrpc (LISTEN)
rpcbind 3822 root 9u IPv6 8328 0t0 UDP *:sunrpc
rpcbind 3822 root 10u IPv6 8330 0t0 UDP *:soap-beep
rpcbind 3822 root 11u IPv6 8331 0t0 TCP *:sunrpc (LISTEN)

And here's the script. It's not too spectacular, just gathering data from /proc/xxx, ps, netstat and lsof:
 
#!/bin/sh

if [ $# -ne 1 ]; then
echo Usage: $0 PID
exit 1
fi

PID=$1

if kill -0 $PID; then
echo Information about process $PID
echo
echo ----- Working Directory -----
echo
readlink /proc/$PID/cwd
echo
echo ----- Command line -----
echo
ps eho command -p $PID
echo
echo ----- Environment -----
echo
strings -f /proc/$PID/environ | cut -f2 -d ' '
echo
echo ----- Resource usage -----
echo
echo CPU : `ps eho %cpu -p $PID`%
echo MEMORY: `ps eho %mem -p $PID`%
echo
echo ----- TCP connections -----
echo
netstat -pan | grep " $PID/" | grep tcp | awk ' { print "Status: "$6" local: "$4" remote: "$5; }'
echo
echo ----- UDP connections -----
echo
netstat -pan | grep " $PID/" | grep udp | awk ' { print "Local: "$4" remote: "$5; }'
echo
echo ----- Open files -----
echo
lsof -p $PID
else
echo Process $PID does not exist
exit 2
fi

Monday, April 23, 2012

Debian: Remove packages in status rc

The status rc says that the package was removed but the configuration files are still on disk. This happens either when the packages are deleted with apt-get remove or they are automatically removed due to a dependency. In this second case, it also happens if the main package was purged and not only removed.

You can determine which package are in this status with:

# dpkg -l | grep ^rc

So executing this will return only the names of all these packages:

# dpkg -l | grep ^rc | awk '{print $2}'

And to purge all these packages:

# dpkg -l | grep ^rc | awk '{print $2}' | xargs dpkg -P

If you want to first see what dpkg would do, use:

# dpkg -l | grep ^rc | awk '{print $2}' | xargs dpkg --no-act -P

Sunday, April 22, 2012

rkhunter: Warning: The file properties have changed

I've got a script running on a regular basis and executing rkhunter. After I finally got to installing all security updates on our new server, I started getting emails saying:

From: root
Subject: [rkhunter] Warnings found for ...
Date: ...
To: ...
Please inspect this machine, because it may be infected.

And I found the following in /var/log/rkhunter.log:

Warning: The file properties have changed:
File: /usr/bin/ldd
Current inode: xxx Stored inode: xxx
Current file modification time: xxx
Stored file modification time : xxx

(as well as for a few other files).
Now I didn't really check in details which files were all changed by these security updates, so I had to check...

# dpkg -S /usr/bin/ldd
libc-bin: /usr/bin/ldd

OK, so ldd is the package: libc-bin
Now checking whether it was updated on that day

# grep libc-bin /var/log/dpkg.log | grep 2012-04-20
2012-04-20 21:08:43 upgrade libc-bin 2.11.3-2 2.11.3-3
2012-04-20 21:08:43 status half-configured libc-bin 2.11.3-2
2012-04-20 21:08:43 status unpacked libc-bin 2.11.3-2
2012-04-20 21:08:43 status half-installed libc-bin 2.11.3-2
2012-04-20 21:08:44 status half-installed libc-bin 2.11.3-2
2012-04-20 21:08:44 status half-installed libc-bin 2.11.3-2
2012-04-20 21:08:44 status unpacked libc-bin 2.11.3-3
2012-04-20 21:08:44 status unpacked libc-bin 2.11.3-3
2012-04-20 21:08:44 configure libc-bin 2.11.3-3 2.11.3-3
2012-04-20 21:08:44 status unpacked libc-bin 2.11.3-3
2012-04-20 21:08:44 status unpacked libc-bin 2.11.3-3
2012-04-20 21:08:44 status unpacked libc-bin 2.11.3-3
2012-04-20 21:08:45 status unpacked libc-bin 2.11.3-3
2012-04-20 21:08:45 status half-configured libc-bin 2.11.3-3
2012-04-20 21:08:45 status installed libc-bin 2.11.3-3

Yes, it was ! So everything is fine. I just need to let rkhunter know, that it should update its data regarding the files it monitors:

rkhunter --propupd

And the warnings were gone !

Thursday, April 19, 2012

Root kit check as cron job and email notification

First, it's always recommended to not use only one tool checking for root kits but multiples. Here is how to configure cron jobs to check the system everyday and email you the findings (only if something was found).

First rkhunter:

00 23 * * * (/usr/bin/rkhunter --versioncheck; /usr/bin/rkhunter --update; /usr/bin/rkhunter --cronjob --report-warnings-only)
This will first check for latest version of rkhunter (--versioncheck), check for updates to database files (--update) and then run rkhunter.

You can enter email address which should receive a message if something was found in /etc/rkhunter.conf:

MAIL-ON-WARNING=henri.benoit@gmail.com xxx.xxx@googlemail.com

Now chkrootkit:

00 23 * * * /usr/sbin/chkrootkit -q | grep -v "bindshell.*465)" > /tmp/chkrootkit.log && [ -s /tmp/chkrootkit.log ] && mail -s 'chkrootkit findings' henri.benoit@gmail.com xxx.xxx@googlemail.com < /tmp/chkrootkit.log

The grep -v part is only there because chkrootkit keep reporting that port 465 is infected when it is actually just postfix running... Other than that, the crontab line above will scan the system, write a log file and mail the contents of the log file if not empty.

Wednesday, April 18, 2012

Debian Lenny: 404 when using apt-get

Support for Debian 5 (Lenny) has been officially discontined on February, 6th 2012. All Lenny software packages have been removed from Debian mirrors and whenever you use apt-get update or apt-get install to update or install new packages, you get an error message:

W: Failed to fetch <URL> 404 Not Found


It is of course recommended to upgrade to Debian 6 (Squeeze). But if you still want to update packaged for Lenny, you need to update the following in /etc/apt/sources.list:

deb http://ftp.de.debian.org/debian/ lenny main
deb-src
http://ftp.de.debian.org/debian/ lenny main

to:

deb http://ftp.de.debian.org/debian-archive/debian/ lenny main
deb-src
http://ftp.de.debian.org/debian-archive/debian/ lenny main

(replace ftp.de.debian.org by the repository you actually want to use)

After an apt-get update you can install Lenny packages again (although it is really recommended to upgrade to Squeeze).

Tuesday, April 17, 2012

chkrootkit: false positive, bindshell INFECTED Port 465

Just installed chkrootkit on our (relatively) new Debian Squeeze server and run it. Wasn't expecting anything sind rkhunter reported everything was fine. But it did indeed report something:

# chkrootkit
...
Checking `bindshell'... INFECTED (PORTS: 465)
...



So checking what's running on port 465:

# netstat -pan | grep ":465 "
tcp 0 0 0.0.0.0:465 0.0.0.0:* LISTEN 8325/master
tcp6 0 0 :::465 :::* LISTEN 8325/master



Since I wasn't too sure what this master was:

# ps -F -p 8325
UID PID PPID C SZ RSS PSR STIME TTY TIME CMD
root 8325 1 0 9292 2404 6 Apr14 ? 00:00:17 /usr/lib/postfix/master



There it is, it's nothing to worry about, just postfix...

So if a mailserver is running tls on port 465, chkrootkit wrongfully identifies bindshell.

If you stop postfix and rerun chkrootkit, nothing is reported:

# service postfix stop
Stopping Postfix Mail Transport Agent: postfix.
# chkrootkit | grep INFECTED
# service postfix start
Starting Postfix Mail Transport Agent: postfix.

Monday, April 9, 2012

postfix/smtp: certificate verification failed for gmail

We host our emails at gmail and on our new server keep getting the following message in /var/log/mail.err:

Apr 9 21:08:16 xxxxxx postfix/smtp[nnnnn]: certificate verification failed for gmail-smtp-in.l.google.com[173.194.70.27]:25: untrusted issuer /C=US/O=Equifax/OU=Equifax Secure Certificate Authority
I tried downloading and rehashing the certificates, but it didn't make this error disappear until I figured out the solution to this problem was much simpler:

/etc/ssl/certs/ca-certificates.crt (or actually the file referenced by this link) contains all the CA certificates of root CAs and intermediate CA certificates. And all I needed to do was to tell postfix to load this file, by adding the following line to /etc/postfix/main.cf

smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt
And restarting postfix:

# service postfix restart
Stopping Postfix Mail Transport Agent: postfix.
Starting Postfix Mail Transport Agent: postfix.

And when sending a new email, no error message !

Friday, April 6, 2012

No entries in /var/log/messages

Since our monitoring script notified us that one of our domains wasn't responding for a few minutes last time, I checked all files in /var/log on the server hosting this domain. Unfortunately I couldn't find anything since /var/log/messages was actually not being written to.

My first thought was to check whether syslog-ng was running:

# ps aux | grep syslog-ng | grep -v grep
root 915 0.0 0.0 6156 612 ? Ss 22:30 0:00 /sbin/syslog-ng

So it is running.

# /etc/init.d/syslog restart
Shutting down syslog services done
Starting syslog services done

Restarting it also works.

# /etc/init.d/syslog status
Checking for service syslog: running


Everything looks fine... Except that /var/log/messages is not being written to...

Re-running SuSEConfig and restarting it didn't help either:

# SuSEconfig --module syslog-ng
Starting SuSEconfig, the SuSE Configuration Tool...
Running module syslog-ng only
Reading /etc/sysconfig and updating the system...
Executing /sbin/conf.d/SuSEconfig.syslog-ng...
Finished.
# rcsyslog restart
Shutting down syslog services done
Starting syslog services done


Then I though delete /var/log/messages might help:

# /etc/init.d/syslog stop
# rm /var/log/messages
# touch /var/log/messages
# /etc/init.d/syslog start


But no changes... Then I tried to trigger the writing of an entry just to make sure that the problem is not that nobody actually writes log entries which should go to /var/log/messages:

# logger -i "New log entry created to check whether it will really be written to disk"

No error message, no trace of this entry anywhere...

So it has to be something in the configuration... So I open /etc/syslog-ng/syslog-ng.conf.in, search for /var/log/messages and find the following lines:

#
# All messages except iptables and the facilities news and mail:
#
destination messages { file("/var/log/messages"); };
log { source(src); filter(f_messages); filter(f_mailwarn); destination(messages); };


There it is ! "filter(f_mailwarn);" is causing the problem. So I removed it, re-run SuSEConfig, restarted syslog_ng and sent a log message:

# SuSEconfig --module syslog-ng
Starting SuSEconfig, the SuSE Configuration Tool...
Running module syslog-ng only
Reading /etc/sysconfig and updating the system...
Executing /sbin/conf.d/SuSEconfig.syslog-ng...
Finished.
# rcsyslog restart
Shutting down syslog services done
Starting syslog services done
# logger -i "New log entry created to check whether it will really be written to disk"

And there it is:

# grep "New log entry created" /var/log/messages
Apr 6 22:48:53 xxx root[1707]: New log entry created to check whether it will really be written to disk

Wednesday, April 4, 2012

mysql: Backup all databases in separate files

In order to backup all databases on a mysql server at once, you can use the following command:

# mysqldump --all-databases -u xxx --password="xxx" | gzip > full.backup.sql.gz

This will create an sql file with a dump of all databases and compress it. It's nice, fast and easy. But it's then not so easy to work with this one single file (especially if only one database needs to be restored).

But this can also be done with a one-liner (though a long one...):

# echo 'show databases' | mysql -u xxx --password="xxx" --skip-column-names | grep -v information_schema | xargs -I {} -t bash -c 'mysqldump -u xxx --password="xxx" {} | gzip > /backup/mysqldump-$(hostname)-{}-$(date +%Y-%m-%d-%H.%M.%S).sql.gz'

This first gets all databases in the local mysql server, dumps each of them and stores them compressed.

Monday, April 2, 2012

Empty qmail remote queue

A few weeks ago, we had some emails which got stuck in the remote queue of qmail. Our server kept trying to process them again but it never succeeded. After a few days I wanted to just empty the queue to stop seeing in the log files messages about failed delivery.

Unfortunately there seems to be no built-in function in qmail to empty a queue (although I'd love to be proven wrong). So I started looking for a solution how the get rid of these messages.

Since qmhandle was already installed on our server I tried using it to empty the queue. This didn't work on our server but here's how to do it (maybe it will work for you...):

Delete all messages containing a specific string in the subject:
 
#qmHandle -S"subject_substring_to_delete"

Delete all messages in both the local and remote queues:

#qmHandle -D

Since this didn't work, I started looking for other solutions.

There's a tool called Qmail-Remove. Unfortunately I wasn't able to download it and it looks like it's not really maintained anyway...

After some more searching, I finally found out that it's pretty easy to empty the queue without any tool.

You just have to stop qmail with:

/etc/init.d/qmail stop

(if it doesn't work try one of these:

/etc/rc.d/init.d/qmail stop
/etc/rc.d/qmail stop
qmailctl stop
)

Then you just need to delete all files in the subdirectories of /var/qmail/queue/remote (or another path if your queues are stored somewhere else):

find /var/qmail/queue/remote -type f -exec rm {} \;

Instead of "remote" you can enter another subdirectory of "queue" to empty another queue e.g. local.

An then start qmail again:

/etc/init.d/qmail start

That's it. Now you can check the queue status, with one of the following:

# qmHandle -s
# qmail-qstat
# qmail.qread
# qmailctl queue

They should all show that your queue is now empty !

Thursday, March 29, 2012

postfix/smtp: invalid sender domain (misconfigured dns?) (in reply to RCPT TO command)


A few days ago, we switched a newsletter function on our new server and got the following error messages in syslog:

Mar 28 18:17:50 xxxxxx postfix/smtp[29479]: 65EA137AE460:
host xxx.xxx.com[nnn.nnn.nnn.nnn] said: 421 invalid sender domain
'yyy.yyy.com' (misconfigured dns?) (in reply to RCPT TO command)


Mar 28 18:17:50 xxxxxx postfix/smtp[29479]: 65EA137AE460:
to=<zzz@zzzzzz.com>,
relay=xxx.xxx.com[nnn.nnn.nnn.nnn]:25, delay=0.51,
delays=0/0/0.26/0.24, dsn=4.0.0, status=deferred (host
xxx.xxx.com[nnn.nnn.nnn.nnn] said: 421 invalid sender domain
'yyy.yyy.com' (misconfigured dns?) (in reply to RCPT TO
command))

The problem was that when our server was installed the mail name (in /etc/mailname) was set to some defaults (in our case debian.vaultronserver.de) which doesn't match our hostname or any DNS entry related to our server. It should actually have been the fully qualified name of our server.


The problem was quickly solved by updating /etc/mailname:

hostname -A > /etc/mailname

And restarting postfix:

/etc/init.d/postfix restart

Script to monitor DNS entries

This summary is not available. Please click here to view the post.

Reverse lookup

This summary is not available. Please click here to view the post.

Friday, March 23, 2012

Match context using grep / awk

With GNU grep you can use the -B <number> to set how many lines before the match and -A <number> for the number of lines after the match.

grep -B 1 -A 5 search_pattern filename

If you want the same number of lines before and after you can use -C <number> e.g.:

grep -C 2 search_pattern filename

This will show 2 lines before and 2 lines after the matched line.

or just -n (for n lines of context). -2 is quicker to type than -A 2 -B 2

if want to show all lines of output after the match? awk can do this:

awk '/search_pattern/,0' filename

On solaris, grep doesn't support any of these paramerters so you have to use awk:

nawk 'c-->0;$0~s{if(b)for(c=b+1;c>1;c--)print r[(NR-c+1)%b];print;c=a}b{r[NR%b]=$0}' b=2 a=4 s="search_pattern" filename

...where "b" and "a" are the number of lines to print before and after string "s".

Thursday, March 15, 2012

How do you find the environment for a running process?

Linux


cat /proc/<pid>/environ


Even though /proc/<pid>/environ has a size of 0, running a cat will still return the environment for this process. This file is just a kind of link to the actual location where the enviroment for this process is stored.

The ouput of the command above is not very readable since all strings are separated by NUL and not newlines. To convert them to newlines, you can use:

strings -f /proc/<pid>/environ


or


(cat /proc/<PID>/environ; echo) | tr "\000" "\n"


or


tr '\0' '\n' < /proc/<pid>/environ


or


xargs --null --max-args=1 echo < /proc/<pid>/environ


or


cat /proc/<pid>/environ | perl -pne '$a=chr(0);s/$a/\n/g'


Note: you can get the pids of a process by using 
$ pidof httpd


This return a space separated list of the PIDs of all httpd processes.


Mac OS X


ps -p <pid> -wwwE


AIX


ps eww <pid>


Solaris


pargs -e <pid>


HP-UX


The only way is to attach gdb to the process and examine _environ.


Windows


On Windows, you can do this by using a tool such as ProcessExplorer to select a particular process and view the values of the environment variables.

You could also write a small .NET program using:
System.Diagnostics.Process.GetProcessById(<pid>).StartInfo.EnvironmentVariables

Sunday, November 20, 2011

Remove Byte-Order-Mark

Problems always occur when UTF-8 encoded script files or XML files are opened and edited. After saving the file the following characters are suddenly at the beginning of the file:
  ï »¿

These characters are called Byte Order Mark (BOM) and mark the text as UTF-8 encoded. The characters are usually inserted automatically by editors. The downside is that the BOM sometimes creates problems in Java or PHP applications.


It's relatively easy to remove these characters from PHP or XML files under Linux.


The first solution is using vim:


Open the file in vim and type the following:
:set nobomb :w
This will remove the BOM and save the file.


An alternative solution without vim uses sed (you could also use tr instead):
cp inputfile.xml inputfile.xml.tmp
cat inputfile.xml.tmp | sed 's/^\xEF\xBB\xBF//' > inputfile.xml
rm inputfile.xml.tmp
This will copy the file, replace the three BOM characters by the empty string (i.e. removing them) and copying it back to the original file (and deleting the temporary file in the end).


That's it !



Thursday, November 17, 2011

Tar

tar (Tape ARchiver) is a Linux command to archive files and directories. It can be used to create an archive file and optionally compress it (using gzip or bzip2)

Usage

Create an archive file
tar cvf archive.tar directory/
The c option will have tar create a new archive file. The following additional options are usually also use:
    v verbose output (list files processed)
    f followed by the path to the archive file to be created (if omitted, the standard output is used - which only makes sense if you redirect it)

The last argument is a list of files/directories to be archived e.g. tar cvg archive.tar dir1 dir2 file3 file4

The files are added to the archive including the directory structure.

Additionally the archive file can be compressed:

Using gzip:

tar cvzf archive.tar.gz directory/
tar cvzf archive.tgz directory/
(.tgz is an extension often used instead of .tar.gz).

Using bzip2:
tar cvjf archive.tar.bz2 directory/

Update an archive file
tar uvf archive.tar directory/
With the u option, tar will update the referenced archive file adding (or updating if already contained) the provided files.

Extract an archive file
tar xvf archive.tar  
The x option will have tar extract the contents of the referenced archive file to the current directory.

List the contents of an archive file
tar tvf archive.tar
The t option will have tar list the files contained in the referenced archive file.

Alternatively, less can also be used to list the contents of a TAR archive:
less archive.tar

Tips

Archive on floppy
 
tar cMvf /dev/fd0 /pfad
/dev/fd0 is the device of the floppy drive.
The M option tells tar to create a multi-volume archive. This means that if the archive is too big for a single floppy it will be split to multiple floppies.

Similarly, you can use the other options above instead of c (to update, list or extract).

Get the list of files to archive from a file
 
tar cvf archive.tar -T tar.list
where tar.list contains the list of files to be added to the archive.

Archive only a subset of the files 
find . -name "*.log" -print | tar cvf archive.tar -T-
This will look recursively for all *.log files and add them to the archive.

-T- tells tar not to use a file as input but the standard input (which in this case is a redirect of find's output).

Exclude invidual files 
tar cvf archive.tar -X exclude.list directory/
The -X option is followed by the path to a path containing a list of files which should not be included in the archive.

Alternatively a pattern can be used to exclude files:
tar cvf archive.tar --exclude=*.log directory/  
This will archive all files and subdirectory in "directory" except log files.