In the age of easy to install, easier to use GNU/Linux distributions, the shell is the worlds best kept secret for beginners. Yes I mean that -usually- green on black screen with unholly glow, the cursor blinking, ever demanding, ever asking you what now, giving no clue about your options.

Why so? Because, a master calls it a butterfly, eventhough a caterpillar calls it death.

This seemingly unfriendly text screen and the technology beyond is capable of doing almost anything on Linux. In fact this is one of the oldest user interfaces there is and surprisingly still the most powerful.

What we are talking about is named as shell in the Unix world. There are lots of shell versions like, bash, ash, csh, tcsh etc. You may have one or more of these on your Unix installation. They may differ in syntax and commands but basically do the same thing; lets you interact with your computer with its own set of rules. There are many ways to access the shell. The standard Linux text console is attached to a shell, so when you login at the text screen you actually start using the shell, or if you are using a GUI like KDE, Gnome or Icewm most probably you are using a terminal program like konsole, xterm, rxvt, aterm. Do not mix this with the shell. These terminals are just guis that run a shell in the background. Whatever you type on this gui is processed by the shell.

The most popular shell in all Unix platforms is BASH (Bourne again shell). If you are using a Linux most probably you have BASH as default, as you would have CShell if were using FreeBSD. Most of this article won't be spesific to a certain shell, but I prefer BASH and all my examples are tried on BASH. Thus it will be better if you use BASH.

Today, I'll show you how powerful the regular Unix commands are if they are combined together with a shell. After this tutorial, I'm sure you will appreciate the console lots more.
 
Lets see what we can do with the CLI (command line interface) and bash, the shell that interprets our inputs.

We can first start with showing some file on the screen like;

cat somefile.txt

And if that file is more then one screen long we can show it seperated into pages like;

cat somefile.txt|more

even better,

cat somefile.txt|less

This basic technique is called piping, meaning that your computer pipes the output of the former command to the next one. cat reads your text file and outputs it as ascii letters to the standart output (screen if else is not stated) and less shows you this output page by page.

Though this is one of the most frequently used techniques (paginating an output of a command via less) you can use almost all commands with others, suiting to your needs.

For example, you can use wc (wordcount) to count the lines on a document

cat somefile.txt | wc -l

and see how many lines long that document is.

You can also use wc -l somefile.txt if the input you want is an already there text file, but we will soon see that is not always the case.

Lets say that you have this e-mail list as follows;

ayn@mydomain.cot
ronald@darktower.cot
eddie@three.cot
ayn@mydomain.cot
tera@yourdomain.cot
eddie@three.cot
tera@yourdomain.cot

and for the sake of the example, assume that this is some very long list, with lots of lines like this. At a quick glance we can easily see that there are concurrent lines, with the same name. What we want is, the exact number of unique e-mail adresses in this list. So here we build our pipeline;

cat maillist.txt|sort|uniq

What this commandline does is, sending the content of the file maillist.txt to sort command, which sorts the file so that the concurrent records come one after eachother and then uniq command takes this input and lets only one of each kind in the output, giving us

ayn@mydomain.cot
eddie@three.cot
ronald@darktower.cot
tera@yourdomain.cot

which is exactly what we have asked for.

If we were not interested in the list but wanted to know just the number of unique e-mails, that would cost us one more pipe;

cat maillist.txt|sort|uniq|wc -l

And you have the number of unique e-mail adresses.

Lets say you want to have the unique domains, don't care about the adresses. No big deal, still easy to do;

cat maillist.txt|cut -d "@" -f 2|sort|uniq

Now we have another player, cut. Cut choses fields from the line. You can distinct them with a delimiter character (like we did -d "@") or can state a character position in the line.

To understand the example better try

cat maillist.txt|cut -d "@" -f 2

first.

If you want to have the list of these domains in a file, a small modification will suffice like;

Now you directed the list not to the standart output (which is the screen) but to a file.

cat maillist.txt|cut -d "@" -f 2 > domainlist.txt
As you see, CLI is there to make the things easier for you, IF you know what you are doing. If you don't know what you are doing, it is better not to use it.

For a last clue, lets look at the basic programming abilities of bash.

Lets say you want to do something with every file in your directory. It is very easy to  do with bash;

for i in `ls ` ; do echo $i  ; done

The format is very easy to understand, yet very powerful;

with "for i in `ls`" you get the listing of all files in the directory, one file at a line, and feed them one by one to the $i variable. Now you can do anything with the clause "do command $i" and finish the loop with done. Whatever you write instead of command will be executed with filenames as arguments.

For example you can list the files contained in all tar.bz2 archives, in the current directory with
for i in `ls *.tar.bz2 `; do tar -jtvf $i  ; done

Here, I've given you a few examples of possible usages of the bash and the console. Don't forget, if you want to automate a task or do something fast, commandline should be your first option.