Welcome to Linux Forums! With a comprehensive Linux Forum, information on various types of Linux software and many Linux Reviews articles, we have all the knowledge you need a click away, or accessible via our knowledgeable members.
Find the answer to your Linux question:
Site Navigation
Linux Forums
Linux Articles
Product Showcase
Linux Downloads
Linux Hosting
Free Magazines
Job Board
IRC Chat
RSS Feeds
Free Publications


In the age of easy to install, easier to use GNU/Linux distributions, the shell is the worlds best kept secret for beginners.
Yes I mean that -usually- green on black screen with unholly glow, the cursor blinking, ever demanding, ever asking you what now, giving no clue about your options.

Why so? Because, a master calls it a butterfly, eventhough a caterpillar calls it death.

This seemingly unfriendly text screen and the technology beyond is capable of doing almost anything on Linux. In fact this is one of the oldest user interfaces there is and surprisingly still the most powerful.

What we are talking about is named as shell in the Unix world. There are lots of shell versions like, bash, ash, csh, tcsh etc. You may have one or more of these on your Unix installation. They may differ in syntax and commands but basically do the same thing; lets you interact with your computer with its own set of rules. There are many ways to access the shell. The standard Linux text console is attached to a shell, so when you login at the text screen you actually start using the shell, or if you are using a GUI like KDE, Gnome or Icewm most probably you are using a terminal program like konsole, xterm, rxvt, aterm. Do not mix this with the shell. These terminals are just guis that run a shell in the background. Whatever you type on this gui is processed by the shell.

The most popular shell in all Unix platforms is BASH (Bourne again shell). If you are using a Linux most probably you have BASH as default, as you would have CShell if were using FreeBSD. Most of this article won't be spesific to a certain shell, but I prefer BASH and all my examples are tried on BASH. Thus it will be better if you use BASH.

Today, I'll show you how powerful the regular Unix commands are if they are combined together with a shell. After this tutorial, I'm sure you will appreciate the console lots more.
 
Lets see what we can do with the CLI (command line interface) and bash, the shell that interprets our inputs.

We can first start with showing some file on the screen like;

cat somefile.txt

And if that file is more then one screen long we can show it seperated into pages like;

cat somefile.txt|more

even better,

cat somefile.txt|less

This basic technique is called piping, meaning that your computer pipes the output of the former command to the next one. cat reads your text file and outputs it as ascii letters to the standart output (screen if else is not stated) and less shows you this output page by page.

Though this is one of the most frequently used techniques (paginating an output of a command via less) you can use almost all commands with others, suiting to your needs.

For example, you can use wc (wordcount) to count the lines on a document

cat somefile.txt | wc -l

and see how many lines long that document is.

You can also use wc -l somefile.txt if the input you want is an already there text file, but we will soon see that is not always the case.

Lets say that you have this e-mail list as follows;

ayn@mydomain.cot
ronald@darktower.cot
eddie@three.cot
ayn@mydomain.cot
tera@yourdomain.cot
eddie@three.cot
tera@yourdomain.cot

and for the sake of the example, assume that this is some very long list, with lots of lines like this. At a quick glance we can easily see that there are concurrent lines, with the same name. What we want is, the exact number of unique e-mail adresses in this list. So here we build our pipeline;

cat maillist.txt|sort|uniq

What this commandline does is, sending the content of the file maillist.txt to sort command, which sorts the file so that the concurrent records come one after eachother and then uniq command takes this input and lets only one of each kind in the output, giving us

ayn@mydomain.cot
eddie@three.cot
ronald@darktower.cot
tera@yourdomain.cot

which is exactly what we have asked for.

If we were not interested in the list but wanted to know just the number of unique e-mails, that would cost us one more pipe;


cat maillist.txt|sort|uniq|wc -l

And you have the number of unique e-mail adresses.

Lets say you want to have the unique domains, don't care about the adresses. No big deal, still easy to do;

cat maillist.txt|cut -d "@" -f 2|sort|uniq

Now we have another player, cut. Cut choses fields from the line. You can distinct them with a delimiter character (like we did -d "@") or can state a character position in the line.

To understand the example better try

cat maillist.txt|cut -d "@" -f 2
first.

If you want to have the list of these domains in a file, a small modification will suffice like;

Now you directed the list not to the standart output (which is the screen) but to a file.

cat maillist.txt|cut -d "@" -f 2 > domainlist.txt
As you see, CLI is there to make the things easier for you, IF you know what you are doing. If you don't know what you are doing, it is better not to use it.

For a last clue, lets look at the basic programming abilities of bash.

Lets say you want to do something with every file in your directory. It is very easy to  do with bash;

for i in `ls ` ; do echo $i  ; done

The format is very easy to understand, yet very powerful;

with "for i in `ls`" you get the listing of all files in the directory, one file at a line, and feed them one by one to the $i variable. Now you can do anything with the clause "do command $i" and finish the loop with done. Whatever you write instead of command will be executed with filenames as arguments.

For example you can list the files contained in all tar.bz2 archives, in the current directory with
for i in `ls *.tar.bz2 `; do tar -jtvf $i  ; done

Here, I've given you a few examples of possible usages of the bash and the console. Don't forget, if you want to automate a task or do something fast, commandline should be your first option.
Rate This Article: poorexcellent
 
Comments about this article
UUoC
writen by: Random unix guy on 2006-03-06 23:39:27
Look it up...
RE: UUoC written by Random unix guy:
Handy Technique
writen by: libdave on 2006-03-07 10:14:16
One technique I find very handy with the shell esp when devloping a script is to write the thing to print to stdout (as opposed to execute the commands). Then when I run it I can see the commands that will be executed; when I am satisfied that the script will do what I see on the screen I pipe it to the shell: ./myscript | sh For example (trivial): === 8< === for i in *.txt do echo rm $i done === 8< === Also if you inadvertently click on the file (if you are using a GUI it won't do anything unintended.
RE: Handy Technique written by libdave:
Not too bad an introduction...
writen by: Matthew Berg on 2006-03-07 10:30:19
... but there's a lot of unnecessary use of "cat" though; e.g. "cat maillist.txt | less" instead of "less maillist.txt", or "cat maillist.txt|sort|uniq|wc -l" instead of "sort maillist.txt | uniq | wc -l" or even "sort -u maillist.txt | wc -l"
RE: Not too bad an introduction... written by Matthew Berg:
args
writen by: nyet on 2006-05-08 16:04:09
do *NOT* hide the command line arguments for the n00bs. You don't need cat !!!
RE: args written by nyet:
Yeah
writen by: cat on 2006-06-06 23:42:52
Yeah, don't let the cat of the bag unnecessarily!
RE: Yeah written by cat:
pipeline
writen by: tanyear on 2006-07-28 22:30:15
the writer use cat at the beginning just to show how pipeline works. in my opinion, use uniq before sort and other command could be better
RE: pipeline written by tanyear:
Careful...
writen by: Nicholas on 2007-09-13 22:28:34
RE: Careful... written by Nicholas:
renaming multiple files in shell
writen by: balusss on 2007-10-13 02:52:27
hi i thought this could be useful to some of you. you can rename multiple files in one go like changing all particular extensions: see this.... [b]for i in `ls *.abc` ;do x=`echo $i | cut -d "." -f 1 `; mv $i $x.xyz ; done [/b] the above line just changes all *.abc files to *.xyz. hope this helps just in case :-) and this article....simple & encouraging...appreciate the efforts of the author. thanks to him.
RE: renaming multiple files in shell written by balusss:
Part 2: Bash substitution
writen by: linuxpaul on 2007-10-26 12:53:14
RE: Part 2: Bash substitution written by linuxpaul:
Even shorter
writen by: Philip on 2007-12-12 13:46:29
RE: Even shorter written by Philip:

Comment title: * please do not put your response text here