Usually, when people mention "shell scripting language," what comes to mind is bash, ksh, sh, or other similar Linux/Unix scripting languages. A scripting language is another way to communicate with the computer. With a graphical window interface (whether Windows or Linux doesn't matter), the user can move the mouse and click on various objects, such as buttons, lists, checkboxes, etc. But this method is very inconvenient every time the user wants the computer/server to perform the same task (for example, batch converting photos, or downloading new movies, MP3s, etc.). To make all these things simple and automated, we can use shell scripts.
Some programming languages, like Pascal, FoxPro, C, Java, etc., need to be compiled before execution. They require an appropriate compiler to let our code accomplish a certain task.
Other programming languages, such as PHP, JavaScript, Visual Basic, etc., do not need a compiler; instead, they require an interpreter, and we can run programs without compiling the code.
Shell scripts are also like interpreters, but they are usually used to call external compiled programs. Then, they capture the output results, exit codes, and handle them according to the situation.
One of the most popular shell scripting languages in the Linux world is bash. And I think (this is my own opinion) the reason is that, by default, the bash shell allows users to conveniently navigate through history commands (previously executed ones). In contrast, ksh requires some adjustments to .profile, or remembering some "magic" key combinations to review history and correct commands.
Alright, I think this introduction is enough. As for which environment suits you best, I'll leave it to you to decide. From now on, I will only talk about bash and its scripts. In the following examples, I will use CentOS 6.6 and bash-4.1.2. Please make sure you have the same version or a higher one.
Shell Script Streams
The shell scripting language is similar to chatting with several people. You just need to imagine all commands as people who can help you do things, as long as you ask them in the right way. For example, you want to write a document. First, you need paper. Then, you need to tell someone the content so they can write it for you. Finally, you want to store it somewhere. Or, say you want to build a house, so you need to hire the right people to clear the site. After they say "the job is done," other engineers can help you build the walls. Finally, when these engineers also tell you "the job is done," you can call the painters to decorate the house. What happens if you ask the painters to decorate before the walls are built? I think they will start complaining. Almost all of these human-like commands will talk. If they finish the job without any problems, they will tell "standard output." If they cannot do what you asked them to do—they will tell "standard error." So, ultimately, all commands listen to you through "standard input."
Quick example—when you open a Linux terminal and type some text—you are talking to bash through "standard input." So, let's ask the bash shellwho am i (Who am I?)then.
root@localhost ~]# who am i <--- 你通过标准输入对 bash shell 说 root pts/0 2015-04-22 20:17 (192.168.1.123) <--- bash shell通过标准输出回答你
Now, let's ask bash something it doesn't understand:
[root@localhost ~]# blablabla <--- 哈，你又在和标准输入说话了 -bash: blablabla: command not found <--- bash通过标准错误在发牢骚了
The first word before the ':' is usually the command complaining to you. In fact, each of these streams has its own index number (LCTT translator's note: file handle number):
- Standard input (stdin） – 0
- Standard output (stdout） – 1
- Standard error (stderr） – 2
If you really want to know what a command's output said, you need to redirect that speech to a file (using the greater-than sign ">" and the stream index after the command):
[root@localhost ~]# blablabla 1> output.txt -bash: blablabla: command not found
In this example, we try to redirect stream 1 (stdout) to a file named output.txt. Let's see what was done to the file contents; you can do this with the cat command:
[root@localhost ~]# cat output.txt [root@localhost ~]#
It seems to be empty. Well, now let's redirect stream 2 (stderr）：
[root@localhost ~]# blablabla 2> error.txt [root@localhost ~]#
Well, we see the complaints are gone. Let's check that file:
[root@localhost ~]# cat error.txt -bash: blablabla: command not found [root@localhost ~]#
Sure enough! We see that all the complaints were recorded in the errors.txt file.
Sometimes, a command produces bothstdoutandstderr. To redirect them to different files, we can use the following statement:
command 1>out.txt 2>err.txt
To shorten the statement a bit, we can omit "1", because by defaultstdoutwill be redirected:
command >out.txt 2>err.txt
Well, let's try doing something "bad." Let's use the rm command to delete file1 and folder1:
[root@localhost ~]# rm -vf folder1 file1 > out.txt 2>err.txt
Now let's check the following output files:
[root@localhost ~]# cat out.txt removed `file1' [root@localhost ~]# cat err.txt rm: cannot remove `folder1': Is a directory [root@localhost ~]#
As we can see, the different streams were separated into different files. Sometimes this is not very convenient, because we want to see what happened consecutively before or after a certain operation when an error occurs. To achieve this, we can redirect both streams to the same file:
command >>out_err.txt 2>>out_err.txt
Note: Please note that I used ">>" instead of ">". It allows us to append to the file rather than overwrite it.
We can also redirect one stream to another:
command >out_err.txt 2>&1
Let me explain. The standard output of all commands will be redirected to out_err.txt, the error output will be redirected to stream 1 (as explained above), and that stream will be redirected to the same file. Let's look at this example:
[root@localhost ~]# rm -fv folder2 file2 >out_err.txt 2>&1 [root@localhost ~]# cat out_err.txt rm: cannot remove `folder2': Is a directory removed `file2' [root@localhost ~]#
Looking at the combined output, we can describe it as: first,rmthe command tries to delete folder2, but it won't succeed because Linux requires the-rkey to allowrmcommand to delete the folder, while the second file2 will be deleted. By providing thermprovide-v(verbose) key, we make the rm command tell us every deleted file or folder.
This is almost everything you need to know about redirection. I say almost, because there is one more important redirection tool, which is called a "pipe." By using the | (pipe) symbol, we usually redirect thestdoutstream.
For example, we have a text file like this:
[root@localhost ~]# cat text_file.txt This line does not contain H e l l o word This lilne contains Hello This also containd Hello This one no due to HELLO all capital Hello bash world!
And we need to find the lines containing "Hello". Linux has agrepcommand that can do this job:
[root@localhost ~]# grep Hello text_file.txt This lilne contains Hello This also containd Hello Hello bash world! [root@localhost ~]#
When we have a file and want to search in it, this works well. But what if we need to find something in the output of another command? Yes, of course, we can redirect the output to a file, and then search in the file:
[root@localhost ~]# fdisk -l>fdisk.out [root@localhost ~]# grep "Disk /dev" fdisk.out Disk /dev/sda: 8589 MB, 8589934592 bytes Disk /dev/mapper/VolGroup-lv_root: 7205 MB, 7205814272 bytes Disk /dev/mapper/VolGroup-lv_swap: 855 MB, 855638016 bytes [root@localhost ~]#
What if you plan to grep something quoted with double quotes and containing spaces!
Note: The fdisk command displays information about disk drives on Linux operating systems.
As we can see, this method is very inconvenient, because we soon mess up the temporary file space. To accomplish this task, we can use pipes. They allow us to redirect one command'sstdoutto another command'sstdinstream:
[root@localhost ~]# fdisk -l | grep "Disk /dev" Disk /dev/sda: 8589 MB, 8589934592 bytes Disk /dev/mapper/VolGroup-lv_root: 7205 MB, 7205814272 bytes Disk /dev/mapper/VolGroup-lv_swap: 855 MB, 855638016 bytes [root@localhost ~]#
As you can see, we get the same result without needing any temporary files. We redirect thefdisk stdoutredirected togrep stdin。
Note: Pipe redirection is always from left to right.
There are a few other redirection types, but we will cover them later.
Display Custom Information in the Shell
As we know, communication with the shell and within the shell is usually carried out in a conversational manner. So, let's create some real scripts that will also talk to us. This will teach you a few simple commands and give you a better understanding of the concept of scripts.
Suppose we are the manager of a company's service desk, and we want to create a shell script to register call information: phone number, username, and a brief description of the problem. We plan to store this information in a plain text file, data.txt, for future statistics. The script itself works in a conversational manner, which will make the service desk staff's lives a bit easier. So, first we need to display prompts. For displaying information, we can use the echo and printf commands. Both are used to display information, but printf is more powerful because we can format the output nicely with it; we can make it right-aligned, left-aligned, or leave dedicated space for information. Let's start with a simple example. To create a file, use your usual text editor (kate, nano, vi, ...), and create a file named note.sh, and write these commands into it:
echo "Phone number ?"
How to Run/Execute a Script?
After saving the file, we can run it with the bash command, passing our file as its argument:
[root@localhost ~]# bash note.sh Phone number ?
Actually, executing the script this way is quite inconvenient. If we don't use thebashcommand as a prefix, it would be more comfortable to execute. To make the script executable, we can use thechmodcommand:
[root@localhost ~]# ls -la note.sh -rw-r--r--. 1 root root 22 Apr 23 20:52 note.sh [root@localhost ~]# chmod +x note.sh [root@localhost ~]# ls -la note.sh -rwxr-xr-x. 1 root root 22 Apr 23 20:52 note.sh [root@localhost ~]#
Note: The ls command displays the files in the current folder. By adding the -la key, it shows more file information.
As we can see, before thechmodcommand is executed, the script only has read (r) and write (w) permissions. After executingchmod +xit gains the execute (x) permission. (I will cover more details about permissions in the next article.) Now, we just run it like this:
[root@localhost ~]# ./note.sh Phone number ?
Before the script name, I added the ./ combination. . (dot) in the Unix world means the current location (current folder), and / (slash) is the folder separator. (In Windows systems, we use backslash \ to indicate the same function) So this whole combination means: "Execute the note.sh script from the current folder". I think it would be clearer if I ran this script with the full path:
[root@localhost ~]# /root/note.sh Phone number ? [root@localhost ~]#
It also works.
If all Linux users had the same default shell, everything would be fine. If we just execute the script, the default user shell will be used to parse the script content and run the commands. Different shells have slight differences in syntax, internal commands, and so on. So, to ensure our script usesbash, we should add#!/bin/bashto the first line of the file. This way, the default user shell will invoke/bin/bash, and only then will the commands in the script be executed:
[root@localhost ~]# cat note.sh #!/bin/bash echo "Phone number ?"
Only now are we 100% surebashthat it will be used to parse our script content. Let's continue.
Reading Input
After displaying the message, the script waits for the user's answer. There is areadcommand used to receive the user's answer:
#!/bin/bash echo "Phone number ?" read phone
After executing, the script waits for user input until the user presses [ENTER] to end input:
[root@localhost ~]# ./note.sh Phone number ? 12345 <--- 这儿是我输入的内容 [root@localhost ~]#
Everything you enter will be stored in the variablephone; to display the variable's value, we can also use theechocommand:
[root@localhost ~]# cat note.sh #!/bin/bash echo "Phone number ?" read phone echo "You have entered $phone as a phone number" [root@localhost ~]# ./note.sh Phone number ? 123456 You have entered 123456 as a phone number [root@localhost ~]#
InbashIn shell, generally we use the$(dollar) sign to indicate that this is a variable, except when reading into a variable and in a few other rare cases where we don't use this $ (will be explained later).
Alright, now we are ready to add the remaining questions:
#!/bin/bash echo "Phone number?" read phone echo "Name?" read name echo "Issue?" read issue [root@localhost ~]# ./note.sh Phone number? 123 Name? Jim Issue? script is not working. [root@localhost ~]#
Using Stream Redirection
Perfect! What remains is to redirect everything to the file data.txt. As the field separator, we will use the / (slash) symbol.
Note: You can choose any delimiter you think is best, but make sure the file content does not contain these symbols, otherwise it will cause extra fields in the text lines.
Don't forget to use ">>" instead of ">", because we want to append the output to the end of the file!
[root@localhost ~]# tail -2 note.sh read issue echo "$phone/$name/$issue">>data.txt [root@localhost ~]# ./note.sh Phone number? 987 Name? Jimmy Issue? Keybord issue. [root@localhost ~]# cat data.txt 987/Jimmy/Keybord issue. [root@localhost ~]#
Note ： tailcommand displays the lastnlines.
Done. Let's run it again to see:
[root@localhost ~]# ./note.sh Phone number? 556 Name? Janine Issue? Mouse was broken. [root@localhost ~]# cat data.txt 987/Jimmy/Keybord issue. 556/Janine/Mouse was broken. [root@localhost ~]#
Our file is growing. Let's add a date at the beginning of each line. This will be useful when fiddling with these statistics later. To achieve this, we can use the date command and specify a certain format, because I don't like the default format:
[root@localhost ~]# date Thu Apr 23 21:33:14 EEST 2015 <---- date命令的默认输出 [root@localhost ~]# date "+%Y.%m.%d %H:%M:%S" 2015.04.23 21:33:18 <---- 格式化后的输出
There are several ways to read a command's output into a variable; in this simple case, we will use ` (backtick, not single quote, on the same key as the tilde ~):
[root@localhost ~]# cat note.sh #!/bin/bash now=`date "+%Y.%m.%d %H:%M:%S"` echo "Phone number?" read phone echo "Name?" read name echo "Issue?" read issue echo "$now/$phone/$name/$issue">>data.txt [root@localhost ~]# ./note.sh Phone number? 123 Name? Jim Issue? Script hanging. [root@localhost ~]# cat data.txt 2015.04.23 21:38:56/123/Jim/Script hanging. [root@localhost ~]#
Hmm... our script looks a bit ugly. Let's beautify it. If you want to manually readreadcommand, you will find that the read command can also display some information. To achieve this, we should use the -p option plus the message:
[root@localhost ~]# cat note.sh #!/bin/bash now=`date "+%Y.%m.%d %H:%M:%S"` read -p "Phone number: " phone read -p "Name: " name read -p "Issue: " issue echo "$now/$phone/$name/$issue">>data.txt
You can directly find a lot of interesting information about various commands from the console, just type:man read, man echo, man date, man ……
Agree? It looks much more comfortable!
[root@localhost ~]# ./note.sh Phone number: 321 Name: Susane Issue: Mouse was stolen [root@localhost ~]# cat data.txt 2015.04.23 21:38:56/123/Jim/Script hanging. 2015.04.23 21:43:50/321/Susane/Mouse was stolen [root@localhost ~]#
The cursor is after the message (not on a new line), which is interesting. (LCTT translator's note: If using the echo command for output display, you can use the -n option to avoid a newline.)
Loops
It's time to improve our script. If the user is answering phone calls all day, wouldn't it be troublesome to have to run it every time? Let's make these activities loop endlessly:
[root@localhost ~]# cat note.sh
#!/bin/bash
while true
do
read -p "Phone number: " phone
now=`date "+%Y.%m.%d %H:%M:%S"`
read -p "Name: " name
read -p "Issue: " issue
echo "$now/$phone/$name/$issue">>data.txt
done
I have swapped theread phoneandnow=datepositions of the lines. This is because I want to get the time after entering the phone number. If I put it at the first line of the loop, then after one loop, the variable now would get the time immediately after the data is stored to the file. And this is not good, because the next call might be 20 minutes later, or even later.
[root@localhost ~]# ./note.sh Phone number: 123 Name: Jim Issue: Script still not works. Phone number: 777 Name: Daniel Issue: I broke my monitor Phone number: ^C [root@localhost ~]# cat data.txt 2015.04.23 21:38:56/123/Jim/Script hanging. 2015.04.23 21:43:50/321/Susane/Mouse was stolen 2015.04.23 21:47:55/123/Jim/Script still not works. 2015.04.23 21:48:16/777/Daniel/I broke my monitor [root@localhost ~]#
Note: To exit the infinite loop, you can press [Ctrl]+[C]. The shell will display ^ to represent the CTRL key.。
Using Pipe Redirection
Let's add more features to our "Frankenstein". I want the script to display some statistics after each call. For example, I want to see how many times each number called me. For this, we should cat the file data.txt:
[root@localhost ~]# cat data.txt 2015.04.23 21:38:56/123/Jim/Script hanging. 2015.04.23 21:43:50/321/Susane/Mouse was stolen 2015.04.23 21:47:55/123/Jim/Script still not works. 2015.04.23 21:48:16/777/Daniel/I broke my monitor 2015.04.23 22:02:14/123/Jimmy/New script also not working!!! [root@localhost ~]#
Now, we can redirect all output tocutcommand, lettingcutit cut each line into pieces (we use the delimiter "/"), then print the second field:
[root@localhost ~]# cat data.txt | cut -d"/" -f2 123 321 123 777 123 [root@localhost ~]#
Now, we can redirect this output to another commandsort：
[root@localhost ~]# cat data.txt | cut -d"/" -f2|sort 123 123 123 321 777 [root@localhost ~]#
Then only unique lines remain. To count unique entries, just add-coption touniqcommand:
[root@localhost ~]# cat data.txt | cut -d"/" -f2 | sort | uniq -c
3 123
1 321
1 777
[root@localhost ~]#
Just add this to the end of our loop:
#!/bin/bash
while true
do
read -p "Phone number: " phone
now=`date "+%Y.%m.%d %H:%M:%S"`
read -p "Name: " name
read -p "Issue: " issue
echo "$now/$phone/$name/$issue">>data.txt
echo "===== We got calls from ====="
cat data.txt | cut -d"/" -f2 | sort | uniq -c
echo "--------------------------------"
done
Run:
[root@localhost ~]# ./note.sh
Phone number: 454
Name: Malini
Issue: Windows license expired.
===== We got calls from =====
3 123
1 321
1 454
1 777
--------------------------------
Phone number: ^C
The current scenario runs through several well-known steps:
- Display message
- Get user input
- Store value to file
- Process stored data
But, if the user is a bit responsible, he sometimes needs to enter data, sometimes needs statistics, or maybe wants to look up something in the stored data? For these things, we need to use switches/cases, and know how to format output nicely. This is useful when "drawing" tables in the shell.
Source: http://linux.cn/article-5591-1.html