home // Programming // Shell Programming
Advanced Topics in Shell Scripting Awk, sed and grep.
Source: www.securitydocs.com
File Size: 75.74 KB
Category: Programming
Last Download : 1 day 17 hours 17 minutes ago
Share this info:
Short Description: Jul 28, 2005 ... To make a shell script do CGI all you ... completely relevant if you want to write CGI based shell scripts, it is however rather centred on ...
Content Inside: Advanced Topics in Shell Scripting Page 1 of 7 Advanced Topics in Shell Scripting by: Colin Sauze, 07/28/2005 http://www.securitydocs.com/library/3489 This article focuses on more advanced topics in shel scripting including a number of common utilities which can help you write more versatile shel scripts. Awk, sed and grep. You wil often here these 3 tools talked about together. They are very useful for getting the output format of other programs into a format which is useful to you in a shel script. Awk is mainly used for splitting up text based around what is cal ed a field separator (e.g. In "a:B:c:d" : is the field separator). Sed is used to alter the contents of a file or the output of a program in some way, its often used to remove unwanted output or to alter files. Grep is used to match patterns within its input or a file. Sed and grep both use a pattern matching language cal ed regular expressions, more wil be explained about these later on and additional references are available at the end of this tutorial. Awk Awk's main use is to split up text into separate fields al owing you to get hold of just part of the input to a script or the output of another program. Awk is actual y capable of doing far more than this, whole books exist about awk and this tutorial cannot hope to ful y explain it so wil focus on field separation as its main use. By default awk wil split its input using spaces to separate each field, the main way awk gets its input is from a pipe from another program although it can read files itself. As an example here we are going to use some text from the output of the ifconfig program in Linux which displays information about the network cards in a machine. We're just going to use the first line of the output here which is "eth0 Link encap:Ethernet HWaddr 00:50:BE:DD:76:CD". Now suppose we want to find out what the network interface's name is (its eth0) in order to compare it to a variable we already have in our shel script (perhaps something we got from a user's parameter). If we type "echo "eth0 Link encap:Ethernet HWaddr 00:50:BF:D7:77:CF" | awk '{print $1}'" we wil get eth0 as our output. This is because awk is using spaces as field separators and is printing the first field ($1). Do not confuse $1 in awk with $1 in the shel script, they are different things. If we wanted to get the next word using space as our separator "awk '{print $2}'" would return link, "awk '{print $3}'" would return "encap:Ethernet" and so on. Now suppose we want only the first 2 digits of the MAC address( the 00:50:BE:DD:76:CD bit), this can be done by specifying a : as the field separator. Field separators are specified by using the -F option to awk, separators can be a single character or several characters. So to get the first 2 digits of the MAC address we must first get the whole address from the original text. This can be done by getting field number 5 from the original input which wil return 00:50:BE:DD:76:CD. We can now send this output to another awk process with a pipe in order to separate it out using a ":", e.g. Echo "eth0 Link encap:Ethernet HWaddr 00:50:BE:DD:76:CD" | awk '{print $5}' | awk -F: '{print $1}'" this wil now return 00. Sed Sed is used to change the output of a program or file in some way. Sed is very useful for removing unwanted output or parts of output as wel as actual y making changes to files like configuration files. As with awk, we are general y going to want to use sed by piping input into it, although it can be told to read from a file. Sed works by taking an input line enclosed either by ' marks or by " marks (it doesn't matter which). The first thing after this is the choice of command, this is a one letter command. The most common command is probably s which means substitute, another common one is delete which wil delete whole lines. This must be fol owed by a / which is then fol owed by the first argument to this sed command, if the command has another argument it is then fol owed by another / etc. The substitute command takes the format s/searchtext/replacementtext/ If we wanted to change the text "the quick brown fox jumps over the lazy dog" to "the quick brown ox jumps over the lazy frog" http://www.securitydocs.com/link.php?action=detail&id=3489&headerfooter=no 7/28/2005
