Linux Shell Scripting Cookbook

Solve real-world shell scripting problems with over 110 simple but incredibly effective recipes

Sarath Lakshman

Lectura muy recomendada, sobre las cosas que podemos hacer con bash. Pongo unos ejemplos que he destacado:

Modificar la extensión de los archivos de JPG a jpg

rename *.JPG *.jpg

Moficar según una expresión regular

&lt;/pre&gt;
&lt;pre&gt;rename 's/ /_/g' *&lt;/pre&gt;
&lt;pre&gt;

Buscar resultados en un diccionario

&lt;/pre&gt;
&lt;pre&gt;#!/bin/bash
#Filename: aspellcheck.sh
word=$1
output=`echo \"$word\" | aspell list`
if [ -z $output ]; then
echo $word is a dictionary word;
else
echo $word is not a dictionary word;
fi&lt;/pre&gt;
&lt;pre&gt;

Presentar la frecuencia de las palabras de un archivo

&lt;/pre&gt;
&lt;pre&gt;#!/bin/bash
#Name: word_freq.sh
#Description: Find out frequency of words in a file
if [ $# -ne 1 ];
then
echo "Usage: $0 filename";
exit -1
fi
filename=$1
egrep -o "\b[[:alpha:]]+\b" $filename | \
awk '{ count[$0]++ }
END{ printf("%-14s%s\n","Word","Count") ;
for(ind in count)
{ printf("%-14s%d\n",ind,count[ind]); }
}'&lt;/pre&gt;
&lt;pre&gt;

Matched string notation (&)

In sed we can use & as the matched string for the substitution pattern such that we can use the matched string in replacement string. For example:

&lt;/pre&gt;
&lt;pre&gt;$ echo this is an example | sed 's/\w\+/[&amp;]/g'&lt;/pre&gt;
&lt;pre&gt;

[this] [is] [an] [example] Here the regex \w\+ matches every word. Then we replace it with [&]. & corresponds to the word that is matched.

Substring match notation (\1)

& is a string which corresponds to match string for the given pattern. But we can also match the substrings of the given pattern. Let’s see how to do it.

&lt;/pre&gt;
&lt;pre&gt;$ echo this is digit 7 in a number | sed 's/digit \([0-9]\)/\1/'&lt;/pre&gt;
&lt;pre&gt;

this is 7 in a number It replaces digit 7 with 7. The substring matched is 7. $pattern$is used to match the substring. The pattern is enclosed in () and is escaped with slashes. For the first substring match, the corresponding notation is \1, for the second it is \2, and so on. Go through the following example with multiple matches: $ echo seven EIGHT | sed ‘s/$[a-z]\+$ $[A-Z]\+$/\2 \1/’ EIGHT seven ([a-z]\+\) matches the first word and $[A-Z]\+$matches the second word. \1 and \2 are used for referencing them. This type of referencing is called back referencing. In the replacement part, their order is changed as \2 \1 and hence it appears in reverse order.

Awk avanzado

&lt;/pre&gt;
&lt;pre&gt;$ seq 5 | awk 'BEGIN{ sum=0; print "Summation:" }
{ print $1"+"; sum+=$1 } END { print "=="; print sum }'&lt;/pre&gt;
&lt;pre&gt;

Passing a variable value from outside to awk By using the -v argument, we can pass external values (other than from stdin) to awk as follows:

&lt;/pre&gt;
&lt;pre&gt;$ VAR=10000
$ echo | awk -v VARIABLE=$VAR'{ print VARIABLE }'&lt;/pre&gt;
&lt;pre&gt;

1 There is a flexible alternate method to pass many variable values from outside awk. For example:

&lt;/pre&gt;
&lt;pre&gt;$ var1="Variable1" ; var2="Variable2"
$ echo | awk '{ print v1,v2 }' v1=$var1 v2=$var2&lt;/pre&gt;
&lt;pre&gt;

Variable1 Variable2 When input is given through a file rather than standard input, use:

&lt;/pre&gt;
&lt;pre&gt;$ awk '{ print v1,v2 }' v1=$var1 v2=$var2 filename&lt;/pre&gt;
&lt;pre&gt;

In the above method, variables are specified as key-value pairs separated by space (v1=$var1 v2=$var2) as command arguments to awk soon after the BEGIN, { } and END blocks.

Compressing or decompressing JavaScript

&lt;/pre&gt;
&lt;pre&gt;$ catsample.js |
tr -d '\n\t' |
\
tr -s ' ' \
| sed 's:/\*.*\*/::g' \
| sed 's/ \?\([{}();,:]\) \?/\1/g'&lt;/pre&gt;
&lt;pre&gt;

The output is as follows:

functionsign_out(){$("#loading").show();$.get("log_
in",{logout:"True"},function(){window.location="";});}

Let’s write a decompression script for making obfuscated code readable as follows:

$ cat obfuscated.txt | sed 's/;/;\n/g; s/{/{\n\n/g; s/}/\n\n}/g'

Acceder a gmail desde linea de comandos

&lt;/pre&gt;
&lt;pre&gt;#!/bin/bash
Filename: fetch_gmail.sh
#Description: Fetch gmail tool
username="PUT_USERNAME_HERE"
password="PUT_PASSWORD_HERE"
SHOW_COUNT=5 # No of recent unread mails to be shown
echo
curl -u $username:$password --silent "https://mail.google.com/mail/
feed/atom" | \
tr -d '\n' | sed 's::\n:g' |\
sed 's/.*\(.*\)&lt;\/title.*\([^&lt;]*\)&lt;\/
name&gt;
\([^&lt;]*\).*/Author: \2 [\3] \nSubject: \1\n/' | \
head -n $(( $SHOW_COUNT * 3 ))&lt;/pre&gt;
&lt;pre&gt;

Bajar todas las imágenes de una pagina web

&lt;/pre&gt;
&lt;pre&gt;#!/bin/bash
#Description: Images downloader
#Filename: img_downloader.sh
if [ $# -ne 3 ];
then
echo "Usage: $0 URL -d DIRECTORY"
exit -1
fi
for i in {1..4}
do
case $1 in
-d) shift; directory=$1; shift ;;
*) url=${url:-$1}; shift;;
esac
done
mkdir -p $directory;
baseurl=$(echo $url | egrep -o "https?://[a-z.]+")
curl –s $url | egrep -o "&lt;img src="[^" alt="" /&gt;]*&gt;" |
sed 's/&lt;img src="\&amp;quot;\([^&amp;quot;]*\).*/\1/g'" alt="" /&gt; /tmp/$$.list
sed -i "s|^/|$baseurl/|" /tmp/$$.list
cd $directory;
while read filename;
do
curl –s -O "$filename" --silent
done &lt; /tmp/$$.list&lt;/pre&gt;
&lt;pre&gt;

Twiter desde consola

&lt;/pre&gt;
&lt;pre&gt;#!/bin/bash
#Filename: tweets.sh
#Description: Basic twitter client
USERNAME="PUT_USERNAME_HERE"
PASSWORD="PUT_PASSWORD_HERE"
COUNT="PUT_NO_OF_TWEETS"
if [[ "$1" != "read" ]] &amp;&amp; [[ "$1" != "tweet" ]];
then
echo -e "Usage: $0 send status_message\n
OR\n
exit -1;
fi
if [[ "$1" = "read" ]];
then
curl --silent -u $USERNAME:$PASSWORD
friends_timeline.rss | \
grep title | \
tail -n +2 | \
head -n $COUNT | \
sed 's:.*\([^&lt;]*\).*:\n\1:'
$0 read\n"
http://twitter.com/statuses/
elif [[ "$1" = "tweet" ]];
then
status=$( echo $@ | tr -d '"' | sed 's/.*tweet //')
curl --silent -u $USERNAME:$PASSWORD -d status="$status" http://
twitter.com/statuses/update.xml &gt; /dev/null
echo 'Tweeted :)'
fi&lt;/pre&gt;
&lt;pre&gt;

define: de google desde consola

&lt;/pre&gt;
&lt;pre&gt;#!/bin/bash
#Filename: define.sh
#Description: A Google define: frontend
limit=0
if [ ! $# -ge 1 ];
then
echo -e "Usage: $0 WORD [-n No_of_definitions]\n"
exit -1;
fi
if [ "$2" = "-n" ];
then
limit=$3;
let limit++
fi
word=$1
lynx -dump http://www.google.co.in/search?q=define:$word | \
awk '/Defini/,/Find defini/' | head -n -1 | sed 's:*:\n*:; s:^[ ]*::'
| \
grep -v "[[0-9]]" | \
awk '{
if ( substr($0,1,1) == "*" )
{ sub("*",++count".") } ;
print
} ' &gt; /tmp/$$.txt
echo
if [ $limit -ge 1 ];
then
cat /tmp/$$.txt | sed -n "/^1\./, /${limit}/p" | head -n -1
else
cat /tmp/$$.txt;
fi&lt;/pre&gt;
&lt;pre&gt;

Buscando enlaces rotos de una web

&lt;/pre&gt;
&lt;pre&gt;#!/bin/bash
#Filename: find_broken.sh
#Description: Find broken links in a website
if [ $# -eq 2 ];
then
echo -e "$Usage $0 URL\n"
exit -1;
fi
echo Broken links:
mkdir /tmp/$$.lynx
cd /tmp/$$.lynx
lynx -traversal $1 &gt; /dev/null
count=0;
sort -u reject.dat &gt; links.txt
while read link;
do
output=`curl -I $link -s | grep "HTTP/.*OK"`;
if [[ -z $output ]];
then
echo $link;
let count++
fi
done &lt; links.txt
[ $count -eq 0 ] &amp;&amp; echo No broken links found.&lt;/pre&gt;
&lt;pre&gt;

ssh con compresión

&lt;/pre&gt;
&lt;pre&gt;ssh -C user@host&lt;/pre&gt;
&lt;pre&gt;

¿ssh con zenity?

&lt;/pre&gt;
&lt;pre&gt;$ ssh username@remotehost 'export DISPLAY=:0 ; zenity --info --text "This&lt;/pre&gt;
&lt;pre&gt;

is a message"'

Lanzar un comando cada x segundos

&lt;/pre&gt;
&lt;pre&gt;watch -n 1 -d 'ls -l /tmp'&lt;/pre&gt;
&lt;pre&gt;

Mysqlcon bash

#!/bin/bash
#Filename: read_db.sh
#Description: Read from the database
USER="user"
PASS="user"
depts=`mysql -u $USER -p$PASS students &lt;