Introduction: Expert-Level Scripting

Beyond the basics, mastering regular expressions, text processing tools (sed, awk), process management, and signal handling enables automation of complex server management tasks.

1. Regular Expressions

1.1 Meta Characters

.      # Any single character
^      # Start of line
$      # End of line
*      # 0 or more occurrences
+      # 1 or more (extended)
?      # 0 or 1 (extended)
[]     # Character class
[^]    # Negated class
|      # OR (extended)
()     # Grouping (extended)

1.2 grep with Regex

grep "^root" /etc/passwd          # Starts with root
grep "bash$" /etc/passwd          # Ends with bash
grep -E "[0-9]{3}" file           # 3 digits
grep -E "error|warning" log       # error or warning
grep -v "^#" config               # Exclude comments

# IP address matching
grep -E "([0-9]{1,3}\.){3}[0-9]{1,3}" file

2. sed (Stream Editor)

2.1 Basic Usage

sed 's/old/new/' file          # Replace first
sed 's/old/new/g' file         # Replace all
sed -i 's/old/new/g' file      # Edit in place

# Line addressing
sed '3s/old/new/' file         # Line 3 only
sed '1,5s/old/new/' file       # Lines 1-5
sed '/pattern/s/old/new/' file # Pattern match

2.2 Advanced sed

sed '/^#/d' file               # Delete comments
sed '/^$/d' file               # Delete empty lines
sed '/pattern/a\new line' file # Add after
sed 's/\(.*\):\(.*\)/\2:\1/' f # Swap order

3. awk (Pattern Scanning)

3.1 Basic Syntax

awk '{print $1}' file          # First field
awk -F: '{print $1}' /etc/passwd
awk '{print NF}' file          # Field count
awk '{print NR, $0}' file      # Line number

3.2 Pattern Matching

awk '/pattern/' file           # Contains pattern
awk '$3 > 100' file            # Field 3 > 100
awk 'NR > 5' file              # After line 5

3.3 Variables and Operations

awk '{sum += $1} END {print sum}' file
awk 'BEGIN {FS=":"} {print $1}' file
awk '{count[$1]++} END {for(k in count) print k, count[k]}' file

4. Process Management

# Background execution
command &
pid=$!
wait $pid

# PID file management
echo $$ > /var/run/app.pid
kill $(cat /var/run/app.pid)

5. Signal Handling

trap 'cleanup; exit' INT TERM
trap cleanup EXIT

cleanup() {
    rm -f /tmp/tempfile
}

6. File Descriptors

exec 3> output.txt    # Open FD 3
echo "data" >&3       # Write to FD 3
exec 3>&-             # Close FD 3

# Process substitution
diff <(sort f1) <(sort f2)

7. Option Parsing

while getopts "vf:n:" opt; do
    case $opt in
        v) verbose=1 ;;
        f) file="$OPTARG" ;;
        n) num="$OPTARG" ;;
    esac
done
shift $((OPTIND - 1))

8. Parallel Processing

for i in {1..10}; do
    process $i &
done
wait

# xargs parallel
cat urls.txt | xargs -P 4 -I {} curl {}

Conclusion

Advanced shell scripting enables complex text processing, process management, and signal handling. Next part covers log management and monitoring for effective server status tracking.