Skip to content

BashGuide

Jump to bottom
Marcel Schmalzl edited this page Apr 10, 2024 · 11 revisions

A more complete and extensive guide is Advanced Bash-Scripting Guide Many examples are based hereof.

Command separators

  • Operator &

    • executes the command in the background in a subshell

        # For example:
  for n in {1..30}; do
  wget "https://i11www.iti.kit.edu/_media/teaching/winter2012/algo2/vorlesung${n}.pdf" &
  #                                                             do in subshell   -----^^^
  done

    • does not wait for the command to finish, and the return status is 0

  • Operator ;
    • Commands separated by a ; are executed sequentially
    • The shell waits for each command to terminate in turn
    • The return status is the exit status of the last command executed

Special chararcters / compound commands

End of options (--)

A double-dash defines the end of options:

ls -lh -- -badName/

Comment ( # )

# introduceds a line comment

$var1 vs. ${var1} (dollar sign + curly braces)

Variable substitution (contents of a variable) (e.g. var1=5; echo $var1 # outputs: 5)

Use parameter substitution (${var1}) instead (same effect but resolves some disambiguities e.g. when concatenating variables with strings).

See also Brace/Variable expansion.

Non-/No-op command (:)

: is a null command. It can be interpreted as a synonym for the shell builtin true.

Exit status is true (/0).

Last exit code ($?)

$? returns the exit code of the last command.

$$ / PID

echo "pid is $$" prints the process ID.

Control structures

while

inputFile="./diff.txt"

while read file; do
    cp ${file}  "./diffedFiles"
done < ${inputFile}

Note that we also use our variable to redirect the contents to the loop (works also to store loop outputs in a variable).

for

Example using an array:

# Declare an array variable
declare -a arr=("element1" "element2" "element3")

for i in "${arr[@]}"; do
    echo "$i"
    # Or do whatever with individual element of the array
done
# You can access them using echo "${arr[0]}", "${arr[1]}" also

Examples

Using ranges with leading zeros:

for i in {000..999}; do
    echo ${i}
done

if conditions (if and fi)

if [[ "$(ls -A ${pathTemp})" ]]     # True if length of string is non-zero
    then                            # Or in one line: `if [[ "$(ls -A ${pathTemp})" ]]; then`
        echo "Path is not empty"
        # ...
    else
        echo "path is EMPTY"
        # ...
elif [[ <cond> ]]
    :   # Elif branch (optional)
else
    :   # Else branch (optional)
fi

Extended test command - [[ expression ]]

Return a status of 0 or 1 depending on the evaluation of the conditional expression expression. The spaces are mandatory.

([ ]: test command; Same as Extended test command / [[ ]]. Prefer [[ ]] over [ ] (see also http://mywiki.wooledge.org/BashFAQ/031). This is more safe but does not work in POSIX-only environments.)

Example

Check if folder /tmp is empty:

[[ "$(ls -A /tmp)" ]] && echo "Not Empty" || echo "Empty"

RegEx-if condition

read -p "Press \`Y\` or \`y\` to continue: " -n 1 -r
echo                           # Print newline after command was entered
if [[ "${REPLY}" =~ ^[Yy]$ ]]
    then
         : # Continue
    else
        exit 1
fi

Strings etc.

Here document

HEADER= cat <<'EOF'
Some longe multiline string
       with indentations.
EOF

# or for commands:
VAR=$(cat <<EOF
blah
EOF
)
  • EOF is just an arbitrary delimiter for the start/end of the here document
  • Quote (like above) your start/end sequence (e.g. EOF -> 'EOF') to avoid escaping
  • -EOF to lstrip any leading whitespaces

Functions

function noParams {
    echo "blah"
    exit
}

function withParams {
    echo $1 
}

# Function call
noParams

# Function call **with** one param
withParams 'Hello'
withParams 'World'

Bash does not have return statements. Processing values of a function call can be achieved by echoing the "return"-value.

res=$(noParams)
# res = "blah"

Traps / traptest

Traps react on signals (show std signals with kill -l; EXIT = 0).

  • Set trap: trap 'cmdOrFctName' <SIGNAL>
  • Signals can be ignored: trap '' SIGINT
  • Reset trap: trap SIGINT (no longer execute traps for SIGINT)

Examples

Catch Ctrl-C

Prints and increments counter until Ctrl-C is pressed (kill with: kill -s SIGKILL <PID>).

counter="0"
trap "echo \$((counter++))" SIGINT SIGTERM
echo "PID is $$"

while :
do
	sleep 3
done

Catch non-zero exit codes and handle in function

#!/bin/bash
set -e

trap 'trapFctName $? $LINENO' EXIT
trapFctName() {
    if [ "$1" != "0" ]; then
      # error handling goes here
      printf "Error %s occurred in line %s\n" $1 $2
    fi
}

`

--------------------------------------------------------------------------------------


# Redirecting
* `>` : output - write (to file (overwrites existing ones)) **redirects only stdout**
* `>>` : append (to file)
* `<` : input - read (from file)

## Example - appending to a file
Appends all `.txt` files in current directory:
    
```bash
cat *.txt >> ~/anotherDirectoryToAvoidInfiniteAppending/all.txt

Example - store std::out to variable but keep outputting to std::out

PIP_STATUS="$(pip3 install --upgrade pip | tee /dev/tty)"

Stderr, Stdout, Stdin

command   identifier1>identifier2
            ^^^      ^    ^^^
redirect    id1     to    id2
std level identifier
Stdin 0
Stdout 1
Stderr 2

Redirecting stderr to stdout: &>word, >word 2>&1, and >&word are exactly the same.

Examples

# Pipe to "null device" (this discards everything it gets)
ls -R * 2> /dev/null

# or redirect output of stdout to a file (no space after the number!!)
find ./ -name "newfile*" 1> log.txt

# stderrr to stdout
programm 2>&1

# stderr and stdout to file
programm &> Datei.txt

Printing (to Stdout, ...)

Use printf in favour of echo (here is why)

# Prints var as a string
printf '%s' "$var"

Exit codes

Some exit codes have special meanings. See also /usr/include/sysexits.h. $? is used to find the return value of the last executed command.

Exit Code Number Meaning Example Comments
0 Success --- ---
1 Catchall for general errors let "var1 = 1/0" Miscellaneous errors, such as "divide by zero" and other impermissible operations
2 Misuse of shell builtins (according to Bash documentation) empty_function() { } Missing keyword or command, or permission problem (and diff return code on a failed binary file comparison).
126 Command invoked cannot execute /dev/null Permission problem or command is not an executable
127 "command not found" illegal_command Possible problem with $PATH or a typo
128 Invalid argument to exit exit 3.14159 exit takes only integer args in the range 0 - 255 (see first footnote)
128+n Fatal error signal "n" kill -9 $PPID of script $? returns 137 (128 + 9)
130 Script terminated by Control-C Ctl-C Control-C is fatal error signal 2, (130 = 128 + 2, see above)
255* Exit status out of range exit -1 exit takes only integer args in the range 0 - 255

Piping

Piping to another process:

command1 | command2

This will send the output of command1 as the input of command2.

Usually, the last item in a pipeline determines the result (return value).

Argument passing

Arguments

Sends the output of one process as command line arguments to another process

command2 `command1`

For example:

cat `find . -name '*.foo' -print`

This will send the output of command1 into command2 as command line arguments. Note that file names containing spaces (newlines, etc.) will be broken into separate arguments, though.

mSearch() {
	grep -nC "${2}" -i "$1" 'blub.txt'
	#           ^--------^--------- second and first argument (actually it begins with zero $0 = script name)
}

Optional arguments

Syntax:

"${<argumentNumb>-${varName}}"
"${<argumentNumb>-<default-value>}"

Example:

mSearch() {
	numPadLines=7
	# "${2-${numPadLines}} ---> Use numPadLines if 2nd argument is unset
	tac "notes.md" | grep -nC "${2-${numPadLines}}" -i "$1" --color=auto --group-separator=***********************************************************
}

Pattern matching

//TODO: For now see: https://www.gnu.org/software/bash/manual/html_node/Pattern-Matching.html#Pattern-Matching

Variables / Parentheses

Synthax Explanation
{ ... } Brace expansion
$var or ${var} Variable/parameter expansion
( ... ) Command group
(( ... )) C-style manipulation + arithmetic expansion and evaluation

Readonly variables

Define constants as read-only

readonly TIMEOUT_S=10

Curly Braces

Braces are reserved words, so they must be separated from the list by blanks or other shell metacharacters.

  • Expanding arrays, as in ${array[42]}
  • Parameter expansion operations, as in ${filename%.*} (remove extension)
  • Expanding positional parameters beyond 9: "$8 $9 ${10} ${11}"
  • Ranges:
$ echo {0..20}
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
  • Special usage in find -exec
    • {} substitutes the matches found
    • -exec takes a program and arguments and runs (-> it does not execute a full bash)
    • To overcome this: $ find $felix -name "PQIS*.zip" -exec bash -c 'echo $(dirname {})' \;

Examples

$var      # use the variable
${var}    # same as above
${var}bar # expand var, and append "bar" too
$varbar   # same as ${varbar}, i.e expand a variable called varbar, if it exists.

OUTPUT="$(ls -1)"
echo "${OUTPUT}"    # Quoting (") does matter to preserve multi-line values

( ) / Single Parentheses

When commands are grouped, redirections may be applied to the entire command list.

  • Command group ( e.g. (a=hello; echo $a))
  • Array init (e.g. Array=(element1 element2 element3))

Examples

1. Colon ( : ) in if fi

Since if requires a non-empty then clause and comments don't count as non-empty, [: serves as a no-op](#:\ /\ Colon\ /\ non-op\ command).

if command; then :; else ...; fi

2. { } vs. ( ) 

count_tmp() { cd /tmp; files=(*); echo "${#files[@]}"; }
pwd; count_tmp; pwd         # pwd : "print working directory"

Output

/home/mrspaceinvader
11
/tmp

Function body executes in a subshell

cd ; unset files
count_tmp() (cd /tmp; files=(*); echo "${#files[@]}")
pwd; count_tmp; pwd

Output

/home/mrspaceinvader
11
/home/mrspaceinvader

Double Parentheses (( ))

Permits arithmetic expansion and evaluation (e.g. a=$(( 5 + 3 )); would set a to 8). Allows C-style manipulation of variables in Bash, for example, (( var++ )) or (( var = 42 )) or ((var+=42)).

Strings

Everything is a string (no typing in bash)

Length of a string

$ blah="blub"
$ echo ${#blah}
4

Slicing

Starts the string at position 1 (bash starts counting at 0) until the end (legth of string)

$ echo ${blah:1:${#blah}}
lah

Or get rid of the last character

$ echo ${blah::-1}
bla

However,

$ echo ${blah:1:}

prints nothing.

Quoting

Better quote everything (e.g. rm "$2" << might contain spaces; without quotes you'll get a list of files to delete)

Double Quotes

Variable references are instantly expanded.

If you want selective expansion inside a string - i.e., expand some variable references, but not others - do use double quotes, but prefix the $-references which you do not want to expand with \; e.g., \$var).

Single Quotes

Strings inside single quotes are not expanded or interpreted in any way by the shell.

Comparison Operators

See [if fi chapter](#if\ /\ Tests\ /\ Expression\ evaluation\ (if\ and\ fi)) for usage.

Integers

  • -eq : is equal to (f.ex.: if [ "$a" -eq "$b" ])

  • -ne : is not equal to

  • -gt : is greater than

  • -ge : is greater than or equal to

  • -lt : is less than

  • -le : is less than or equal to

  • < : is less than (within double parentheses) (f.e.x.: (("$a" < "$b")))

  • <= : is less than or equal to (within double parentheses)

  • > : is greater than (within double parentheses)

  • >= : is greater than or equal to (within double parentheses)

Strings

  • = / == : is equal to (f.ex.: if [ "$a" = "$b" ]; Note the whitespace framing of = )

    Note: The == comparison operator behaves differently within a double-brackets test than within single brackets:

[[ $a == z* ]]   # True if $a starts with an "z" (pattern matching).
[[ $a == "z*" ]] # True if $a is equal to z* (literal matching).

[ $a == z* ]     # File globbing and word splitting take place.
[ "$a" == "z*" ] # True if $a is equal to z* (literal matching).
  • != : is not equal to
  • < / > : is less/greater than, in ASCII alphabetical order (< / > needs to be escaped within a [ ] construct)
  • -z : string is null, that is, has zero length
  • -n : string is not null

Famous :(){ :|:& };:

Long form

:()
{
	:|:&
};
:

Explanation

:() #Define new function
	#named ':'
{ #Begin function definition
#block
:|:& #Pipe the very ':' function through itself,
		#creating two processes, and make the
		#resulting copy run in the background
		#(the & part)
} #End function definition block
;: #Call ':' for the first time, initiating a chain
#reaction: each instance of ':' will create two
#more instances, ad infinitum

Debugging and useful commands

Exit on first failure

set -e

Print/echo each command before execution

set -o xtrace

-> Also prints evaluation of

Additional Ressources

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License *.

Code (snippets) are licensed under a MIT License *.

* Unless stated otherwise

Home

Python 3

(un)fold
Snippets
General
Libs

Linux/bash

(un)fold
Guides
Scripts

Git

(un)fold

C/C++

(un)fold

Video

(un)fold

Databases

(un)fold

Misc

(un)fold

Windows

(un)fold

Mac

(un)fold

SW recommendations

(un)fold

(Angular) Dart

(un)fold

Becoming a Software Eng

Clone this wiki locally