Linux shell/Generics, basics, modes, shell scanning, snippets

• Linux shell/Bash prompt PS1, settings and history
• Linux shell/Commands
• Linux shell/Functions
• Linux shell/Generics, basics, modes, shell scanning, snippets
• Linux shell/Productivity tools
• Linux shell/Skeletons

Shell modes

The shell can be used in two different ways:

interactive mode

which allows you to enter more than one command interactively to the shell. We have been doing this already in terms of entering commands. However, interactive mode allows you to enter a series of commands; and

shell scripts

which allow you to enter a series of commands or write complex programs in terms of an edited text file which is then executed by the shell once written.

Linux basics

• Everything is a number
• Everything is configurable
• Everything is a file
• Shell handles substitutions, redirections, etc.
• Double quotes " allow substitution ($ expansions)

Interactive shell process step-by-step

This is an order in which shell interprets the command line.

1. Reads one line a time unless bash recognizes a command that covers more than one line, it reads the entire command before processing it.
2. History Expansion, eg !!
3. Alias substitution - substitute a string for the first word of a simple command
4. Parsing (isolate strings of characters in) the command line into tokens or words.
   $ cp ~/letter . -> tokens: cp, ~/letter, and .
5. The shell then scans each token for special characters and patterns
6. Performs command line expansion for special characters
   1. Brace expansion
   2. Tilde expansion
   3. Parameter and variable expansion
   4. Arithmetic expansion
   5. Command substitution
   6. Word splitting
   7. Pathname expansion
   8. Process substitution
7. Quote removal - removes from the command line single quotation marks, double quotation marks, and backslashes that are not a result of an expansion
8. Evaluate permissions

Example

sudo sh -c 'echo "1"> /proc/sys/net/ipv4/ip_forward'
     \                                             /
      this new shell scan'll have root permissions

Environment default variables

$?    last command exit status, 0 -success, 1 -error 
$0    name of the shell script  
$#    number of parameters passed to the script  
$1, $2, .. $n      parameters given to the script; the number of parameters is $#  
$*    A list of all the parameters in a single variable.  
$@    A list of all the parameters in a single variable always delimited  
$$    process ID of the shell script often used inside of the script  
      to generate unique temporary filenames such as /tmp/tmpfile_$$

Execution Operators (&& and ||)

&& -conjunction (“and-ing”) continues on success of previous command (exit status=0)
|| -disjunction (“or-ing”) the second command is only executed if the first command failed with an exit status > 0

Evaluating commands

• "A ; B" Run A and then B, regardless of success of A
• "A && B" Run B if A succeeded
• "A || B" Run B if A failed
• "A &" Run A in background

Execution operators and command grouping

Often this grouping is used to exit script on failure.

my_command || { echo 'my_command failed' ; exit 1; }

• Use { } in place of ( )
• important ; after exit. This might be Bash specific.
• spaces after { and before }. This might be Bash specific.

Using ( ) makes the command inside them run in a sub-shell and calling a exit from there causes you to exit the sub-shell and not your original shell, hence execution continues in your original shell.

Return exit codes

0: Zero = true (success) 
1: Nonzero = false (fail)

Snippets

prints EC2 info wrapped into HTML

Code below prints EC2 info wrapped into HTML5 table tags.

display_instances () {
  echo -e "<h3>Hosts</h3>"

  EC2INFO=$(aws ec2 describe-instances --output text \
  --filters "Name=tag:Name,Values=*${FILTER}*" \
  --query 'Reservations[].Instances[].[Tags[?Key==`Name`] | [0].Value,InstanceType,InstanceId,LaunchTime,ImageId,State.Name]' | \
  grep -v tal | sort | sed '1i\Name Type Id LaunchTime ImageId State Mem vCPUs BurstOnFullCredit' | column -t)

  # Add Mem vCPUs BurstOnFullCredit columns 
  EC2INFO=$( \ 
    while read -r line
    do
      (( COUNTER++ ))
      EC2TYPE=$(echo -e "$line" | awk '{ print $2 }')
      EC2DETAIL=$(ec2type $EC2TYPE)
      EC2DETAIL=$(sed 's/\(.*\)\sFullCreditBurst/\1/g' <<< "$EC2DETAIL") # explanation [1]
      echo -e "$line $EC2DETAIL"
    done <<< "$(awk 'NR>1' <<< "$EC2INFO")" | sed '1i\Name State Id LaunchTime ImageId Type Mem vCPUs BurstOnFullCredit' # explanation [2] 
  )
  awk 'BEGIN { print "<table>" }
       { print "<tr><td>" $1 "</td><td>" $2 "</td><td>" $3 "</td><td>" $4 "</td><td>" $5 "</td><td>" $6 "</td><td>" $7 "</td><td>" $8 "</td><td>" $9 "</td></tr>" }
       END   { print "</table>" }' <<< "$EC2INFO"
}

# [1]   
# every \ backslash prefixes metacharacter of sed's RegEx
# \w -word, \s -any whitespace, \(\) -capture group parensis, \{\} -quantificators, \s\? -white space is optinal
# \1 \2 -1st and 2nd capture groupe [[:digit:]] -POSIX characters space, does not need to be escaped

# [2]
# heredoc (inject/pipe?) into awk to skip with skipping 1st line(NR>1 condition means NumberedRow>1). 
# All wrapped within subshell, subshell is double quoted to prevent stripping new lines and white spaces. 
# The subshell STDOUT is <<< heredock'd into while-read line loop, then added header line with sed to $EC2INFO variable

function verify_dependencies

function verify_dependencies(){
    if [[ "a$(which aws)" == "a" ]]; then
      pip install awscli
    fi
    echo "${FUNCNAME[0]} Ended" # <- note special 'env:FUNCNAME'
}

References

• Processing the Command Line The Bourne Again Shell, Pearson 2006
• Sh(POSIX) vs Bash Stackoverflow
• LDP Reference Cards tldp.org
• unofficial bash strict mode