In previous articles in our Intro to the CLI series, we’ve gone over why to master using the command line, some command line basics and tools, and tips for customizing your CLI shell. In case you missed any, feel free to catch up using the links below:

Part 1: What and Why of the CLI
Part 2: Introduction to Shell Commands
Part 3: Command Line Tools
Part 4: Customizing Your CLI Shell

And now, you’re going to learn the basics of shell scripting to automate complex jobs and build entirely new applications! 

Shell Scripting – Automate & Build New Applications

When you watch a movie or a streaming series, what you’re really watching is an actor and crew executing a script. That script is read top to bottom and left to right until they reach the end. Similarly, computers have become really good at reading scripts and doing exactly what they say.

These scripts can be very long and do things like providing local development environments by managing Docker, as in the case of Docksal, and these scripts can also be very simple, such as “hello world“. 

In this article, we’re going to cover a few topics related to shell scripting and some specific shell scripts you can run. 

• Hello World Script
• Bash Script vs Zsh Script
• Shell Script Comments
• Shell Script Echo
• sh File Extension
• Path Environment Variable
• Environment Variables
• Shell Script Logic
• Shell Script Variables
• If/Else Statement in Shell Scripting
• Shell Script While Loop
• Shell Script For Loop
• Shell Script to Take Input from User
• Advanced Shell Script Examples

Hello World Script

To write a hello world script, complete the following steps:

1. Open a new file called hello-world.sh in a text editor.

2. Type the following into the new file:

``

#!/bin/bash
# This script prints Hello World to the screen
echo "Hello World"

``

3. Save the file.

4. From terminal, while in the same directory as hello-world.sh, run the command:

$ chmod 755 hello-world.sh:

5. Next, run the command:

$ ./hello-world.sh

6. You should now see Hello World printed on the screen. 

Congratulations! You have just written and executed your first-ever shell script from scratch! That is a pretty big step on the path to becoming a seasoned developer! 

Even though the hello world script is simple, there’s a lot to unpack here. The really amazing news is that all shell scripts essentially work the same way, so understanding hello-world.sh and CLI shell basics mean you will be able to read almost any script.

Invoking a Shell Interpreter with a Shebang

Let’s start at the first line of hello-world.sh:

#!/bin/bash

That first part, the  #! is referred to as the shebang. The shebang tells the operating system that whatever immediately comes next is the program it should invoke to read and execute the rest of the contents of the file. The program is referred to as the script’s interpreter. In this case, it calls bash, which is located in the /bin folder.  bin stands for binary.

There are a lot of files in your /bin folder.  You can view all contents of that folder on your machine by running:

$ ls /bin

In your /bin folder, you may recognize some files that your operating system can execute aside from bash and zsh, like date, cp, ls and mv, among others.  This is where the executable binary files for those commands live on your system. 

When you run a shell script that starts with #!/bin/bash, you’re actually telling the computer to open up a non-interactive, non-login shell to run the commands in the file. 

Even though it’s non-interactive, the computer still has access to the terminal you executed the shell script from, and will be able to take input and print output to the session running it. This might be a tad confusing, so if you want to dig into it more, here’s a much more detailed explanation of non-interactive shells and shell scripts.  For the purposes of this article, all we need to know is that after the first line, everything is just CLI shell commands and some Bash scripting syntax.

Bash Script vs Zsh Script

You might be asking yourself, why use Zsh for the shell and Bash for the shell scripting language. The very simple answer is that Bash is more established as a scripting tool, so writing Bash scripts means your code is going to work on more machines. 

No matter if you’re on Fedora, Ubuntu, macOS, FreeBSD, or any other *NIX operating system, the hello world script will run the same way. You can depend on being able to work with Bash scripts on pretty much any *nix server or remote system you can access. It’s a universal language for tying together command line tools.

Zsh is an amazing interactive CLI shell, but it simply is not as universally adopted.

Other Shell Scripting Options

If you’re already versed in another language like Perl, Python, Ruby, or even PHP, you can absolutely use those to interpret and run your shell script.  You have the freedom to tell your computer to use any installed program to try to evaluate the file’s contents. 

For example, to evaluate a file using Python3, all you need to do is add this line to the top of your Python script:

#!/usr/bin/python3

Shell Script Comments

The second line of your hello world script is:

# This script prints Hello World to the screen

In a Bash script, the # character means everything after that symbol on the same line is a shell script comment and should be ignored when executing the script.  You will notice the shell script comment doesn’t get printed to the screen when you run hello-world.sh.

Shell script comments added in the code are referred to as inline comments. They can be very helpful for explaining how a bit of code works or what the intended outcome is.  Shell script comments are a convenient way to explain your work to any other developers who will work on the same code at a later date. 

Shell Script Echo

The third and last line of the hello world script is:

echo "Hello World"

The shell script echo tells the computer to print whatever comes after it to the standard output, which by default is the terminal screen.  The echo command works outside of shell scripts as well.  If you try pasting that line of script in as a command in your terminal right now, you should see:

Redirecting Shell Output

By default, the output from a command execution is printed to the terminal screen.  This is the “standard output”. In part 3 of this Intro to the CLI series, we reviewed redirecting this same output, specifically when using the pipe operator, |.

If you run $ ls -a | grep .git, for example, the output of the ls -a command is piped into the input for grep and only searches that input or the matching pattern .git.

The other way to redirect output is using the redirect operators, > and >>. These operators are most often used to stream a command’s output into a file.

The > overwrites any file contents.  For example, if you run:

$ echo "Hello" > hello.txt

…you will end up with a new file that has the word “Hello” as the only line.  

And then, if you were to run the following: 

$ echo "World" > hello.txt

….the file will contain the single line “World”, having overwritten the previous contents of the file. Be careful with this operator! 

The >> operator appends the output from a command to a file beneath the existing content.

For example, to continue on from the previous example, if you were to run:

$ echo 'Hello World' >> hello.txt  

…when you look at the contents, you should see two lines: the first being “World”, and the second being “Hello World”.

A very common use of redirect operators is creating and updating log files for an application. As you begin to debug your own applications, especially as they grow in size and complexity, it’s very helpful to redirect script output into a file that you can search and review.

sh File Extension

Now that you’ve looked at the insides of the hello world script example, let’s step back and look at the file itself and how it was executed.  

The file extension .sh stands for “shell script” and is actually just for human readability purposes. To the shell, the sh file extension is just a text file. Because the operating system reads from the first line to determine how to interpret the file contents, the actual name of the file doesn’t matter to the machine, but it is much easier to communicate the nature of the file to other developers, or even yourself later, when you use a file extension. 

Let’s test this quickly by renaming hello-world.sh to just hello-world and try running it again the same way we did before.

Permissions to Run the Shell Script

The next step after creating and saving hello-world.sh is to run the strange-looking command:

chmod 755 hello-world.sh 

Understanding File System Permissions in the Shell

Every file in a file system has permissions settings, also called modes, that allow the file to be read, written to, or executed by three different classes of potential users. The classes of potential users are:

1. User – the file owner
2. Group – a group of specified users who all share the same file permissions on the machine
3. Others – anyone outside of the above two classes

The ls command allows you to see file and directory details.  ls -l actually shows us who has what permissions.  

The line associated with the hello world script is:

-rwxr-xr-x  1 dwaynemcdaniel  staff   79 Feb  4 09:46 hello-world

Let’s quickly break down what the above is telling you. 

The first column: -rwxr-xr-x explains your file permissions.  The first character tells you if this is a file - or a directory d.

The next three characters: rwx tell you that the file owner can read, write, and execute the file. 

The middle three characters: r-x show the permissions of any users specified in a group. In this case, anyone in the group could read or execute the file, but not write to it. Groups come from the time when Unix systems, which involves all users sharing the same physical machine.

The final three characters: r-x in the first column refer to the permissions for any other user or agent, like Bash; again in this case, they can read or execute the file, but not write to it.  The rest of the line tells you:

– Column 2: How many links exist to this file from elsewhere on the system
– Column 3: Name of the file owner
– Column 4: What group owns the file
– Column 5: Size of the file in Bytes
– Column 6: Last modified date of the file
– Column 7: The file or directory name

Changing File Access to Run Shell Scripts

chmod lets us reassign permissions for each folder or directory, affecting the ability to read, write, or execute (when possible) a file or folder. 

The 755 used in the instructions for running hello-world.sh tells the computer to set permissions for the user, group, and others, following a pretty neat mathematical notation. 

The possible file permissions are:

– 0 = No permissions
– 1 = Execute only
– 2 = Write only
– 3 = Write and execute (1 + 2 = 3)
– 4 = Read only
– 5 = Read and execute (1 + 4 = 5)
– 6 = Read and write (2+ 4 = 6)
– 7 = Read, write and execute (1 + 2 + 4 = 7)

There are multiple other ways to set file access modes with chmod, but the numbering system is easy to remember and very handy.  755 is generally considered to be a safe option for things you will run locally, as it would give Bash, and any other users, a way to read and execute your shell scripts without being able to change the file it’s reading.  

Evaluating a File to Run Your Shell Script

Running $ ./hello-world.sh actually executes the shell script. The first part: ./, tells the operating system to evaluate whatever file is at the end of that path.

The . refers to the directory the shell is currently working in. When the operating system is evaluating a file, it’s reading the first line and looking for the shebang, #!, in order to open a new non-interactive shell to execute whatever command it finds at the end of the following specified path.  In this case, /bin/bash, which we already covered above. 

You might be wondering why you can’t just call the command hello-world.sh directly like you can with other commands like ls, mv or bash. Well, you can actually make that happen by adding your shell script into a folder in $PATH.

Path Environment Variable

The path environment variable, or $PATH, is important because the shell uses it to keep track of a list of directories that contain executable files.  When you type a command like ls, the shell searches through the default list of folders listed in $PATH until it finds an executable file called ls, which then is executed. This is how your CLI shell runs all commands, aside from files you tell it to specifically evaluate, as you did with the script example.  

To see what is currently stored in $PATH on your computer, you can run the following:

echo $PATH

If you’re using macOS, your path is likely going to look like this by default:

/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin

$PATH is stored as a colon-delimited list, meaning each entry is separated by a :.  These are the default directories that your operating system checks:

– /usr/local/bin
– /usr/bin
– /bin
– /usr/sbin
– /sbin

Take a second to check out what is located in each folder. It’s always a good idea to learn what’s available, even if you run some of the commands infrequently.  

Environment Variables in the Terminal

We just covered the path environment variable (or $PATH ). Being an environment variable means that the shell script  is storing a value that can be accessed by any process across the whole environment of your operating system. 

$Path is just one of many environment variables.  For example, another one is $HOME, though in this series we introduced it by its automatic special alias, the tilde, ~, as part of navigating the file system. 

If you run $ echo $HOME in your CLI shell, you will see what directory this points to.  It’s a good idea to never modify $HOME.  $Path, on the other hand, gets modified often.

To add a new directory to $PATH, you would add a line to your .zshrc folder that follows the formula:

export PATH=path/to/directory:$PATH

The export command is what makes environment variables available to the shell and any processes that the shell spawns. The rest of that command sets $PATH to be your newly specified directory plus the existing contents of $PATH. 

Some programs you install will have you add a line that exports the path to your .zshrc, which in turn will update $PATH every time the shell is loaded. For example, tools like PHP, Python3, Mamp, NodeJS, and npm will either automatically add, or ask you to add, specific lines to your .zshrc file as part of their installation process.

Run Shell Scripts from Anywhere in the File System

In order to execute hello-world.sh as a command, no matter what your PWD is, you have three options: 

1. Add the file’s directory to $PATH
2. Move the file into a system directory that is already in $PATH
3. Combine other options and make a new folder, move the executable file there, and add it to $PATH 

All three of these produce the same end result: you will be able to call hello-world.sh from anywhere.  

Because reusing a shell script often enough will require dealing with $PATH, many developers choose to drop the .sh at this point, as we did in the earlier example. 

Here is the result of any of those options, after renaming the script to just hello-world:

Shell Script Logic

Now you’ve made a simple shell script, changed permissions, and executed the shell script. The next step is understanding how to add programming logic to a script. Simple shell scripts just execute one command after another and stop when they’re done. The real power of a good CLI shell script, however, comes from the fact you can add logic to run commands only in certain conditions, multiple times, or even await user input to continue. 

While not an exhaustive list or meant to be a full tutorial, let’s quickly cover some of the most common and useful scripting logic operations.

Shell Script Variables

To create a variable, which is a defined entity that holds an assigned value, you simply declare it to equal a value:

VARIABLE1=1

To use a variable in a shell script, you add a $ to the front to mark it as a variable to be read in command execution.

echo $VARIABLE1

While not required, a common best practice for shell script variables is to name them with capital letters. Some developers believe this makes it easier to read the scripts and understand what’s going on.

If Else Statement in Shell Script

An if/else statement in shell scripting uses the following process: 

Test a condition to see if it is true, and if it is, execute certain commands, otherwise, do something else:

``
if [[ true ]]; then
command
else
command
fi
``

Shell Script While Loop

A while loop in shell scripting uses the following process:

Test to see if a condition is true and if it is, re-execute the commands, otherwise, stop and move on to the next part of the script:

``
while [[ true ]]; do
command

done

``

Shell Script For Loop

A for loop in shell scripting uses the following process:

Evaluate a variable or file, and repeatedly run certain commands until you have met a defined condition or reached the end of the list. 

The two most common syntaxes for writing a shell script for loop are:

1.  Define the variable i and each time you execute the list, increase the value of i by 1. Stop if i is equal or greater than 10:

``
for (( i=0; i<10; i++)
do
    Commands
done
``

2. Run the commands once for every value on a list and stop when you are out of values. You can reference the value inside the commands:

``
for value in list
do
    commands
done
``

Shell Script to Take Input from User

A shell script that takes input from a user works as follows:

The read command will wait for the shell user’s input before continuing, saving that input to a variable.

read VARIABLE

Advanced Shell Script Examples

While the hello world script example was pretty great, let’s look at a more complex advanced shell script example that employs the logic we just covered.  Each step will be explained by inline comments.  While you can copy/paste the entire example, we encourage you to actually type it out to gain some muscle memory. 

``
#!/bin/bash
# Clear the screen when this script runs
clear 

# Ask for the user name and await user input, store the input in NAME
echo "What is your first name" && read NAME

# Print NAME to screen in a greeting
echo "Hi $NAME"
echo ""

#Create a list of things to like or dislike
LIKELIST='dogs cats movies books'

# Loop through each value on the list that just was created
for LIKE in $LIKELIST
do
    # Ask the user if they like the current value from the list and store the answer in LIKEANSWER
    echo "Do you like $LIKE? (Answer only y or n)" && read LIKEANSWER

    # If they reply with a lower case 'y', print a positive reply
    if [[ $LIKEANSWER == 'y' ]]; then
        echo "Awesome! I like $LIKE too!"
        echo ""     
        # Echoing "" prints a new line to the screen.
     
    # If they type anything else other than 'y', including n, print another message
    else

        echo "That's OK. I understand not everyone is a $LIKE person "
        echo ""

    fi

done

# Tell the user thank you
echo "Thanks for running this script!"
echo ""``

After you have saved the file, run chmod 755, and executed the script, you should see something like this:

The above example uses some syntax and commands not introduced in this series, and was meant to encourage you to explore shell scripting even more. There are many awesome guides and tons of documentation available – just an Internet search away!

Git Hooks are Shell Scripts

Inside your repository’s .git folder is a directory called hooks. By default, this folder contains 13 files with names like commit-msg.sample, pre-push.sample, and pre-rebase.sample.

All of those files are shell scripts that correspond to specific events in a Git workflow. To allow Git to run any of those shell scripts when a related trigger occurs, simply remove the .sample from the filename.  Git is looking specifically for files without an extension. 

Inside those files are all sorts of examples to help you better leverage Git, and you can write your own shell scripts to automate just about anything. 

For fun, try adding the following line to the commit-msg hook:

curl -s https://icanhazdadjoke.com && echo ""

While maybe not the most professional example, you now know how to add logic to a Git hook! Credit to Edward Thompson for the inspiration from his tool git-dad.

CI/CD Pipelines Rely on Shell Scripts

One of the more practical reasons you need shell scripting skills as a modern developer is because of their use in CI/CD pipelines. From a simplified perspective, a CI/CD service just runs shell scripts on a remote server to cause actions. Of course, there are many additional features, like secrets management and application monitoring built into CI/CD platforms like CircleCI, GitHub Actions, and GitLab, but at their heart, they just execute scripts. 

For example, here is part a Bash workflow template from GitLab:

``

before_script:
  - echo "Before script section"
  - echo "For example you might run an update here or install a build dependency"
  - echo "Or perhaps you might print out some debugging details"after_script:
  - echo "After script section"
  - echo "For example you might do some cleanup here"
``

While you might not use Bash for the actual shell scripting language, once you know how one scripting language works, you basically understand how all of them work, though syntaxes will be different.  

The ins and outs of every CI/CD platform takes time to learn deeply, but having basic shell scripting knowledge means you can get a lot further, a lot faster.  

Consider Contributing to Open Source

Before we wrap up our Intro to the CLI series, I wanted to give a final encouragement to share your work and consider contributing to open source. Everything in this series came from researching the documentation of open source tools and reading through many examples of code. 

All developers are standing on the shoulders of giants, most of the time not knowing what part of your shared code someone else might find helpful.

However, be careful not to share passwords and configuration information in public repositories, but if you made a neat script that solves a specific problem for you, consider sharing it in a public repo and write up how it helped you in the README.  

Shell Scripting Resources

There are many resources to tap into out there to learn more about shell scripting. Here are four recommendations:

1. Codecademy’s Bash/Shell course
2. Bash Academy
3. Ryans Tutorials – Bash Scripting Tutorial
4. LinuxHint’s 30 Bash Script Examples

The community of developers who have been on this journey of discovery and learning is an amazing resource as well; we’ve all been a complete noob at one point.  Never be afraid to ask for help. 

Thank you for learning what the CLI stands for. No matter who you are, if you use a computer, learning to use the CLI shell is transformative. There is nothing to fear and the benefits of learning these tools far and away outweigh the costs.

GitKraken’s set of Git tools are designed to help speed developers on your learning journey.  The GitKraken CLI‘s autocomplete suggestions will help you navigate the file system and use Git like a pro in no time. GitLens will help you collaborate better with other developers by making each line’s authorship and commit history more clear.  We invite you to download and install both to make the best use of the CLI shell on your journey!