Bob's Blog

How to Start Writing Scripts

Someone recently asked me, "How can I learn to write scripts?"

I think that they thought that they would have to do some study first. No, it's easy to get started, you could do that right now!

Were they asking about shell scripts? That is, Bash scripts? Or were they asking about some other scripting language such as Python? I didn't know, but that didn't really matter. The first thing that I told them was that it was easy to get started. Then I asked them about the specific scripting language they were interested in.

In an earlier article I explained how the command-line interface is so powerful, and how it's actually an interactive programming environment. If you know just enough to use the command line, or the "shell", then you're ready to start writing some simple scripts.

I've been using the UNIX family of operating systems since the mid 1980s, almost a decade before the first release of Linux, and I still learn something new from time to time. There's no need to wait until you think you know how, let's get started!

We Need An Excuse — What Shall We Do?

Let's say that you frequently wonder what time it is, both the current date and time, and where that is within the month. There are commands for that, try typing these:

$ date
Sat May 13 08:38:50 EDT 2023
$ cal
      May 2023        
Su Mo Tu We Th Fr Sa  
    1  2  3  4  5  6  
 7  8  9 10 11 12 13  
14 15 16 17 18 19 20  
21 22 23 24 25 26 27  
28 29 30 31           

Something that I learned only recently is that the cal command has an option to show a span of three months — the current month along with the month before and the month after.

$ cal -3
                            2023
       April                  May                   June          
Su Mo Tu We Th Fr Sa  Su Mo Tu We Th Fr Sa  Su Mo Tu We Th Fr Sa  
                   1      1  2  3  4  5  6               1  2  3  
 2  3  4  5  6  7  8   7  8  9 10 11 12 13   4  5  6  7  8  9 10  
 9 10 11 12 13 14 15  14 15 16 17 18 19 20  11 12 13 14 15 16 17  
16 17 18 19 20 21 22  21 22 23 24 25 26 27  18 19 20 21 22 23 24  
23 24 25 26 27 28 29  28 29 30 31           25 26 27 28 29 30     
30                                                                

Let's say that you have a friend who lives in New Orleans, which is in the Central time zone of North America, and you want to know what time it is there. Your shell interface has an environment in which several variables should be set to appropriate values. One of those is TZ, the time zone. It should be set for wherever your computer is. But you can ask the shell to run just one command with that one variable set to something else. Look at what happens, this is about an hour earlier and it shows "CDT" instead of "EDT" as it does by default for me. If you actually live in the Central time zone, you could use "America/New_York", or "Europe/Paris", or "Asia/Saigon", or any valid time zone name.

$ TZ=America/Chicago date
Sat May 13 07:40:11 CDT 2023

And so, let's say that you think it's useful, or interesting, or at least it's an adequate excuse for learning how to write a script (play along here), to run a command to show the current date and time, then a command to show a three-month calendar, and then a command to show the time in New Orleans.

$ date
Sat May 13 08:41:29 EDT 2023
$ cal -3
                            2023
       April                  May                   June          
Su Mo Tu We Th Fr Sa  Su Mo Tu We Th Fr Sa  Su Mo Tu We Th Fr Sa  
                   1      1  2  3  4  5  6               1  2  3  
 2  3  4  5  6  7  8   7  8  9 10 11 12 13   4  5  6  7  8  9 10  
 9 10 11 12 13 14 15  14 15 16 17 18 19 20  11 12 13 14 15 16 17  
16 17 18 19 20 21 22  21 22 23 24 25 26 27  18 19 20 21 22 23 24  
23 24 25 26 27 28 29  28 29 30 31           25 26 27 28 29 30     
30                                                                
$ TZ=America/Chicago date
Sat May 13 08:41:50 EDT 2023

However, that's quite a bit of typing, and you have to remember and accurately type several things:

• The -3 option for the cal command, and...
• The syntax for running a command with the TZ environment variable set to something else, and...
• The name of the time zone in which New Orleans is located.

Let's make this easy by writing a script!

How to Start Creating and Naming It

A script is just a text file, you create it and modify it with your favorite text editor. nano and pico are very simple and easy to use. Personally, I prefer the much more powerful vim editor, but it's an awfully overwhelming place to start.

Now, you'll need to give your script file a name. It can be almost anything, but you're writing a script to make things easy. Don't give it a weird name that makes it difficult to use. Don't use any punctuation marks other than a dash, an underbar, or a period, and don't include white space.

The operating system comes with very useful commands named time and date, so don't use those names. Since the purpose of your script is to check the time, maybe you want to name it checktime, or maybe check-time. Or maybe you want to follow the UNIX family tradition of short and easy to type (and sometimes hard to remember) names, and you'll use ct. That name isn't used, at least not on my system.

My personal preference is that everything should be all lower-case unless it somehow needs to be different, and it very seldom if ever really needs to be different. Why press more keys than absolutely necessary? But I know that other people feel, some of them quite strongly indeed, that a name formed of multiple words strung together should be in what's often called "Camel Case", and so checkTime instead of checktime.

Meh. Whatever. Any of those potential names will work. Pick whatever you want and start editing your new shell script file!

You might ask, "Don't I have to give my script a name that ends '.sh'?"

No. This isn't Windows. You can name a file whatever you want. The only thing that will pay attention to file name extensions will be some graphical desktop tools that try to emulate Windows. You can name your script file checktime.sh if you want, but there's no need, and that has no advantage over simply checktime. To me, a disadvantage is that you would always have to always type the added .sh, at least until you realized that Bash does command completion in addition to filename completion — the <Tab> key is your friend!

As you start creating your file, the one thing you must do is to make its first line explain what the file is. As we're creating a Bash shell script, make its first line contain exactly this:

#!/bin/bash 

That is, a hash character "#", and an exclamation mark "!" (which some people pronounce "bang"), and then the full path to the program that should interpret the following script. Since we're writing a Bash shell script, in this case that program is /bin/bash. If you were writing a Python script, it would start #!/bin/python or maybe #!/bin/python3 and so on.

Advanced topic: On FreeBSD the Bash shell isn't part of the base installation. You add it as a package and it's installed as /usr/local/bin/bash. But if you're already using FreeBSD, then you probably already know about shell scripts.

Creating Your Script

Now you simply put the sequence of commands into the file, one line for each command. You can add blank lines to make it easier to read.

Comments can make it easier to understand. Everything from a "#" character to the end of a line is there for humans to read, the shell will ignore it.

If you want to be somewhat fanatical about saving space, even though there's no real cost of how many characters are in a shell script, you could make your script look like this:

#!/bin/bash
# This is my script for checking the date and time.
date	# Output the date and time in the local timezone.
cal -3	# Display calendar for last month, this month, and next month.
TZ=America/Chicago date	# Output the date and time in the Central Time Zone,
			# used in Chicago and also New Orleans.
			# Running "date" command with environment
			# variable "TZ" changed to America/Central. 

Or, to be less fanatical and more readable, at least according to my personal tastes:

#!/bin/bash

# This is my script for checking the date and time.

# Output the date and time in the local timezone.
date

# Display calendar for last month, this month, and next month.
cal -3

# Output the date and time in the Central Time Zone, used in Chicago and New Orleans.
# Running "date" command with environment variable "TZ" changed to US/Central.
TZ=America/Chicago date 

Neither is more correct or appropriate than the other. It's just a matter of personal style and taste. Or maybe you work somewhere that imposes a corporate coding style. If so, do what they tell you to do.

How to Install and Run Your Program

You have to tell the operating system to treat your new script file as executable. That's the "x" bit you see reported in detailed listings.

$ ls -l check-time
-rw-rw-r-- 1 cromwell cromwell 387 May 13 08:57 check-time
$ chmod +x check-time
$ ls -l check-time
-rwxrwxr-x 1 cromwell cromwell 387 May 13 08:57 check-time*

Notice that the mode changed from 664 to 775, and now three x's appear for the user who owns it (you), the group that owns it (you should be the only member), and all other users.

Depending on how your system is set up, your ls may be defined as an alias which does the normal listing but also uses text color and a few appended punctuation marks to indicate directories, symbolic links, and files with particular content types, or which are executable.

Now you seem to be ready to run it, but it will probably fail like this:

$ check-time
check-time: command not found

What? You're certain that you spelled it correctly, and just turned on the executable bits. What's wrong?

Your PATH environment variable probably does not contain ".", the current directory. There's a security risk associated with doing that, so distributions don't.

You can run your new program by specifying exactly where it is and what it's named. Since I'm assuming that you started playing along with this story in your home directory, any of the following could work because "~" means your home directory, which is under /home:

$ ./check-time
[... output appears here ...]
$ ~/check-time
[... output appears here ...]
$ /home/yourlogin/check-time
[... output appears here ...]

However, the people who built your Linux distribution assumed that you very likely would start writing your own programs. So, they probably will have set up your PATH to include something just for you. See what yours is:

$ echo $PATH
[... one long line appears here ...]

The long line contains several entries separated by colons or ":" characters. Look for something under your home directory, such as /home/yourlogin/bin. Create that directory if needed, and move your new script into it, changing the details of the following as needed:

$ mkdir /home/yourlogin/bin
$ mv check-time /home/yourlogin/bin

Now you're ready to run your new shell script, no matter where you are at the moment:

$ check-time
[... output appears here ...]

Congratulations! You have created and installed your own shell script!

Extending and Improving Your Script

You can use the echo command to provide narration. We're working on a very simple script with pretty obvious output, but let's see how to do that:

#!/bin/bash

# This is my script for checking the date and time.

# Output the date and in the local timezone.
echo "The current date and time, locally, is:"
date

# Display calendar for last month, this month, and next month.
echo "Here's the calendar for three months centered on now:"
cal -3

# Output the date and in the Central Time Zone, used in Chicago and New Orleans.
echo "Here's the date and time in Chicago (and New Orleans):"
TZ=America/Chicago date 

You might decide that your echo command strings provide adequate explanation within your script, and it's redundant to have comments and the echo strings. That's fine, whatever you think is best!

Where To Go From Here

As you become more familiar with the command-line interface, you will learn and come to appreciate various commands. You will start to think of them as verbs — you could sort the content of a file or a data stream to put it into forward or reverse alphabetic or numeric order. You could grep a file or a data stream to extract just the lines containing certain patterns. You could awk it to select just certain fields of certain lines, and output them in a rearranged order. You could sed a data stream to modify it, or apply that change to a file to modify its content without using an interactive editor.

As you become familiar with the environment, you will think about tasks in a sequence of three major stages:

1. What sequence of processing steps is needed? Extracting just certain lines, then just certain fields out of those, sorting and counting and so on. You will need to think about this as a pipeline of processes, in which the output of one process becomes the input of the next process. Most of the fundamental UNIX-family commands are designed to support this.
2. What sequence of commands could somehow carry out those conceptual tasks?
3. Which specific options and syntax are needed to make that happen?

Then, if the overall task is something you anticipate wanting to repeat in the future, you can create a new shell script for it.

Shell scripts let you create arbitrarily complex tools to analyze patterns, calculate statistics, and generate reports about web server activity logs, security logs, and many more things of interest to security analysts, system administrators, web site administrators, and other.

I'll come back to this general topic from time to time, and show you example scripts providing answers for which I don't know of any pre-existing tools. But it's easy to write your own as a shell script! As I add suggestions and explanations, I'll add links to them here.