Custom Shell Scripts
Text Processing with Variables
The most popular interpreters, and corresponding scripting languages for processing text strings, are probably Perl and Python. The current versions of Bash, however, include enough string processing operators to be more than adequate for the simplest and most common tasks. The main ones are:
#> NAME=Marco #> echo ${NAME%co} Mar #> echo ${NAME/co/36} Mar36 #> echo ${NAME^^c} MarCo #> NAME=MARCO echo ${NAME,,*} marco
The statements above show how the shell can perform removal, substitution, and case modification, respectively, on parts of a variable.
The %
operator removes the shortest matching pattern (co
in the example) from a variable, starting from its end. The slashes delimit which part of the string should be replaced, and which other string to use. The caret (^
) and comma (,
) operators convert any occurrence of the matched characters to uppercase (c
) or lowercase, respectively (the asterisk is a shortcut for "any character"). All these operators have other formats and variants: To know how they work, check the Bash man page [3].
It is also possible to extract and replace specific parts of a string, whatever their value. I will show how to do it, together with another very useful feature of the shell, with a final, simple script (Listing 2).
Listing 2
Replacing Specific Parts of a String
01 #! /bin/bash 02 03 FIRSTNAME=$1 04 SURNAME=$2 05 PHONE=$3 06 07 cat<<INITIAL_DATA 08 09 Initial Data: 10 First Name : $FIRSTNAME 11 Surname : $SURNAME 12 Phone Number : $PHONE 13 14 INITIAL_DATA 15 16 FIRSTNAME="${FIRSTNAME,,}" 17 FIRSTNAME="${FIRSTNAME^}" 18 19 20 SURNAME="${SURNAME,,}" 21 SURNAME="${SURNAME^}" 22 23 PHONE_1="${PHONE:0:3}" 24 PHONE_2="${PHONE:4:3}" 25 PHONE_3="${PHONE:6}" 26 27 cat<<FORMATTED_DATA 28 29 Formatted Data: 30 First Name : $FIRSTNAME 31 Surname : $SURNAME 32 Phone Number : ($PHONE_1) $PHONE_2-$PHONE_3 33 34 FORMATTED_DATA 35 exit
This code is a really bare-bones formatter of names and phone numbers: No matter how you write them, the script returns name and surname with the first letter capitalized and the phone number in a standard format.
Listing 3 shows what you get when you run the script from Listing 2.
Listing 3
Output from
#> phone-formatter LEia ORGaNA 5553236856 Initial Data: First Name : LEia Surname : ORGaNA Phone Number : 5553236856 Formatted Data: First Name : Leia Surname : Organa Phone Number : (555) 236-6856
As is, the script is not complete, because it only formats one entry at a time. In a future installment, I will show how to work on multiple records. Even in this form, however, the code is useful as a first approach to text formatting and parsing, because the same tricks may be adapted, for example, to process generic spreadsheets in CVS format.
In Listing 2, lines 3 to 5 show how you can pass different parameters to a script every time you run it. The variables $1
, $2
, and $3
contain the parameters that were passed in from the command line. You may use them directly, but for readability, it is much better to copy them into $FIRSTNAME
, $SURNAME
, and $PHONE
.
Line 7 marks the beginning of a so-called "here document," which ends on line 14; cat
, a Linux command, simply prints out whatever you pass to it.
A here document is a really cool way to use variables and generate shell output. You can think of a here document as a pseudo-file or, more appropriately, template that is embedded into a shell script. The template ends with a line containing the same string used to mark its beginning (INITIAL_DATA
).
Inside the template, you can put as many variables as you like, and every time you use it, it will contain the current values of all those variables.
You can use here documents to custom generate any kind of complex, pre-formatted data on the fly, from database records to sequences of commands to pass to any other program.
Lines 16 to 21 just apply the case modification tricks already presented, and lines 23 to 25 show how to extract substrings. The final part of the script uses another here document to display the reformatted input values.
Conclusion
In this installment, I have covered the basic components that make shell scripts so flexible and usable. Next month, I'll play with more complex data structures like shell arrays and hashes. In the meantime, I highly recommend that you copy the provided script [5] and experiment both with the string processing commands and the here document layout to determine how much you can customize them. Happy scripting!
Infos
- Bash: http://www.gnu.org/software/bash/
- "Tutorial – Desktop News Feeds" by Marco Fioretti, Linux Magazine, issue 217, December 2018, pp. 90-95: http://www.linuxpromagazine.com/Issues/2018/217/Read-Me
- Seven expansion types: https://tiswww.case.edu/php/chet/bash/bash.html
- Generate random numbers in a given range: https://bytefreaks.net/gnulinux/bash/bash-get-random-number-that-belongs-in-a-range
- Code in this article: ftp://ftp.linux-magazine.com/pub/listings/linux-magazine.com/219/
« Previous 1 2 3
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
TUXEDO Computers Unveils Linux Laptop Featuring AMD Ryzen CPU
This latest release is the first laptop to include the new CPU from Ryzen and Linux preinstalled.
-
XZ Gets the All-Clear
The back door xz vulnerability has been officially reverted for Fedora 40 and versions 38 and 39 were never affected.
-
Canonical Collaborates with Qualcomm on New Venture
This new joint effort is geared toward bringing Ubuntu and Ubuntu Core to Qualcomm-powered devices.
-
Kodi 21.0 Open-Source Entertainment Hub Released
After a year of development, the award-winning Kodi cross-platform, media center software is now available with many new additions and improvements.
-
Linux Usage Increases in Two Key Areas
If market share is your thing, you'll be happy to know that Linux is on the rise in two areas that, if they keep climbing, could have serious meaning for Linux's future.
-
Vulnerability Discovered in xz Libraries
An urgent alert for Fedora 40 has been posted and users should pay attention.
-
Canonical Bumps LTS Support to 12 years
If you're worried that your Ubuntu LTS release won't be supported long enough to last, Canonical has a surprise for you in the form of 12 years of security coverage.
-
Fedora 40 Beta Released Soon
With the official release of Fedora 40 coming in April, it's almost time to download the beta and see what's new.
-
New Pentesting Distribution to Compete with Kali Linux
SnoopGod is now available for your testing needs
-
Juno Computers Launches Another Linux Laptop
If you're looking for a powerhouse laptop that runs Ubuntu, the Juno Computers Neptune 17 v6 should be on your radar.