Bash Tips: Debugging and Configuration
By
Even tried and trusted configurations that have proved their value through the years can be tripped up by special cases – for example, all of your users logging in with the same IP address. Experienced administrators and Bash aficionados will then enable debugging and launch a systematic search for configuration errors.
At LinuxTag in Berlin, Germany, the more members that turn up for the annual event, the more difficult it becomes to use IMAP to retrieve their email. This time, the venue’s network connection was more than generously dimensioned, and the system load had been tested up front; with an uptime of more than 500 days, the server had a load of less than 0.4. But people still couldn’t access their mail.
Crush
New this year were NAT-based addresses assigned to organizer’s notebooks. In previous years, when IP address scarcity had probably not been an issue, the LinuxTag network team’s DHCP server assigned public addresses instead. From the IMAP server’s point of view, users hiding behind a NAT gateway appear to all have the same IP address. Some Courier implementations of imapd restrict the number of connections from a single IP as a security measure. It looks like nobody envisaged a crush of clients on a private network like this.
Although the organizers’ technical team quickly identified the problem, the challenge was to find the right place to change the configuration as quickly as possible. The developers of the Courier IMAP server spread this functionality across half a dozen individual programs: Launched by courierlogger, couriertcpd first accepts all connections for SSL-encrypted IMAP requests on TCP port 993. It uses a ruleset to decide whether or not to accept connections from a host.
It then hands off to couriertls, another wrapper that handles the task of opening the SSL connection. couriertls then calls imapd proper via another intermediate step (imaplogin), and the daemon hands over the message to the client.
From Pillar ...
In many cases, the system administrator needs to find out exactly how the programs call each other. This is all the more the case when you need to shut down the whole service and its programs, change a parameter in the configuration somewhere, and then relaunch everything – which is what you need to do to adjust the clients per IP threshold, for example.
A runlevel-specific script in /etc/init.d controls these processes. In the course of ongoing modularization and configurability, a conglomerate of scripts and parameter definitions with various conventions has developed from what was a single large file, /etc/rc.local, in the good old days of BSD. (Many administrators are still skeptical about the designated successors such as Upstart, Systemd, and the like).
... to Post
The shell’s -x option helps the administrator systematically visualize the order of the calls. If you use an editor to add
set -x
below the introductory Shebang line in /etc/init.d/courier-imapd, you can switch Bash to logging mode. From now on, the shell doesn’t just do what you tell it, it also reports what it’s doing via standard output. This means you can see which configuration files are parsed, which parameters are set, what lockfiles are created, and what subprocesses are started. In very tricky cases, the -e option also can be useful: It ensures the script is interrupted immediately if just one call generates an error code. Normally the script
#!/bin/bash ls xy id
would continue to run the correct calls in the second and fourth lines, but would output a message about the missing xy in line 3 before Bash went on to execute id. However, if you used the set -e option after the Shebang line, the script would terminate after xy.
The two flags remain active for all subsequent calls in this shell, which means you need to reset them later on by issuing set +x or set +e, or remove the line from the source code.
Clear Distinction
Separating code and data is regarded as exemplary; in particular, you should never set configurations directly in the source code. The range of formats for setting configurations is huge, from XML files, through blank-separated lists, through special configuration languages. Bash developers will tend to go for the simple, built-in mechanism of assigning shell variables in a separate file, as in:
maxconnections=30 PORT=993
Instead of manually parsing the file, you can execute it as a separate script, and to do so, you seem to only need to call the script name, as in ./configuration.sh. The problem is that this creates a new shell that manages its own variables. The calling script remains blissfully unaware of this (Figure 1).
Using uppercase for variable names doesn’t change this, contrary to a widespread misconception, as you can see from APPLE in Listings 1 and 2. In fact, using uppercase is merely a convention for identifying environment variables, and you need the export keyword to make them stick.
Listing 1: top.sh
01 #!/bin/sh 02 03 # Remove to see more detail! 04 # set -x 05 06 echo “I am top.sh” 07 08 echo “set apple to red, APPLE to RED” 09 10 apple=red 11 APPLE=RED 12 13 echo “call bottom.sh” 14 15 ./bottom.sh 16 17 echo “exporting APPLE” 18 export APPLE 19 20 ./bottom.sh 21 22 echo “Value of exported APPLE” 23 echo “after returning from subshell: >$APPLE<.” 24 25 echo “apple now has the value >$apple<.” 26 27 echo “now source bottom.sh” 28 29 . ./bottom.sh 30 # alternative notation: source ./bottom.sh 31 32 echo “apple now has the value >$apple<.” 33 exit 0
Listing 2: bottom.sh
01 #!/bin/sh 02 03 echo “this is bottom.sh” 04 05 echo “apple now has the value >$apple<.” 06 echo “APPLE now has the value >$APPLE<.” 07 08 echo “change the value to green:” 09 10 apple=green 11 APPLE=GREEN
This trick will not help you read configuration settings, as the output from the test scripts just goes to prove. Although you can send data to the subprogram, you can’t get them back again. You need the source keyword for this – or alternatively, a dot character (.) (Listing 1: line 29). In this case, Bash treats the child script as if it had copied its source code into the calling script.
Confusion
The paths of many a startup or configuration script are winding, but not unfathomable. The -x or -e shell options can help administrators perform a systematic search. You always need to take a close look when you integrate configuration files; after all, a single dot (.), as a synonym for source, can mean a huge difference in the execution of a subshell [1].
Armed with these tools, the LinuxTag admins located the MAXPERIP parameter and increased it, so the organizers could open the doors to LinuxTag for business as usual.
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
AlmaLinux OS Kitten 10 Gives Power Users a Sneak Preview
If you're looking to kick the tires of AlmaLinux's upstream version, the developers have a purrfect solution.
-
Gnome 47.1 Released with a Few Fixes
The latest release of the Gnome desktop is all about fixing a few nagging issues and not about bringing new features into the mix.
-
System76 Unveils an Ampere-Powered Thelio Desktop
If you're looking for a new desktop system for developing autonomous driving and software-defined vehicle solutions. System76 has you covered.
-
VirtualBox 7.1.4 Includes Initial Support for Linux kernel 6.12
The latest version of VirtualBox has arrived and it not only adds initial support for kernel 6.12 but another feature that will make using the virtual machine tool much easier.
-
New Slimbook EVO with Raw AMD Ryzen Power
If you're looking for serious power in a 14" ultrabook that is powered by Linux, Slimbook has just the thing for you.
-
The Gnome Foundation Struggling to Stay Afloat
The foundation behind the Gnome desktop environment is having to go through some serious belt-tightening due to continued financial problems.
-
Thousands of Linux Servers Infected with Stealth Malware Since 2021
Perfctl is capable of remaining undetected, which makes it dangerous and hard to mitigate.
-
Halcyon Creates Anti-Ransomware Protection for Linux
As more Linux systems are targeted by ransomware, Halcyon is stepping up its protection.
-
Valve and Arch Linux Announce Collaboration
Valve and Arch have come together for two projects that will have a serious impact on the Linux distribution.
-
Hacker Successfully Runs Linux on a CPU from the Early ‘70s
From the office of "Look what I can do," Dmitry Grinberg was able to get Linux running on a processor that was created in 1971.