Better Bash
Shell scripts from hell: Shebang
ByIn the beginning was the double pound sign and the exclamation mark – or at least shell scripts always start this way. The inventor, Dennis Ritchie, really didn’t know how much pain this was going to cause users.
When a user opens an interactive program, the shell unfolds more magic than you might imagine, especially if you happen to be talking about scripts. Bash is so omnipresent that people tend to forget it is also code that follows certain rules.
Bash has very little to do with accepting key presses. This job is handled by the terminal driver, which feeds a pseudo-terminal either locally at the console or via a detour through SSH or an X client. The pseudo-terminal passes the input on to the interactive shell, which sends a prompt to the terminal beforehand.
The shell waits until it receives an EOL and then interprets the string up to that point according to Bash syntax. If the string is an internal command – such as a while loop, an if condition, or an assignment that uses = – the shell executable can take the necessary action directly. This also applies to the many shell built-ins, such as ulimit or history, or any shell functions you defined yourself.
If none of these cases applies, Bash assumes the user wants to launch an external program, but it first needs to find the program. To do so, it iterates against the content of the PATH environmental variable, which it first separates by the delimiting colons.
Bash opens each element as a path and searches for a file with the execute flag set and with the name of the command input. Recent versions of Bash use a cache for this to avoid the need to search the physical filesystem for the complete path, but this doesn’t really change the approach just described.
External Binaries
If the command interpreter finds something, it initially leaves the rest of the job to the kernel by enabling the execve() call in a new process. It passes in the full path, the command name input, and – if command-line arguments exist – the arguments to the process. The kernel opens the file and checks the first 2 bytes. If they reveal that the file is a genuine binary (e.g., in ELF format), the kernel launches it directly. In the meantime, the shell waits for the program to terminate and then proceeds to process the next command.
#! All Change!
However, if the first two bytes are #!, the control flow takes a convoluted path back to linux/fs/binfmt_script.c in the kernel; Listing 1 shows some simplified code. This triggers a complicated mechanism:lines 10 through 19 delimit the #! line with an end marker and remove any white spaces that occur before this. Line 20 eliminates any blanks that directly follow the Shebang, because Unix creator Dennis Ritchie explicitly permitted this behavior in an email back in 1980.
Listing 1: Kernel Parses Shebang
01 static int 02 load_script(struct linux_binprm *bprm, struct pt_regs *regs) { 03 const char *i_arg, *i_name; 04 char *cp; 05 struct file *file; 06 07 if ((bprm->buf[0] != '#') || (bprm->buf[1] != '!')) 08 return -ENOEXEC; 09 10 if ((cp = strchr(bprm->buf, '\n')) == NULL) 11 cp = bprm->buf+BINPRM_BUF_SIZE-1; 12 *cp = '\0'; 13 while (cp > bprm->buf) { 14 cp--; 15 if ((*cp == ' ') || (*cp == '\t')) 16 *cp = '\0'; 17 else 18 break; 19 } 20 for (cp = bprm->buf+2; (*cp == ' ') || (*cp == '\t'); cp++); 21 if (*cp == '\0') 22 return -ENOEXEC; /* No interpreter name found */ 23 i_name = cp; 24 i_arg = NULL; 25 for ( ; *cp && (*cp != ' ') && (*cp != '\t'); cp++); 26 while ((*cp == ' ') || (*cp == '\t')) 27 *cp++ = '\0'; 28 if (*cp) 29 i_arg = cp; 30 [...] 31 }
If trimming in line 21 fails to create a meaningful string, there is no interpreter name and the function reports an error in line 22. If this is not the case, the interpreter name is now available in i_name; this is typically /bin/sh. Lines 24 through 29 then take precisely one argument from the remaining line – if it exists – and store it in i_arg. The kernel ignores the rest of the first line.
Finally, the function calls itself recursively within the kernel using the command interpreter it extracted and reappends the original command name and the arguments. For example, if /tmp/runme contains an initial line of #!/foo/bar --myarg, PATH=/tmp is true, and if the user types runme alpha beta, the kernel actually calls execve("/foo/bar", "bar", "--myarg", "/tmp/runme", "alpha", "beta"). But if the kernel fails to determine an interpreter by following this procedure, Bash takes control of the execution, assumes that the file contains valid shell code, opens the file, and interprets its content.
Weird
You might be wondering why the kernel takes this roundabout approach. Ritchie’s response to this was that it allows shell scripts to be launched by exec() calls, that the process display and accounting show more intuitive names, and – this might surprise you – that you can assign set UID flags to shell scripts, too.
Unfortunately, this idea caused administrators no end of security worries. If a Unix user links to a set UID script that starts with #!/bin/sh, and cunningly names the link -i, the resulting call is /bin/sh -i, which gives the user a very convenient, interactive root shell. This explains why Linux dumps the privileges that result from additional flags in the extra Shebang loop.
At the same time, there is also an attack vector for a race condition in which the file changes between parsing and execution. All told, this mechanism – no matter how well-meant it was on Ritchie’s part – has mainly caused confusion. In a survey, Sven Mascheck investigated this aspect in 48 Unix derivatives, finding very little common ground, but a number of vulnerabilities.
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
Fedora 40 Beta Released Soon
With the official release of Fedora 40 coming in April, it's almost time to download the beta and see what's new.
-
New Pentesting Distribution to Compete with Kali Linux
SnoopGod is now available for your testing needs
-
Juno Computers Launches Another Linux Laptop
If you're looking for a powerhouse laptop that runs Ubuntu, the Juno Computers Neptune 17 v6 should be on your radar.
-
ZorinOS 17.1 Released, Includes Improved Windows App Support
If you need or desire to run Windows applications on Linux, there's one distribution intent on making that easier for you and its new release further improves that feature.
-
Linux Market Share Surpasses 4% for the First Time
Look out Windows and macOS, Linux is on the rise and has even topped ChromeOS to become the fourth most widely used OS around the globe.
-
KDE’s Plasma 6 Officially Available
KDE’s Plasma 6.0 "Megarelease" has happened, and it's brimming with new features, polish, and performance.
-
Latest Version of Tails Unleashed
Tails 6.0 is based on Debian 12 and includes GNOME 43.
-
KDE Announces New Slimbook V with Plenty of Power and KDE’s Plasma 6
If you're a fan of KDE Plasma, you'll be thrilled to hear they've announced a new Slimbook with an AMD CPU and the latest version of KDE Plasma desktop.
-
Monthly Sponsorship Includes Early Access to elementary OS 8
If you want to get a glimpse of what's in the pipeline for elementary OS 8, just set up a monthly sponsorship to help fund its continued existence.
-
DebConf24 to be Held in South Korea
Busan will be the location of the latest DebConf running July 28 through August 4