Run statistics on typed shell commands
Programming Snapshot – Shell Stats

© Lead Image © Scott Rothstein, 123RF.com
In the history log, the Bash shell records all commands typed by the user. Mike Schilli extracts this data with Go for a statistical analysis of his typing behavior.
How did the long command to connect to the database server work again? Shell power users know this memory problem and have taught themselves tricks over the years to repeat or resend modified versions of previously sent command lines. For example, the sequence !!
repeats the last command (useful to prepend a sudo
you might have forgotten); with Ctrl+R you can find and reuse commands that were sent further back in time by using search patterns. Bash remembers the entire typing history and shows it with the history
command, which lists the tail end of what it has in memory (Figure 1).

If you have ever taken a look at the .bash_history
file in your home directory, you will know its little secret. This is where Bash simply appends every command sent, regardless of if it was successful or aborted with an error. If you ask for the last commands you entered, Bash simply looks there (Figure 2).

Timestamp It!
With a small change, Bash not only logs typed commands, but even adds the date and time when the commands were entered. To do this, you need to set the following environment variable (preferably in the .bash_profile
init file):
export HISTTIMEFORMAT="%F %T: "
From then on, Bash adds a comment line with the Unix epoch stamp before each command recorded in the .bash_history
logfile, while the history
command prints the human readable date and time for each command in the display list (Figure 3).

This newly discovered data treasure trove begs for some kind of evaluation, of course. How about determining the days of the week on which a user was most diligent in terms of typing commands? Or how about identifying the most frequently typed commands in order to find new ways to reduce the future typing overhead by using abbreviations or shell scripts?
Callback as an Abstraction
Listing 1 [1] defines the histWalk()
function, which finds the global history logfile for a user in the user's home directory, browses it line by line, and calls a user-provided callback function for each command entry. In this way, other analysis programs can define their own callbacks, pass it to histWalk()
, and receive the history data including timestamps this way, without having to worry about the details of the history logfile's location or format.
Listing 1
histWalk.go
01 package main 02 03 import ( 04 "bufio" 05 "os" 06 "os/user" 07 "path/filepath" 08 "strconv" 09 ) 10 11 func histWalk(cb func(int64, string) error) error { 12 usr, err := user.Current() 13 if err != nil { 14 panic(err) 15 } 16 home := usr.HomeDir 17 histfile := filepath.Join(home, ".bash_history") 18 19 f, err := os.Open(histfile) 20 if err != nil { 21 panic(err) 22 } 23 24 scanner := bufio.NewScanner(f) 25 var timestamp int64 26 for scanner.Scan() { 27 line := scanner.Text() 28 if line[0] == '#' { 29 timestamp, err = strconv.ParseInt(line[1:], 10, 64) 30 if err != nil { 31 panic(err) 32 } 33 } else { 34 err := cb(timestamp, line) 35 if err != nil { 36 return err 37 } 38 } 39 } 40 return nil 41 }
The logfile opened for reading in line 19 by os.Open()
is read line-by-line by the scanner created in line 24. To do so, it grabs the reader interface of the opened file, which is offered via the File
structure, and takes in a new line with every call to Scan()
; scanner.Text()
then returns the text line as a string.
If the line starts with a comment character, it is a timestamp for a command in the subsequent log line. Accordingly, Listing 1 stores the seconds field in the timestamp
variable (after cutting off the leading comment character) and sends the scanner off into the next round. If there is no hash sign at the beginning, it is a command line, and the program jumps to the else
branch starting in line 34, where it calls the callback function entered by the user. As parameters, it passes in the previously saved timestamp and the currently read shell command.
First Class Functions
In Go, functions are first class citizens: They can be assigned to arbitrary variables or passed into other functions in parameter lists. For example, user programs can use the histWalk()
function offered by Listing 1, assign it a callback function that is completely adapted to their special needs, and let it fill existing data structures in the local scope with the results.
The program logic should be easy peasy moving forward. Because of Go's strict typing, however, converting the previously read time value in seconds into an integer can try your patience, because the scanner reads it as a string. The ParseInt()
function from the standard strconv package expects the string to be parsed as its parameter (line[1:]
in line 29 cuts off the first letter – that is, the comment character), as well as the base of the integer number (10
for a decimal number) and the maximum number of bits (64
). Back comes a int64
type value; this is an important detail: Otherwise the lights would go out on January 19, 2038, because by that date the supply of 32-bit integers for the seconds is exhausted.
One use for the histWalk()
function is shown in Listing 2, which runs an analysis of the user's typing activities, broken down for the days of the week (Figure 4). Since the timestamp (stamp
) passed to the callback is an integer, the standard time.Unix()
function called in line 12 easily converts it to the Go internal time.Time
format for time and date values. The Weekday()
function called with the converted value then determines the day of the week for the given date. The int()
wrapper converts the value that exists as a structure in Go's standard time
package into an integer between
(Sunday) and 6
(Saturday).
Listing 2
dow.go
01 package main 02 03 import ( 04 "fmt" 05 "time" 06 ) 07 08 func main() { 09 var countByDoW [7]int 10 11 err := histWalk(func(stamp int64, line string) error { 12 dt := time.Unix(stamp, 0) 13 countByDoW[int(dt.Weekday())]++ 14 return nil 15 }) 16 17 if err != nil { 18 panic(err) 19 } 20 21 for dow := 0; dow < len(countByDoW); dow++ { 22 dowStr := time.Weekday(dow).String() 23 fmt.Printf("%s: %v\n", dowStr, countByDoW[dow]) 24 } 25 }
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you've found an article to be beneficial.
News
-
Arch Linux 2023.12.01 Released with a Much-Improved Installer
If you've ever wanted to install Arch Linux, now is your time. With the latest release, the archinstall script vastly simplifies the process.
-
Zorin OS 17 Beta Available for Testing
The upcoming version of Zorin OS includes plenty of improvements to take your PC to a whole new level of user-friendliness.
-
Red Hat Migrates RHEL from Xorg to Wayland
If you've been wondering when Xorg will finally be a thing of the past, wonder no more, as Red Hat has made it clear.
-
PipeWire 1.0 Officially Released
PipeWire was created to take the place of the oft-troubled PulseAudio and has finally reached the 1.0 status as a major update with plenty of improvements and the usual bug fixes.
-
Rocky Linux 9.3 Available for Download
The latest version of the RHEL alternative is now available and brings back cloud and container images for ppc64le along with plenty of new features and fixes.
-
Ubuntu Budgie Shifts How to Tackle Wayland
Ubuntu Budgie has yet to make the switch to Wayland but with a change in approaches, they're finally on track to making it happen.
-
TUXEDO's New Ultraportable Linux Workstation Released
The TUXEDO Pulse 14 blends portability with power, thanks to the AMD Ryzen 7 7840HS CPU.
-
AlmaLinux Will No Longer Be "Just Another RHEL Clone"
With the release of AlmaLinux 9.3, the distribution will be built entirely from upstream sources.
-
elementary OS 8 Has a Big Surprise in Store
When elementary OS 8 finally arrives, it will not only be based on Ubuntu 24.04 but it will also default to Wayland for better performance and security.
-
OpenELA Releases Enterprise Linux Source Code
With Red Hat restricting the source for RHEL, it was only a matter of time before those who depended on that source struck out on their own.