Use AI and Go to program a command-line predictor
Programming Snapshot – Smart Predictions with Go

© Lead Image © alphaspirit, 123RF.com
Because shell command sequences tend to reoccur, smart predictions can save you time typing. We first let the shell keep notes on what gets typed, before a Go program guesses the next command and runs it for you.
When I'm developing new Snapshot articles, I regularly catch myself typing the same commands in the terminal window time and time again. Text or code files modified by vi
are sent to a staging area by git add foo.go
, git commit
feeds them to the local repository clone, and git push origin v1:v1
backs them up on the server. New builds of the Go source code in programming examples are triggered by the go build foo.go bar.go
command, before tests are run by go test
, and so on. Excessive typing like this needs to be automated. Because software development dinosaurs like myself keep fighting IDEs, I need a homegrown approach.
Although the shell history will find old commands, locating the command you need in this massive list, and running it again, requires some manual work. This is rarely worthwhile because retyping is often quicker than browsing 10 entries up the list or using a search string. The key is that you normally type shell commands in a defined order. For example, vi
edits a Go file, then git
saves the results, and go build
compiles them. Learning this context, a smart tool would be quite capable of determining what comes next. Also the command sequences I use seem to depend on the directory in which I run them. In a Go project, I use the commands I listed earlier. For a text project, I would possibly use others, such as make publish
to generate HTML or PDF files.
If a tool had access to the historical sequence of commands I issued in the past, and of the directories in which I ran them in, it could offer a good preselection of the commands likely to follow. In 90 percent of the cases, users would be able to find the next command and run it again. A dash of artificial intelligence accelerates and improves the whole thing, too. Figure 1 shows an example of a flowchart for a shell session. The edges in the graph mark the transitions between the commands and the percentages next to them the probability – derived from the history file – of a certain transition taking place. All paths originating from a state therefore add up to 100 percent.

Logger and Predictor
To analyze which command sequences the user has typed in the shell so far, I first need a process to continuously log every single manually typed command. The Bash or Z shell (Zsh) history
mechanisms are not suitable for this, because they at best record the commands themselves along with a timestamp [1]. For the predictor, however, I at least want the tool to include the directory in which the command was run for useful suggestions to be generated later.
The newer Zsh offers a preexec()
hook for general interception of a typed command. I assigned a function body to the hook in line 4 of Listing 1. The shell always triggers it just before executing a command line and passes the contents of the command line to it as a string in the first parameter. My preexec()
hook in turn calls the cmdhook()
function defined directly before it. It strings together the current time and directory, adds the command line after this, separates the three components with spaces, and appends the results as a new line at the end of the myhist.log
file in my home directory. Listing 2 shows some entries that accumulated there after I spent some time writing this article.
Listing 1
zshrc.sh
Listing 2
myhist.log
Line 5 in Listing 1 defines the shell function g()
, which I'll call later to receive suggestions from the shell for the next command to execute. I wanted the command to be just one letter in length in order to avoid typing, and "g" makes sense if you're programming in Go.
After setting g()
in motion with the g
command followed by the Enter key, the shell function calls the pick
command (line 6). This is a Go program (which you can see starting in Listing 4) that scans the myhist.log
file, using an algorithm to decide on a list of the most likely commands to follow the last one.
Listing 3
bashrc.sh
Listing 4
history.go
From the list of likely commands, the user needs to select the desired command using the arrow keys (or the vi
mappings J and K) and then press the Enter key (Figure 2). The shell then executes the selected command directly – it could hardly be more finger-friendly. To do so, the shell function in g()
fields the command string returned by pick
and executes it with the built-in eval
function.

Teaching a New Dog Old Tricks
Using an old trick from a Snapshot column three years ago [2], the compiled Go program (pick
) outputs the user menu to Stdout
(file descriptor number 1
, because the promptui Go library I used can't do it any other way) and lets the user pick an item. It finally outputs the choice to Stderr
(file descriptor number 2
), which the g()
shell function in Listing 1 then receives. The wacky 3>&1 1>&2 2>&3
construction (in line 6) redirects Stderr
(number 2
) back to Stdout
(number 1
), so that the command line to be executed ends up in the shell cmd
variable (line 6). Last but not least, eval
then takes the variable and executes the string it contains (line 8).
Figure 2 shows the predictive shell tool in action. For historical reasons, I still write articles in the plain new documentation (PND) format, which borrows slightly from Perl's plain old documentation (POD) format. After editing the article text in t.pnd
, I call g
, which offers the most likely subsequent commands for selection based on the shell history gleaned from myhist.log
. These commands include git add
for the text file, make
(an alias named m
for me) to generate an article from it, a re-edit of the file with vi
, and finally the command
git add -p .
that I often use to interactively promote modified file contents to the staging area.
However, instead of Zsh, Linux distributions traditionally tend to use Bash, which does not offer the preexec()
hook used by my new logging component. Lucky for me that someone on GitHub has gone through the trouble of porting this eminently useful function to Bash [3]. As the first step, I installed the shell script stored on GitHub. To run it on new logins to the shell, I inserted the line from Listing 3 into the .bash_profile
file. After checking it's there, the second step involves loading the .bash-preexec.sh
script and running it.
The algorithm that predicts what is likely to be the next user command learns from the sequence of previously entered shell commands that the preexec()
hook has written to myhist.log
. Listing 4 iterates through the logfile in the history()
function, creating a HistEntry
type entry from each line. This structure, defined in line 8, contains an attribute for the Cmd
and Cwd
fields, which are the command entered by the user and the directory where the shell was located when this happened, respectively.
In the for
loop that starts in line 21, the scanner from the bufio package loads the logfile lines, ignores the timestamp in the first column, and checks whether the command in the third column looks okay. The loop also ignores all commands that only consist of the g
shortcut; although preexec
logs this too, the predictor runs aren't going to help the oracle with its predictions.
If an empty command makes its way into the logfile (e.g., because the user quit prediction mode by pressing Ctrl+C), continue
skips the line in question. The history()
function adds valid entries to the end of the hist
array slice as HistEntry
type variables, which return hist
finally returns to the caller in line 36.
Memory Aid
Based on historic data, the predictor in Listing 5 now runs the predict()
function for the current directory (cwd
) to guesstimate the next command the user will probably want to run. It fields the array slice with the HistEntry
structures and iterates through them in the for
loop starting in line 8.
Listing 5
predict.go
In each round, the predictor stores the shell command currently processed, which lies in h.Cmd
, and saves it in the lastCmd
variable, so that the next round of the loop can access the previous value. Starting in the second round, the code saves information about which command followed which previous one in a two-level hash map named followMap
starting in line 16 and increments the associated integer value. In other words, at the end of the for
loop, the program knows how often command B followed command A. Accordingly, the algorithm evaluates the probability of command B following command A.
If there is only a single command for the current directory in the logged history, the algorithm cannot do much and takes the diplomatic approach of suggesting ls
in line 26. However, if followMap
lists some commands that usually follow the preceding command stored in lastCmd
, the algorithm dumps each of those subsequent commands into a structure with a counter that reflects their frequency. It then uses sort.Slice()
to sort an array slice of these structures in descending order by the counter, starting in line 47. Sorting a hash map like this by its numeric values would be a snap in a scripting language such as Python, but Go requires significantly more overhead because of its strict type checking.
The output, at the end of the predict()
function, is the items
variable – an array slice containing the commands that, based on their order, are most likely to follow the current shell command. Finally, the pick
program in Listing 6 offers them up to the user.
Listing 6
pick.go
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you've found an article to be beneficial.
News
-
Fedora 39 Beta is Now Available for Testing
For fans and users of Fedora Linux, the first beta of release 39 is now available, which is a minor upgrade but does include GNOME 45.
-
Fedora Linux 40 to Drop X11 for KDE Plasma
When Fedora 40 arrives in 2024, there will be a few big changes coming, especially for the KDE Plasma option.
-
Real-Time Ubuntu Available in AWS Marketplace
Anyone looking for a Linux distribution for real-time processing could do a whole lot worse than Real-Time Ubuntu.
-
KSMBD Finally Reaches a Stable State
For those who've been looking forward to the first release of KSMBD, after two years it's no longer considered experimental.
-
Nitrux 3.0.0 Has Been Released
The latest version of Nitrux brings plenty of innovation and fresh apps to the table.
-
Linux From Scratch 12.0 Now Available
If you're looking to roll your own Linux distribution, the latest version of Linux From Scratch is now available with plenty of updates.
-
Linux Kernel 6.5 Has Been Released
The newest Linux kernel, version 6.5, now includes initial support for two very exciting features.
-
UbuntuDDE 23.04 Now Available
A new version of the UbuntuDDE remix has finally arrived with all the updates from the Deepin desktop and everything that comes with the Ubuntu 23.04 base.
-
Star Labs Reveals a New Surface-Like Linux Tablet
If you've ever wanted a tablet that rivals the MS Surface, you're in luck as Star Labs has created such a device.
-
SUSE Going Private (Again)
The company behind SUSE Linux Enterprise, Rancher, and NeuVector recently announced that Marcel LUX III SARL (Marcel), its majority shareholder, intends to delist it from the Frankfurt Stock Exchange by way of a merger.