Use fzf and fzy to add fuzzy search tools to the shell
Near Miss
Fuzzy finders retrieve useful results from data streams even if there are no exact matches.
Today, fuzzy searches are an integral part of everyday IT life. They correct typos, detect similarities, and offer a way to find what you need with reasonable overhead, even in unstructured data. The basic principle is based on the Levenshtein distance [1], word distances defined in the 1960s (see the "Levenshtein" box). Developers have extended and optimized this algorithm more or less from the start.
Levenshtein
The Levenshtein distance describes the minimum number of changes (replace, delete, insert) needed to transform one string into another. The weighted number of these actions determines how similar the two strings are. Weighting means that, for example, two swapped letters (like in a typo) are less important than, say, inserting additional characters. But the approach also involves a number of problems. Weighting depends on the concrete task (spell check, search, etc.) and the language.
Having said this, many standard tools ignore fuzzy searches or limit themselves to highly simplified variants up to this day. For example, the grep tool from the coreutils package processes arbitrarily complex regular expressions (patterns), but it does not support fuzzy searching. Fuzzy searching is supported by agrep [2] and ugrep [3] at the command line, and there are several other, less well-known tools. Agrep impresses here with a best-match option.
Fzf [4] and fzy [5] (see the "Little Brother" box) enrich the shell construction kit, adding two new, powerful tools. They act as interactive filters, i.e., by default, they source their data from the standard input and return the results to the standard output. Interactive in this context means that they provide interfaces for entering the search patterns, which allow the search patterns to be adapted and refined at runtime.
Little Brother
Fzf is already in use as a standard tool on many systems today. According to its developer, the enhanced fzy tool offers an improved algorithm and a faster search speed. In addition, fzy limits itself to the bare essentials in terms of options, and this simplifies the application. At first glance, fzy really only looks like a variant of fzf, but this is deceptive. Although the output is similar, fzy can at most be considered fzf's little brother. It is also piped, but there is also a very simplified Finder, which gives users a kind of menu with a configurable prompt. But this already exhausts the list of similarities. This becomes particularly clear if you look at fzy's options. By default, its Finder uses only 10 lines of the terminal to display the list of results. This can be changed with the -l
option. Fzy can only work with positive patterns; it does not support exclusions. In interactive mode, only a handful of keyboard shortcuts are available. They are listed by the tool's fairly concise man page.
fzf
Fzf stands for Fuzzy Finder. The tool acts as a general-purpose filter, which also acts as a menu generator in interactive mode. Calling fzf
launches the tool in the shell. This normally just displays all the files in the current directory. Fzf uses a built-in find
command to do this, which is the default if the program is not given any data via the standard input. This feature helps to keep the commands short. You can manage this behavior via the FZF_DEFAULT_COMMAND
environment variable, specifying a different command if needed. You can store the options that you want to pass in to fzf as defaults in the FZF_DEFAULT_OPTS
environment variable.
Most of the time, the program is used in conjunction with a pipe. This means that it receives its input data via the standard input (Listing 1). This takes you to interactive mode, which consists of two parts. The visible interface, the Finder, supports interactions: You can move around with the arrow keys or by pressing Ctrl+K and Ctrl+J. The terminal displays the current line in bold or inverse type depending on the settings. Pressing the Enter key accepts the selected line and sends it to the standard output or to the terminal.
Listing 1
locate and fzf via Pipe
$ locate / | fzf [...] .../Images-sda5/_1130192.png.out.pp3 .../Images-sda5/_1130192-1.png.out.pp3 .../Images-sda5/_1110520.jpg.out.pp3 .../Images-sda5/_1090399.dng.xmp .../Images-sda5/_1090399.dng.pp3 .../Images-sda5/_1040511a.pts .../Images-sda5/_1040511.pts .../Images-sda5/_1040511-6.pts .../Images-sda5/DANCE!-2020-2021 [...]
In addition, a prompt (>
) is displayed at the bottom edge of the Finder, prompting you for input. If you already passed a search pattern into fzf when you called it (as an argument of the -q
option), the search pattern can be edited in this line. The second part works in the background, managing the index and the fuzzy search. In the second-to-last line in Figure 1, the tool shows you what is happening in this regard. A constantly changing symbol shows that it is creating an index from the input, which it will then search through for the pattern. The current size of this index is also shown at the bottom.
Search Pattern
The Finder is an essential feature of fzf. However, the tools's capabilities are not limited to editing existing search patterns. It also evaluates the patterns incrementally. Each additional character entered refines the search. In Extended Search mode, which is enabled by default, you can also enter additional patterns separated by spaces. The program assigns special meanings to some of the special characters used in these patterns (see Table 1). You can combine several search patterns as described in Listing 2.
Listing 2
Combining Search Patterns
Pattern1$ Pattern2 ^Pattern3 !Pattern4
Table 1
Searches for Power Users
Character | Example | Result |
---|---|---|
|
|
Lines that start with |
|
|
Lines that end with |
|
|
Lines that do not contain |
|
|
Lines that contain precisely |
|
– |
Token for fuzzy search (default) |
You can use the arrow keys to navigate within the search pattern in the Finder. The highlighted line shows the selection. Alternatively, some default key mappings are available (see Table 2). All actions involving selecting or deselecting of multiple lines require you to launch the tool in multiple-selection mode with the -m
option. On top of this, the Finder also supports mouse control. Scrolling with the mouse wheel, clicking, and double-clicking lets you enable and select. If you hold down the Shift key, a mouse click selects multiple items. A double-click ends the input.
Table 2
Controls
Key | Explanation |
---|---|
Ctrl+K |
Move up |
Ctrl+J |
Move down |
Ctrl+C |
Cancel |
Alt+F |
Jump one word to the right in the search pattern |
Alt+B |
Jump one word to the left in the search pattern |
Ctrl+A |
Jump to the start of the search pattern |
Ctrl+E |
Jump to the end of the search pattern |
Tab |
Select multiple consecutive lines |
Shift+Tab |
Deselect the following selected lines successively |
Ctrl+I |
Select or deselect arbitrary lines |
Otherwise the program works with key bindings adapted to Emacs by default. Additional or deviating key bindings can be set with the --bind
option. The -e
command-line option disables the fuzzy search, returning only exact hits. Other important fzf options are summarized in Table 3.
Table 3
Important Options
Option | Explanations |
---|---|
|
Display only exact matches, disable fuzzy search |
|
Case (in)sensitive |
|
Select algorithm: |
|
Disable search, only provide interface |
|
Do not sort output |
|
Reverse order of input |
User Interface Options |
|
|
Enable/disable multiple selection; default, return only one match |
|
Continue search before the first/after the last entry |
|
Height of Finder as a percentage of the terminal size |
|
Start software with |
|
Do not start Finder for a single match |
|
Do not start Finder if there is no match |
|
Use |
|
Define form, size, and type of preview |
You can define how the program determines and evaluates matches with a number of parameters. The input lines are evaluated on the basis of the criteria from Table 4. index
can only occur at the end. Fzf uses this option by default if you do not specify anything. Fzf evaluates the inputs using these criteria, which you explicitly select as arguments for the --tiebreak=CRITERION
option. By default, it uses length
or length,index
. Stipulating end
tells the software to evaluate the lines in reverse order.
Table 4
Modifiers
Option | Explanation |
---|---|
|
Prefer shorter lines with matches |
|
Prefer matches at the start of the line |
|
Prefer matches at the end of the line |
|
Prefer lines nearer to the start of the input stream |
One typical fzf application is a fault-tolerant variant of the locate
command. The latter searches for files on the local system by reference to the filenames in a previously created database. This search is non-fuzzy; it does not return results in case of typos or abbreviations. In the example in Listing 3, locate /
creates a list of all entries and returns it to fzf via a pipe. This stipulates all the arguments passed in as patterns for the command. If you pass in L
for no arguments, the entire list appears, and you can enter patterns directly in fzf's interactive mode.
Listing 3
Fuzzy Search with locate
L () { locate / | fzf -q "${*}" ; }
If you have a very large database, you will probably want to exclude many matches to improve the clarity of the results. You can do this by passing in arguments to the function. In Listing 4, the pattern searches for something
but excludes butnotthat
. If the search still returns too many matches, such as in randomly-named browser caches, more additional patterns in interactive mode will lead to better results. At the command line, you must quote the exclamation mark in front of the excluding pattern with a backslash (/!
) or use quotes around longer sequences. Otherwise, the shell will interpret the character as a history
command and remove it. In interactive mode, \!
has a different meaning.
Listing 4
Excluding Matches
> L something \!butnotthat [...] .../Music/The Temptations - Papa was a rolling stone.mp3.mp3 .../Images/png/2-watermark.png .../Music/Frumpy - How the Gypsy was Born.mp3 .../Music/2/Frickle - Some People, Follow Me.mp3 .../Music/Byron Metcalf - Heart Warriors+.mp3 .../Images/aerial image no channel water border.png [...]
Fzf supports a sophisticated preview mode for the matches. The argument in the --preview
option is the command line for generating the preview. This is shown for the file currently selected with the cursor in a separate window that Fzf generates on the right (Figure 1). You can only use text output in a terminal, which means that, for many files, the metadata or the previously converted or extracted text contents appear as a preview instead of the binary data.
less
is really useful as a universal previewer (Listing 5). After all, the pager is particularly well suited for displaying different file types with lesspipe [6]. You just need to specify --preview 'less {}'
as the command. This is all it takes to display the contents of text files, for example, but also to archive content or the metadata of multimedia files, assuming that lesspipe is installed correctly.
Listing 5
Match Preview
$ fzf --preview 'PreviewCommand'
You can define both the preview window's position and size using --preview-window POSITION
. The main arguments for this option are the position (top
, bottom
, left
, right
), the size (as a percentage of the terminal width), and the mode (wrap
, cycle
, hidden
). You need to enter this information in a colon-separated format without any spaces (Listing 6).
Listing 6
Preview Position
$ fzf --preview-window=top:55%:wrap:cycle
In addition to the preview, there is another way to generate a view of the content via actions. In the program, Actions means a whole group of functions that you can enable in the interactive Finder. You can also trigger Actions through events. The only events fzf can currently handle are change
, if the search query changes, and backward-eof
if the search query is empty and you still try to delete it using the backspace key. The change event is often used in combination with first
(jump to first hit); abort
is the default for backward-eof
.
Key Bindings
One of fzf's special features is the ability to call other programs within the running program. This feature, which is controlled by the execute
action, offers a wide range of possibilities. For example, you can open selected files to view or edit them. Fzf expects a command-line argument for the execute
action. It then executes the command line in a subshell. The {}
string stands for the selected files.
The --bind
option is used to bind actions to a key or keyboard shortcut. You can use more than one action for a key binding. This is demonstrated by an example from the man page (Listing 7): The two forms are equivalent.
Listing 7
Defining Key Bindings
fzf --multi --bind 'ctrl-a:select-all+accept' fzf --multi --bind 'ctrl-a:select-all' --bind 'ctrl-a:+accept'
Creating your own definitions makes it possible to adapt the way fzf works to match other programs or even to add new functions. To group the --bind
arguments, use single or double quotes if there are any spaces.
The current version of fzf supports only a subset of keys for bindings (see Table 5). The program ignores other combinations, such as Shift+Enter, which outputs a warning such as un-Supported key: shift-enter. But the software does accept mouse clicks in combination with the up arrow, down arrow, and double-clicks. Many actions simply delete characters or jump to certain positions in the line. Table 6 summarizes other, more specialized functions. Most of them are not normally mapped to keys, so they are not available by default.
Table 6
Actions
Action | Explanation |
---|---|
|
Apply the current entry |
|
If an entry is selected, apply it |
|
Change the Finder prompt |
|
Deselect the previously selected entry |
|
Display preview |
|
Refresh preview |
|
Select all |
|
Deselect all |
|
Toggle all matches |
|
Find toggles |
|
Show (hide) preview window |
|
Toggle sorting |
|
Disable search |
|
Reload and rebuild index |
|
Replace search with current selection |
|
Use |
Table 5
Supported Keys
Group | Keys |
---|---|
Single keys |
Enter, Space, Tab, Esc, Del, Up Arrow, Down Arrow, Right Arrow, Left Arrow, Pos1, End, Ins, F1 to F12, A to Z, \, ] |
Ctrl combinations |
Space, Shift+7, A to Z |
Alt combinations |
Space, Up Arrow, Down Arrow, Right Arrow, Left Arrow, Backspace, A to Z |
Shift combinations |
Up Arrow, Down Arrow, Right Arrow, Left Arrow, ^, Space, Tab |
Three-key combinations |
Ctrl+Alt+A to Ctrl+Alt+Z, Alt+Shift+Up Arrow, Alt+Shift+Down Arrow, Alt+Shift+Right Arrow, Alt+Shift+Left Arrow |
There are two ways to execute external commands. If you use the execute
action, fzf executes the command line, waits until the end of the command, and then displays its output. In many cases, this is precisely the desired behavior. To prevent fzf from waiting – and thus not accepting further input while doing so – you can append an ampersand to run the executed commands in the background. If you don't need the output of the executed commands, use the execute-silent
action. The program will then run the commands but discard the output.
Buy this article as PDF
(incl. VAT)