Indexing Web Pages?

It is possible, and in fact quite easy, to use Recoll to index and search the web pages you visit. First, download and install the Firefox add-on called Recoll WE [5]. Second, enable web indexing from the GUI Index configuration panel, or by setting the processwebqueue variable to 1 in the configuration file. After that, one click on the Recoll WE button in Firefox will save a complete copy of the current page in a folder, where Recoll will later find, copy, and index it. The only limitation of this method is that Recoll stores all the copies in a cache of fixed size. No matter how big you make it, if that cache fills up, Recoll will automatically remove the older files it contains. You must explicitly archive any web page you want to keep indefinitely.

Index Maintenance

The command-line program that manages the actual indexing is called recollindex. You can launch recollindex from the File menu of the GUI, from the command line, or from a cron job. There are two ways, accessible from the Preferences | Indexing schedule panel, to keep an index up to date. One way is to make recollindex run all the time in the background, indexing and re-indexing files as soon as they are created or modified. This approach may slow down your computer, so avoid it unless you really, really need an index that is as up to date as possible.

In most cases, it is much better to only run recollindex at regular intervals, possibly every few days. By default, this program will analyze and index only the files that change after its previous run. You can, however, rebuild the whole index in any moment – from the GUI or by passing the -z option to recollindex. To update a different index, use the -c option to show recollindex the path to the corresponding configuration directory.

Basic and Advanced Search

Running basic searches in Recoll could not be any simpler: Enter the search term in the top text box, click the Search button, and browse the results (Figure 2). The Advanced Search panel (reachable from the Tools menu) and the web interface (Figure 3) can filter results in many ways, based on keywords or the type, age, and size of the file. All options are quite easy to understand by trial and error, so I will focus on an even more powerful feature: the language by which Recoll understands and can perform very complex queries. Once you know how to write a query, you can pass it to the recollq command line utility or enter it in the text box of the desktop interface. You'll need to set the search mode to Query language.

Figure 3: The Recoll web interface is almost as easy to use as the desktop one, and it provides the same option to save the results of any search as a spreadsheet.

Figure 2: The Recoll search result window: a clean listing, with buttons to preview or open every file discovered, as well as a show query link to check what Recoll did.