DynamicIsearchFiltering

This page is about using Isearch+, that is, library isearch+.el, to dynamically add and remove any number of Isearch filter predicates (aka search filters) while searching incrementally. This Isearch+ feature is available starting with Emacs 24.4.

The predicate that is the value of ‘isearch-filter-predicate’ is advised by additional predicates that you add, creating a complex suite of predicates that act together.

This lets you search only given zones of text, where those zones can be defined on the fly in complex ways that are easy to specify.


Note: This is different from the feature provided by companion Isearch+ library isearch-prop.el, which also lets you constrain search to zones of text. When you use that library you first define the zones, and then search is limited to them. With dynamic filtering, search is not limited to certain zones ahead of time. Instead, after a search hit is found the filters are applied to see if it should be retained as a search hit. This is a posteriori filtering, whereas isearch-prop.el uses a priori filtering. Each has its advantages.
You can use the dynamic filtering described here at the same time as the zone-constraining provided by isearch-prop.el - you can use them together.

Isearch Filter Predicates

An Isearch filter predicate is a function that accepts two buffer positions, BEG and END, as its first two arguments. These values are the beginning and ending positions of a search hit. If the return value of the function is ‘nil’ then the search hit is excluded from searching; otherwise it is included.

The value of standard variable (but not a user option) ‘isearch-filter-predicate’ is the filter predicate used by Isearch. By default, the value is predicate ‘isearch-filter-visible’, which returns non-‘nil’ for any search hit that is visible (not rendered invisible by a text property, overlay property, etc.)

Using Filter Predicates Easily

If you search the Emacs Lisp source code, you will find only two uses, so far, of variable ‘isearch-filter-predicate’, even though such filtering has been around since Emacs 23. It’s hardly ever used. Why?

Because it’s not so easy to use, out of the box. And it’s not thought of as a way to refine searches, but rather as a way to wall off certain areas from searching.

Yes, those are in fact the same thing, but I don’t think people think this way . . . because Isearch does not make it particularly easy to use filters. Isearch+ tries to do that, to let you refine searches by adding filters incrementally.

The idea is simple: Isearch+ defines some keys that prompt you for a filter. You can enter any filter predicates at the prompts. There are also some predefined predicates that you can choose from, using completion. You can combine predicates using AND, OR, and NOT.

A Filter Predicate Can Help You Search

A filter predicate does essentially the same thing as the search pattern that you type at the Isearch prompt. Each restricts the search space (the buffer text) to certain zones: those that satisfy the predicate and those that match the search pattern.

But a predicate can be much more general than is the predefined pattern-matching provided by Emacs Isearch.

Suppose that you want to find lines of text that contain ‘cat’, ‘dog’, and ‘turtle’. There is no simple search pattern that lets you do this. A regexp would need to explicitly express each possible order, and there are 6 of them - not so simple.

But a predicate can just check each line for ‘cat’ AND check for ‘dog’ AND check for ‘turtle’. It is usually much easier to combine simple patterns than it is to come up with a complex pattern that does the same thing. And the way to combine patterns in Emacs Isearch is to use one or more filter predicates.

A Filter Predicate Can Perform Side Effects

A filter predicate can even perform side effects, if you like. Only the return value is used by Isearch. For example, if you wanted to more easily see the cursor position each time search stops at a search hit, you could use something like this as a filter predicate. (This requires library crosshairs.el, which highlights the current column and line using crosshairs.)

    (lambda (beg end)
       (save-excursion (goto-char end)) ; Go to end of search hit.
       ;; Avoid calling `crosshairs' when inside `isearch-lazy-highlight-search'.
       (unless isearchp-in-lazy-highlight-update-p (crosshairs))
       t)  ; Return non-nil always - no real filtering.

The side-effect-producing call to function ‘crosshairs’ is guarded by variable ‘isearchp-in-lazy-highlight-update-p’ here, so that it is invoked only when the cursor is moved to a search hit, not also when lazy highlighting is performed. (Filtering applies also to lazy highlighting: it filters out search hits that are not being used. But in this case no real filtering is done, and there is no need to show crosshairs moving across the buffer during lazy highlighting.)

(You can choose that crosshairs-showing filter predicate by the name ‘crosshairs’ when prompted for a predicate. It corresponds to predicate ‘isearchp-show-hit-w-crosshairs’.)

What You Can Do

Here are some of the things you can do using dynamic filtering:

    1. Search for lines: `C-M-s .+’.
    2. Refine that search to lines that contain a match for ‘cat’: `C-z . cat’.
    3. Do the same for ‘dog’, and then ‘turtle’ (`C-z . dog’, then `C-z . turtle’).

Filtering Commands Available During Search

The following commands are available during Isearch. They are all on prefix key ‘C-z’, by default. They are on prefix keymap ‘isearchp-filter-map’, which you can bind to any key in ‘isearch-mode-map’. If you forget a ‘C-z’ key, you can use ‘C-z C-h’ while searching to show them all.

         (defun foo ()
           "Isearch with filter predicate `my-filter-pred'."
           (interactive)
           (let ((isearch-filter-predicate  'my-filter-pred))
             (isearch-forward)))

Typically you add (`C-z &’, `C-z %’, etc.) a filter predicate to those already active, or you remove one (‘C-z -’). Adding is implicitly an AND operation: the list of current predicates must all be satisfied. You can also OR a predicate against either the entire ANDed list of predicates (`C-z ||’) or against only the last-added one (`C-z |1’). And you can complement either the entire ANDed list (`C-z ~~’) or just the last-added predicate (`C-z ~1’).

This ORing and NOTing, together with adding and removing predicates in a given order (implicitly ANDing them), gives you complete Boolean combination flexibility.

The list of filter predicates is always a conjunction. But you can use, as any of the conjuncts, a predicate that implements a disjunction or a negation. Or you can replace the entire list by a single predicate that implements a disjunction or a negation.

When you use one of the commands that adds a filter predicate as advice to ‘isearch-filter-predicate’ you can be prompted for two things: (1) a short name for the predicate and (2) text to add to the Isearch prompt as a reminder of filtering. The optional short name is a convenience for referring to the predicate - for adding it again or removing it, for example.

Two user options control this prompting:

In addition, whatever the value of these options, when you add a filter predicate you can override the option values by using a prefix argument. A non-positive prefix arg overrides the option for name prompting, and a non-negative prefix arg overrides the option for prompt-prefix prompting. (So zero, e.g., ‘M-0’, overrides both.)

Option ‘isearchp-show-filter-prompt-prefixes-flag’ controls whether prefixes for filters are added to the Isearch prompt. You can toggle this option during search using ‘C-z p’.

User option ‘isearchp-filter-predicates-alist’ contains filter predicates that are available as completion candidates whenever you are prompted for one. This is an important option. The alist entries can be of several forms, which affect the behavior differently.

In particular, instead of choosing a filter predicate as a completion candidate, you can choose a function that creates and returns a filter predicate, after prompting you for some more information.

This is the case, for example, for function ‘isearchp-near-before-predicate’. It is used in the predefined alist entry `("near<..." isearchp-near-before-predicate)’, which associates the short name `near<...’, as a completion candidate, with the function.

When you choose this candidate, function ‘isearchp-near-before-predicate’ prompts you for another pattern for Isearch to match, a max number of units of nearness, and which units to measure with. It constructs and returns a predicate that checks those match parameters. As usual, you can be prompted for a short name and an Isearch prompt prefix to associate with the newly defined predicate, so that you can easily choose it again (no prompting).

Similarly, candidate `not...’ prompts you for a predicate to negate, and candidate `or...’ prompts you for two predicates to combine using ‘or’.

For the completion candidates that are predefined, this naming convention is used:

Filter predicates that you add dynamically are added as completion candidates for the current Emacs session. If option ‘isearchp-update-filter-predicates-alist-flag’ is non-‘nil’ then they are also added to ‘isearchp-filter-predicates-alist’. That updated option value is NOT SAVED, however. If you want to save your additions to it for future Emacs sessions then use ‘M-x customize-option isearchp-filter-predicates-alist’.

You can use command ‘isearchp-reset-filter-preds-alist’ (not bound) to reset the filter predicates available for completion to those in option ‘isearchp-filter-predicates-alist’. A prefix arg with ‘C-z 0’ also resets this, along with resetting to the unadvised value of ‘isearch-filter-predicate’.

If option ‘isearchp-lazy-dim-filter-failures-flag’ is non-‘nil’ then search hits that are skipped because they are removed by filtering are nevertheless lazy-highlighted, but using a face that dims the background. You can toggle this highlighting of filter-failure search hits using ‘M-s h d’ (command ‘isearchp-toggle-dimming-filter-failures’).

The dimming face for this is hard-coded as having background color #9abfca, unless you also use library isearch-prop.el (recommended). If you use isearch-prop.el then you can control the dimming color using option ‘isearchp-dimming-color’. It specifies a given background color to use always, or it specifies that the current background color is to be dimmed a given amount.


See Also: Isearch+


CategorySearchAndReplace