Icicles - Search Commands, Overview

iciclesimage
Previous: Icicles - Isearch EnhancementsIciclesIciclesIndexNext: Icicles - Search-And-Replace

Icicles Search Commands, Overview

This section provides an overview of Icicles search.

See also:

Introduction: On Beyond Occur...

You’ve no doubt used standard Emacs command ‘occur’ (OccurMode). It finds all lines in a buffer that match a regexp that you enter. It displays the matching lines as links in buffer ‘*Occur*’ – you can click a link to navigate to the corresponding line in the original buffer. Using buffer ‘*Occur*’ is similar to using the output of the Emacs ‘grep’ command.

Command ‘icicle-occur’ is similar to ‘occur’, but instead of entering a regexp (with ‘RET’) you type a regexp and then use ‘S-TAB’ to show the matching lines in buffer ‘*Completions*’. As usual in Icicles, you can complete to a single candidate, or cycle among candidates to choose one. To navigate to a match in the original buffer, use ‘C-RET’, ‘C-mouse-2’, ‘C-next’, or ‘C-prior’. One advantage of ‘icicle-occur’ over ‘occur’ is that you can change the regexp on the fly to match different sets of lines.

Another, major advantage is that you can use progressive completion to find lines using multiple regexps. A similar, but less interactive, effect can be had using chained ‘grep’ commands, but it is otherwise not possible with other search methods such as regexp Isearch. A regexp simply cannot express intersection (“and”) except in the limited form of “followed by”.

Here’s a screenshot of using ‘icicle-occur’ to find lines that match the regexp `regexp.*match’. The text matching the input regexp is highlighted with green. The line highlighted with pink corresponds to the current candidate during cycling (with ‘C-next’). The current cycling candidate is shown in the minibuffer and is highlighted in ‘*Completions*’.

IciclesOccurScreenshot

For more about ‘icicle-occur’, see OccurMode.

Command ‘icicle-search’ is a generalization of ‘icicle-occur’. You enter an initial, search-context regexp (using ‘RET’), which defines a set of completion candidates: all of the matching strings in the current buffer (by default). These candidates are called search contexts.

Command ‘icicle-occur’ is really ‘icicle-search’ with an implicit initial regexp of `.*’ (which you do not enter, however). For ‘icicle-occur’, the search contexts, that is, the initial completion candidates, are all of the lines of the buffer. (`.’ matches any character except a newline).

This means that ‘icicle-occur’ does not, like ‘grep’ and ‘occur’, give you just one chance to filter the lines by providing a regexp to match. It lets you dynamically change the filtering regexp, changing the search hits on the fly. That is the general idea of Icicles search: define search contexts, then filter them dynamically by matching your current minibuffer input.

What if you want to start out with only a subset of the buffer lines, and then match those dynamically – in other words, you are sure you want to limit your searching to only some of the lines? In that case, you just use ‘icicle-search’, not ‘icicle-occur’, providing it with a context-defining regexp that matches only the lines you want.

For example, you might use ‘.*for.*’, to start with only the lines containing ‘for’ as the contexts to search. Again, ‘icicle-occur’ is just a shortcut version of ‘icicle-search’ for the common case where you want to dynamically match any of the lines.

More generally, with ‘icicle-search’, the candidates need not be single, complete lines; they can be any strings in the buffer, including multiple-line strings. Your initial regexp is used over and over to find the set of matching strings in the region or buffer that you search. These strings then serve as the completion candidates.

For example, you might use a search-context regexp of “[A-Z][^.?!]+[.?!]” to search sentences, “\\(.+\n\\)+” to search paragraphs, or “\\([^\f]*[\f]\\|[^\f]+$\\)” to search pages. (`^L’ is the page-separator character. (That’s in fact how convenience commands ‘icicles-search-sentences’, ‘icicles-search-paragraphs’, and ‘icicles-search-pages’ are defined.)

`\f’ is the form-feed, or page-separator, character. You input `\f’, `\t’, and `\n’ using ‘C-q l’, ‘C-q TAB’, and ‘C-j’, respectively. See Icicles - Dot, Dot, Dot for information about multi-line dot (`.’), which matches also newline.

Again, you can use progressive completion (‘M-*’ or ‘S-SPC’) to match several different regexps within the same page or the same sentence. For example, you could find all sentences that contain ‘character’, ‘delete’, and ‘backward’, in any order, as follows:

  C-c ` [A-Z][^.?!]+[.?!] RET character S-SPC delete S-SPC backward

When you visit a search context, both ‘icicle-occur’ and ‘icicle-search’ highlight that hit. For ‘icicle-occur’, the search context is the current line. For ‘icicle-search’, it is whatever your search-context regexp matches.

Here is a screenshot that shows some of the highlighting features of ‘icicle-search’:

IciclesSearchScreenshot

Icicles Search Key Bindings

There are many Icicles search commands, most of which are bound to keys when you are in Icicle minor mode. They are all placed on the same prefix key, ‘M-s M-s’. Starting with Emacs 23, a single ‘M-s’ is the standard Emacs prefix key for search. Just hit the key twice for Icicles search.

The most general Icicles search command is ‘icicle-search’, which is bound to ‘M-s M-s M-s’. It is also bound to `C-c `’. (In some modes these keys are bound to a mode-specific form of Icicles search.)

The Icicles search keys are generally mnemonic. Some of the commands also have an alternative key binding (in parentheses in the list below).

Remember too that you can also invoke some of these same commands using a prefix argument with the generic ‘icicle-search’ keys. For example, you can invoke the commands bound to ‘M-s M-s m’ using a zero prefix argument with ‘icicle-search’ – e.g., ‘C-0 M-s M-s M-s’ or `C-0 C-c `’.

Here are the suffix keys on the ‘M-s M-s’ prefix key:

(You need library Bookmark+ for ‘icicle-search-bookmark-list-marked’. You need library Dired+ for ‘icicle-search-dired-marked’.)

There are many ‘icicle-search-*-bookmark’ commands, for searching within bookmarks of various types.

And there are several ‘icicle-imenu-*’ commands for navigating among definitions of different kinds. For each of the ‘icicle-menu*’ commands there is a ‘-full’ version that searches the full text of a definition.

When you use one of these full-definition search commands, the completion candidates can be quite large, spanning several lines each. In this context it can be handy to hide, in buffer ‘*Completions*’, the lines that do not match your current minibuffer input. You can do this at any time by using command ‘icicle-toggle-hiding-non-matching-lines’, bound to `C-u C-x .’, to toggle user option `icicle-hide-non-matching-lines-flag’.

The commands that search zones of text that have a given character (text or overlay) property value work with any kind of property. They work specially for properties ‘face’ (or ‘font-lock-face’) and ‘mumamo-major-mode’. If you use library MuMaMo, which lets you, in effect, use multiple major modes at the same time in the same buffer, then you can use ‘M-s M-s c’ and ‘M-s M-s O’ to search the zones corresponding to a given major mode. See the doc string for command ‘icicle-search-char-property’ for more information.

Command ‘icicle-search-thing’ (‘M-s M-s t’) searches the text of ThingAtPoint things. It prompts you for the thing type: `sexp', ‘sentence’, `list', ‘string’, ‘comment’, etc. It ignores comments according to options `icicle-ignore-comments-flag’ and `icicle-hide-whitespace-before-comment-flag’. You can toggle this ignoring using `C-M-;’ at any time. When comments are ignored, the candidate things (e.g. sexps) to be searched are only those outside of comments.

How Icicles Search Works

All Icicles search commands operate in the same general way:

  1. Unlike standard incremental search, Icicles search commands search the entire buffer, not just the part that follows the cursor. If the region is active, however, then the search is confined to the region. Some Icicles search commands let you search across multiple buffers, multiple files, or multiple bookmarks, including region bookmarks. Searching within one or more such regions of text is a first way to limit the context of a search.
  2. You limit the search context in a second way, by providing some information, such as a regexp or a text or overlay property, that defines zones of text that you want to search. You can use (lax) completion against previous input to enter the regexp or the text or overlay property. In some cases, the information (e.g. regexp) to define the search context is provided automatically by the search command; for example, ‘icicle-occur’ assumes that you want to search lines.
  3. If you use a regexp to define the search context, and if that regexp has subgroups, that is, subexpressions of the form `\(...\)’, then you are prompted for the subgroup to use to define the search context. 0 means the entire regexp match is used as a context. 1 means that whatever the first subgroup matches is used as a context, and so on. Using a subgroup thus limits the search context in a third way. It lets you find a search match within a larger search-match context. For example, you might choose an EmacsLisp argument list as the search context, specifying that it must follow `(defun ’: `(defun [^(]*\(([^(]*)\)’. Subgroup 1 is the argument list. Specifying a subgroup search context helps you become more familiar with regexps. Icicles search highlighting (see below) shows you the subgroup matches instantly.
  4. You can limit the set of search contexts in a fourth way, by using `M-&’ to provide predicates that search-context candidates must satisfy. Command ‘icicle-search’ and its derivative functions use candidates of the form (CONTEXT . MARKER), where CONTEXT is a string, the search hit (search context), and MARKER is a buffer marker that locates the CONTEXT. Predicates you supply to the `M-&’ prompt must expect such candidates. Only contexts that satisfy the predicate are found. For example, if the predicate is (lambda (x) (commandp (intern-soft (car x)))), then only contexts that name Emacs commands are found. Or, if you have a predicate ‘verbp’ that tests whether a word is an English verb form, then you can use that to limit word matches to verbs. In this way, you can combine purely syntactic searching (regexp or text-property match) with more semantic search criteria. After building up a complex predicate by using `M-&’, you can save it to a variable with `C-M-&’ and then reuse it later with `C-=’. See also M-&: Satisfying Additional Predicates.
  5. Icicles finds all of the qualified search contexts, and presents them to you as completion candidates. As always for Icicles completion, the number of search hits (matching candidates), is displayed in the mode line of buffer ‘*Completions*’ – e.g., ‘72 candidates’.
  6. You can navigate among the source-buffer search contexts, using the multi-command action keys (‘C-next’, ‘C-prior’, ‘C-RET’, ‘C-mouse-2’). The contexts are highlighted in the source buffer(s). You can scroll the current search-hits buffer forward and backward using ‘C-M-v’ and ‘C-M-S-v’ (aka ‘C-M-V’). Whenever the destination would be off-screen, user option `icicle-recenter’ is passed to ‘recenter’ to make it visible.
  7. As always in Icicles, your current minibuffer input filters the set of current candidates – the search contexts, so that only those that contain matches to your input remain as candidates. This is a second level of matching: looking for a refinement pattern within the search contexts. And this constitutes a fifth way you can limit the set of search contexts.
  8. As always in Icicles, this input can be a regexp. This is ordinary apropos completion, applied to searching. You do not type ‘RET’ to enter this regexp, and you can change it on the fly to change the set of search hits. Icicles searching is thus incremental, in the sense that changing your input dynamically changes the set of matching search hits. Icicles searching is not incremental with respect to the initial, context matching, however.
  9. As always in Icicles, you can type some input and then hit `C-~’ to remove all candidates that match that input. Then type some other input and hit `C-~’ again to remove those matches. Or you can use `M-&’ to define a predicate, and then hit `C-~’ to remove all candidates that satisfy that predicate. And so on. And you can use ‘S-mouse-2’ or the ‘delete’ key to remove individual search hits. These techniques let you chip away at the search set, removing hits that are uninteresting. This is a very powerful technique for both searching and search-and-replace (see next), and it constitutes a sixth way to limit the set of search contexts.
  10. You can sort the search hits in various ways. This can facilitate navigation and comparison of hits, as well as search-and-replace (see #11). And you can define your own Icicles search commands that provide custom search orders for particular kinds of search. It is likely that you have never considered being able to sort search hits, but if you think about it you will see that this can be handy. If you are searching across multiple buffers, files, or bookmarks, sorting helps you compare, visit, and replace related hits from the different sources, instead of having to handle all of the hits from each source in turn.
  11. You can replace text while you search, forward, backward, or randomly. You can replace entire search contexts or just the parts that match your current input. You can use any replacement string that is allowed by ‘query-replace-regexp’. In Emacs 22 or later, this includes `\,’, to substitute the result of a Lisp evaluation (see ReplaceRegexp for an explanation of `\,’). Use the alternative-action keys for replacement: ‘C-S-RET’, ‘C-S-mouse-2’, ‘C-S-down’, ‘C-S-up’, ‘C-S-next’, ‘C-S-prior’, ‘C-S-end’, and ‘C-S-home’. At the first use, you are prompted for the replacement string; it is used thereafter. You can use `M-|’ (‘icicle-all-candidates-list-alt-action’) to replace all matches. See Icicles - Search-And-Replace.
  12. When you visit a search context (using ‘C-mouse-2’ or ‘C-down’, for example), the part of the candidate that matches your input is highlighted. An entire search context is highlighted in face `icicle-search-main-regexp-current’, and the part that matches your input is highlighted in face `icicle-search-current-input’. All other search contexts are also highlighted (in face `icicle-search-main-regexp-others’). The effect is similar to the Emacs 22+ lazy search highlighting of Isearch (except that the highlighting is not in fact lazy).
  13. User option `icicle-search-highlight-all-current-flag’ controls whether the input matches are highlighted within each search context or only within the current context. Together with ‘icicle-expand-input-to-common-match’, it controls whether the input-match highlighting covers an expanded common match among all matches or just the exact input match. See screenshot, above.
  14. If you do not use a subgroup to define the search context (as in #3, above), that is, if the search context corresponds to the entire search regexp, then up to eight context levels (subgroups) are each highlighted differently, using faces `icicle-search-context-level-1’ through `icicle-search-context-level-8’. This context-level highlighting is not done if user option `icicle-search-highlight-context-levels-flag’ is ‘nil’.

Here is a screenshot (from icicles-cmd1.el) that shows context-level highlighting. The regexp defining the search context is this:
(\([-a-z*]+\) *\((\(([-a-z]+ *\([^)]*\))\))\).*
IciclesSearchContextLevelsScreenshot
If you type input in the minibuffer, then its matches are highlighted on top of this. For example, if your current input is ‘e’, then each ‘e’ in the search context of the screenshot would be highlighted with face ‘icicle-search-current-input’ (black text on bright green background, by default). Your input match always takes highest priority.

You might have noticed that out of these 14 search features, 6 constitute independent ways in which you can narrow or limit the set of search hits among which you can navigate. And another one (sorting) further facilitates your observation and selection of search hits. Restricting the search space and making search-hit patterns more evident are in fact what search is all about, and Icicles offers you some unique tools to do this.

For several Icicles search commands, including ‘icicle-search’ (`C-c `’), you provide an initial regexp to define the search contexts (step 1, above). Why use two regexps to search (steps 1 and 4, above)? To make things simpler. Regular expressions are powerful for searching, but they can also be cumbersome sometimes. Why not use one simple regexp to set up a set of candidates and then, optionally, use a second simple regexp to filter those candidates?

This is the same idea as that behind progressive completion. And speaking of which, how would you find a line that contains a given set of words (each of them), but in an arbitrary (unknown) order? Progressive completion. Which lines in this doc section contain the words ‘which’, ‘you’, and ‘how’, in any order? If you are reading this doc in file icicles-doc2.el, then just use ‘icicle-occur’ with progressive completion:

  C-c ' which S-SPC you S-SPC how

That narrows things down to four lines that you can then navigate among. Progressive completion gives Icicles search a power boost.

Like ‘icicle-occur’ (`C-c '’), commands ‘icicle-search-word’ (`C-c $’) and ‘icicle-search-keywords’ (`C-c ^’) are variants of ‘icicle-search’ that differ only in the regexp used. Each accepts your input and converts it to a regexp that does the right thing. ‘icicle-search-word’ just adds `\b’ before and after the word you type, so that it matches at word boundaries.

‘icicle-search-keywords’ wraps the keywords you provide as input with regexp grouping (`\(...\)’) and alternative (`\|’) syntax, so that search looks for any of the keywords.

Keywords” here is an understatement. Each keyword is actually a regexp and is treated as such, unless you use `C-`’ to turn on escaping of regexp special characters. In that case, each keyword is matched as a substring. At the `C-c $’ prompt, you can use completion to choose keywords that you have already entered, or you can use ‘C-RET’ to enter new keywords.

As a shortcut, you can use the search string during Isearch as the initial regexp for ‘icicle-search’. You do this by hitting ‘S-TAB’ during Isearch. This ends Isearch and passes its search string to ‘icicle-search’. This can be a handy way to start ‘icicle-search’, picking up its search pattern by using, say, `C-s C-w C-w...’. It lets you see the matches of your regexp highlighted incrementally, as you type it. See Icicles Search from Isearch.

Search Outside the Defined Search Contexts

For each of the predefined Icicles search commands, including for ‘icicle-search’ itself, you can alternatively choose to search, not the search contexts as you define them, but the non

-contexts, that is, the buffer text that is outside (in between) the search contexts as defined.

For example, if you use ‘icicle-search-thing’ and you define sexps as the search contexts, then this feature lets you search the zones of text that are not within a sexp. Or if you use ‘icicle-search-text-property’ (‘C-c "’), you can search the zones of text that do not have a text-property value that you specify (e.g., property ‘face’ with faces ‘font-lock-comment-face’ and ‘font-lock-comment-delimiter-face’ – which means all code outside comments).

To turn this context-complementing feature on and off, hit `C-M-~’ (‘icicle-toggle-search-complementing-domain’) during completion. This is a toggle, and it affects only future search commands, not the current one.

Search Multiple Buffers, Files, and Bookmarks

If you provide a prefix argument to most Icicles search functions, then you can search multiple buffers, files, or bookmarks.

As a convenience, some specialized Icicles commands are defined that correspond to ‘icicle-search’ with the various prefix-argument cases: ‘icicle-search-bookmarks-together’, ‘icicle-search-buffer’, and ‘icicle-search-file’. If you often use `C-c `’ with one of the prefix-argument options, then you might want to bind one or more of these commands. These commands are also available in the Icicles menu bar menu (or the Search menu, if it exists).

User Options for Icicles Searching

You can Customize the following user options to control search and replacement behavior.

Gotcha: Remember that the expanded common match pertains to the entire completion candidate. In the context of Icicles search, if you are interested in multiple matches of your input within the same search context, and you want to be sure to catch each match, then turn off common-match expansion.
Why? The search context as a whole is compared with the other search contexts when looking for the expanded common match. Your input is matched against the entire context (search hit), and the expanded common match is (typically) the longest match that is common to the other search contexts. Do not expect the longest common match of your input against all occurrences in the search contexts. What counts is the longest single match for a given context.
For example, if your input is ‘then’ and two of the search hits are ‘But then X and then Y’ and ‘W and then Z’, the expanded common match will be ‘and then’, not ‘then’. The matches highlighted for your input thus do not include each occurrence of ‘then’ in the search hits, but rather each occurrence of ‘and then’.

It can sometimes be useful to highlight all regexp matches using a large (or ‘t’) value of ‘icicle-search-highlight-threshold’ and a ‘nil’ value of ‘icicle-search-cleanup-flag’, and then set ‘icicle-search-highlight-threshold’ to zero and use an Icicles search function again with a different regexp to search through the same region or buffer. This lets you see the relation between the two sets of regexp matches.

Using Regexps With Icicles Search

You can use Icicles search to find text entities of a certain kind – sentences, paragraphs, file names, URLs, and so on. A convenient way to do this is to use `C-=’ (‘icicle-insert-string-from-variable’) or ‘C-x r i’ (‘insert-register’) to insert a predefined regexp that matches a particular kind of text entity. Which of these you use depends on whether the regexp string is saved in a variable (`C-=’) or a register (‘C-x r i’).

For example, suppose you are in a mail client and you want to move between mail headers. If you use a regexp that matches the header field you want (e.g. the sent date or sender) then Icicles search highlights all such occurrences and lets you navigate among them – instant mail browser! Or, suppose you are in a C++ or Perl file and you want to navigate among function definitions or other definitions. If you have a canned regexp that matches the start of a definition, then you can use `C-=’ to quickly turn ‘icicle-search’ into a code browser. In a log file, navigate among date or time entries or IP addresses… Of course, most programming modes and mail clients already provide other ways to navigate, but you get the idea – Icicles search provides a general way to navigate among things, as long as you can match them with regexps, and `C-=’ lets you quickly access a library of predefined regexps.

You can find useful regexps to store in variables in the standard EmacsLisp libraries. Grep for ‘font-lock-keywords’ or ‘regexp’ in the Emacs ‘lisp’ directory and its subdirectories.

Imenu regexps occurring as parts of different values of ‘imenu-generic-expression’ for different buffer types can be used as variable values for `C-=’. They all work fine with ‘icicle-search’, turning it into a navigator for the given mode. See, for example, generic-x.el and lisp-mode.el. Here is a regexp for Javascript function definitions from generic-x.el:

  "^function\\s-+\\([A-Za-z0-9_]+\\)"

And ‘lisp-imenu-generic-expression’ (in lisp-mode.el) provides regexps for Lisp function, variable, and type definitions. Here is the variable-definition regexp:

  "^\\s-*(\\(def\\(c\\(onst\\(ant\\)?\\|ustom\\)\\|ine-symbol-macro\\|parameter\\|var\\)\\)\\s-+\\(\\(\\sw\\|\\s_\\)+\\)"

You certainly do not want to type a regexp like that into the minibuffer (and the function-definition regexp is twice as complex)! Put it into a variable or register once and use `C-=’ or ‘C-x r i’ from then on to retrieve it – simple.

If it’s so simple, then why not let a command do it? This is exactly what command ‘icicle-imenu’ (bound to `C-c =’) does. You do not need to bother looking up Imenu regexps and assigning them to variables for use with `C-=’ and ‘icicle-search’‘icicle-imenu’ does that for you automatically. See Icicles Imenu.

In sum: For complete interactivity, type a regexp dynamically as input to ‘icicle-search’. For isolated special regexps that you use, save them in variables and use `C-=’ with ‘icicle-search’. For well-defined sets of regexps, especially if used frequently, define a command that uses ‘icicle-search’. There is a spectrum of use cases for ‘icicle-search’.

Command ‘icicle-search’ is very general and very powerful. It might never replace incremental search – either regexp or literal string search, but in some cases it can be quite handy. Think of it as another tool to add to your search-tool belt. Admittedly, it does take a little getting used to. Remember, in particular, that the initial, context regexp you enter (with ‘RET’) cannot be changed without re-executing ‘icicle-search’.

And remember too that ‘C-l’ (‘icicle-retrieve-previous-input’) is your friend – it clears the minibuffer during cycling, retrieving your last real input. Use it to modify your second and subsequent regexps on the fly – those that filter the initial candidate list further. You can repeat ‘C-l’ to retrieve older completion inputs, and you can use ‘C-S-l’ (that is, ‘C-’L’) to cycle previous inputs in the other direction. See Icicles - History Enhancements.

Oh – And do not forget that you can do things like take the complement of your fine-tuning regexp matches, within the context of your coarse-tuning matches (see set complement). For example, use `^.*defun.*$’ as the main regexp, to find all lines containing ‘defun’. Then type ‘icicle’ to match only the lines with ‘defun’ that also contain ‘icicle’. Then complement (`C-~’) that set, to see the lines that contain ‘defun’ but not ‘icicle’.

And you can then save that set of matches, and then subtract it from another set of matches in a different search… You get the idea. When performing set operations combined with ‘icicle-search’, keep in mind that the saved set does not include any position information – it is only a set of matching strings. So, in particular, a set-union operation (`C-+’) is not useful with ‘icicle-search’ (adding a saved set of strings without positions is useless). Still, you can do things like match lines that contain ‘defun’ followed somewhere by `()’, and then subtract the (saved) set of lines in the same region that contain ‘icicle’. Try it in an Icicles library, using regexps `.*icicle.*$’ and `^*.defun.*().*$’.

One more reminder: When you save a set of completion candidates (`C-M->’), make sure that you actually have a set of candidates to save! It is not enough to just enter a regexp at the ‘icicle-search’ prompt. You must also use some Icicles command, such as ‘TAB’, ‘S-TAB’, ‘next’, or ‘down’ to tell Icicles how to create the candidate set – how to match the regexp.


See also:


Previous: Icicles - Isearch EnhancementsIciclesIciclesIndexNext: Icicles - Search-And-Replace

DrewsElispLibraries referenced here: Lisp:highlight.el, Lisp:icicles.el

CategoryBookmarking CategoryCommands CategoryComments CategoryCompletion CategoryRegexp CategoryRegion CategoryModes CategoryDirectories CategoryFiles CategoryProgrammerUtils CategoryCode CategorySearchAndReplace CategoryXML