MultilineRegexp

Last edit

Summary: Rückgängig gemacht auf den Stand vom 2013-08-24 11:21 UTC

Changed:

< Note: For a newline in IncrementalWikiModes use `C-q C-j'. Typing "\n" does not work. You don't have to use C-q C-m C-q C-j or similar variants for Windows or Mac files.

to

> Note: For a newline in IncrementalSearch use `C-q C-j'. Typing "\n" does not work. You don't have to use C-q C-m C-q C-j or similar variants for Windows or Mac files.

Changed:

< (while (re-WikiModes-forward

to

> (while (re-search-forward


A word of warning before we start: Multi-line regular expressions rarely work as expected for FontLock. Why is that, you ask? Emacs basically knows which line you are editing, and font-lock works on these lines you are editing. Multi-line regular expressions would require Emacs to skip backwards multiple lines and see if any of the multi-line regular expression match. That either involves a lot of matching, unsolved questions (skip back to where – the beginning of the buffer in the worst case), or caching. So while multi-line regular expressions work fine when loading a file, font-lock can get confused when you edit the lines containing the beginning or the end of the expression. See MultilineFontLock.

With that out of the way, back to basics:

To match any number of characters, use this: .* – the problem is that . matches any character except newline. What many people propose next works, but is inefficient if you assume that newlines are not that common in your text: \(.|\n\). Better match multiple lines instead: \(.*\n?\)*. The newline is optional so that the expression can end in the middle of a line. Better yet: [\0-\377[:nonascii:]]* avoids “Stack overflow in regexp matcher” for huge texts, e.g., > 34k.

Note: For a newline in IncrementalSearch use ‘C-q C-j’. Typing “\n” does not work. You don’t have to use C-q C-m C-q C-j or similar variants for Windows or Mac files.

Parsing example

The following code snippet parses entries that look like this:

    A term :: a definition 
    which may span multiple lines 
    and is terminated by a blank line or
    the end of the file
    
    Another term :: You get the idea

Code:

    (defun parse-definitions ()
      (let ((dict nil))
        (while (re-search-forward
                (concat "^\\(.*?\\)"               ;; a term
                        "\\s-*::\\s-*"             ;; the separator
                        "\\(.*\\(?:\n.*\\)*?\\)"   ;; definition: to end of line,
                                                   ;; then maybe more lines
                                                   ;; (excludes any trailing \n)
                        "\\(?:\n\\s-*\n\\|\\'\\)") ;; blank line or EOF
                nil :no-error)
          (add-to-list 'dict (cons (match-string-no-properties 1)
                                   (match-string-no-properties 2))) )
        dict))

The regular expression matching the definition may need some more explaining.

What we have is this:

    "\\(.*\\(?:\n.*\\)*?\\)"

Taking it appart from the outside, we have a group. This is the second group; the first group matched the term. Therefore the later call to (match-string-no-properties 2) will return the definition matched.

    "\\( ... \\)"

Inside the group, we start with all characters except newline. This will match starting at point up to the end of the line. If we are at the end of the line, this will match the empty string before the newline.

    ".*"

Next we have a complex little bugger. On the outside, we have a group that is matched repeatedly, but as little as possible due to the *? at the end. Furthermore, this group will not record the matched substring (thus being faster). It is a so-called shy group due to the ?: at the very beginning of the group.

    "\\(?: ... \\)*?"

Inside the shy group, we match the newline, and all the characters following it except another newline. That is, we match the newline we did not match two paragraphs up, and the next line, ending again in front of a newline:

    "\n.*"

Associated XEmacs Bug

Above does not work for me in XEmacs 21.5-b22 or XEmacs 21.4.17 either.

If I change the third string to

                        "\\(.*?\\(?:\n.*\\)*?\\)"  ;; definition: to end of line,

instead of

                        "\\(.*\\(?:\n.*\\)*?\\)"   ;; definition: to end of line,

to make the initial wildcard match non-greedy, the match works!

As Alex points out below, “.” will never match a newline, but perhaps this observation of greediness will help diagnose this XEmacs bug.

This must be a bug in XEmacs.

Hm. can .* ever match \n? One of the main gotchas is that there is not (as far as I know) an equivalent of Perl’s /s modifier, so . can’t match \n [I was also testing in scratch]

You are correct. That’s the whole point of having the \n in there. I’ve taken in further appart. – AlexSchroeder


CategoryRegexp