| The pcretest program |
| -------------------- |
| |
| This program is intended for testing PCRE, but it can also be used for |
| experimenting with regular expressions. |
| |
| If it is given two filename arguments, it reads from the first and writes to |
| the second. If it is given only one filename argument, it reads from that file |
| and writes to stdout. Otherwise, it reads from stdin and writes to stdout, and |
| prompts for each line of input, using "re>" to prompt for regular expressions, |
| and "data>" to prompt for data lines. |
| |
| The program handles any number of sets of input on a single input file. Each |
| set starts with a regular expression, and continues with any number of data |
| lines to be matched against the pattern. An empty line signals the end of the |
| data lines, at which point a new regular expression is read. The regular |
| expressions are given enclosed in any non-alphameric delimiters other than |
| backslash, for example |
| |
| /(a|bc)x+yz/ |
| |
| White space before the initial delimiter is ignored. A regular expression may |
| be continued over several input lines, in which case the newline characters are |
| included within it. See the test input files in the testdata directory for many |
| examples. It is possible to include the delimiter within the pattern by |
| escaping it, for example |
| |
| /abc\/def/ |
| |
| If you do so, the escape and the delimiter form part of the pattern, but since |
| delimiters are always non-alphameric, this does not affect its interpretation. |
| If the terminating delimiter is immediately followed by a backslash, for |
| example, |
| |
| /abc/\ |
| |
| then a backslash is added to the end of the pattern. This is done to provide a |
| way of testing the error condition that arises if a pattern finishes with a |
| backslash, because |
| |
| /abc\/ |
| |
| is interpreted as the first line of a pattern that starts with "abc/", causing |
| pcretest to read the next line as a continuation of the regular expression. |
| |
| The pattern may be followed by i, m, s, or x to set the PCRE_CASELESS, |
| PCRE_MULTILINE, PCRE_DOTALL, or PCRE_EXTENDED options, respectively. For |
| example: |
| |
| /caseless/i |
| |
| These modifier letters have the same effect as they do in Perl. There are |
| others which set PCRE options that do not correspond to anything in Perl: /A, |
| /E, and /X set PCRE_ANCHORED, PCRE_DOLLAR_ENDONLY, and PCRE_EXTRA respectively. |
| |
| Searching for all possible matches within each subject string can be requested |
| by the /g or /G modifier. After finding a match, PCRE is called again to search |
| the remainder of the subject string. The difference between /g and /G is that |
| the former uses the startoffset argument to pcre_exec() to start searching at |
| a new point within the entire string (which is in effect what Perl does), |
| whereas the latter passes over a shortened substring. This makes a difference |
| to the matching process if the pattern begins with a lookbehind assertion |
| (including \b or \B). |
| |
| If any call to pcre_exec() in a /g or /G sequence matches an empty string, the |
| next call is done with the PCRE_NOTEMPTY and PCRE_ANCHORED flags set in order |
| to search for another, non-empty, match at the same point. If this second match |
| fails, the start offset is advanced by one, and the normal match is retried. |
| This imitates the way Perl handles such cases when using the /g modifier or the |
| split() function. |
| |
| There are a number of other modifiers for controlling the way pcretest |
| operates. |
| |
| The /+ modifier requests that as well as outputting the substring that matched |
| the entire pattern, pcretest should in addition output the remainder of the |
| subject string. This is useful for tests where the subject contains multiple |
| copies of the same substring. |
| |
| The /L modifier must be followed directly by the name of a locale, for example, |
| |
| /pattern/Lfr |
| |
| For this reason, it must be the last modifier letter. The given locale is set, |
| pcre_maketables() is called to build a set of character tables for the locale, |
| and this is then passed to pcre_compile() when compiling the regular |
| expression. Without an /L modifier, NULL is passed as the tables pointer; that |
| is, /L applies only to the expression on which it appears. |
| |
| The /I modifier requests that pcretest output information about the compiled |
| expression (whether it is anchored, has a fixed first character, and so on). It |
| does this by calling pcre_fullinfo() after compiling an expression, and |
| outputting the information it gets back. If the pattern is studied, the results |
| of that are also output. |
| |
| The /D modifier is a PCRE debugging feature, which also assumes /I. It causes |
| the internal form of compiled regular expressions to be output after |
| compilation. |
| |
| The /S modifier causes pcre_study() to be called after the expression has been |
| compiled, and the results used when the expression is matched. |
| |
| The /M modifier causes the size of memory block used to hold the compiled |
| pattern to be output. |
| |
| Finally, the /P modifier causes pcretest to call PCRE via the POSIX wrapper API |
| rather than its native API. When this is done, all other modifiers except /i, |
| /m, and /+ are ignored. REG_ICASE is set if /i is present, and REG_NEWLINE is |
| set if /m is present. The wrapper functions force PCRE_DOLLAR_ENDONLY always, |
| and PCRE_DOTALL unless REG_NEWLINE is set. |
| |
| Before each data line is passed to pcre_exec(), leading and trailing whitespace |
| is removed, and it is then scanned for \ escapes. The following are recognized: |
| |
| \a alarm (= BEL) |
| \b backspace |
| \e escape |
| \f formfeed |
| \n newline |
| \r carriage return |
| \t tab |
| \v vertical tab |
| \nnn octal character (up to 3 octal digits) |
| \xhh hexadecimal character (up to 2 hex digits) |
| |
| \A pass the PCRE_ANCHORED option to pcre_exec() |
| \B pass the PCRE_NOTBOL option to pcre_exec() |
| \Cdd call pcre_copy_substring() for substring dd after a successful match |
| (any decimal number less than 32) |
| \Gdd call pcre_get_substring() for substring dd after a successful match |
| (any decimal number less than 32) |
| \L call pcre_get_substringlist() after a successful match |
| \N pass the PCRE_NOTEMPTY option to pcre_exec() |
| \Odd set the size of the output vector passed to pcre_exec() to dd |
| (any number of decimal digits) |
| \Z pass the PCRE_NOTEOL option to pcre_exec() |
| |
| A backslash followed by anything else just escapes the anything else. If the |
| very last character is a backslash, it is ignored. This gives a way of passing |
| an empty line as data, since a real empty line terminates the data input. |
| |
| If /P was present on the regex, causing the POSIX wrapper API to be used, only |
| \B, and \Z have any effect, causing REG_NOTBOL and REG_NOTEOL to be passed to |
| regexec() respectively. |
| |
| When a match succeeds, pcretest outputs the list of captured substrings that |
| pcre_exec() returns, starting with number 0 for the string that matched the |
| whole pattern. Here is an example of an interactive pcretest run. |
| |
| $ pcretest |
| PCRE version 2.06 08-Jun-1999 |
| |
| re> /^abc(\d+)/ |
| data> abc123 |
| 0: abc123 |
| 1: 123 |
| data> xyz |
| No match |
| |
| If the strings contain any non-printing characters, they are output as \0x |
| escapes. If the pattern has the /+ modifier, then the output for substring 0 is |
| followed by the the rest of the subject string, identified by "0+" like this: |
| |
| re> /cat/+ |
| data> cataract |
| 0: cat |
| 0+ aract |
| |
| If the pattern has the /g or /G modifier, the results of successive matching |
| attempts are output in sequence, like this: |
| |
| re> /\Bi(\w\w)/g |
| data> Mississippi |
| 0: iss |
| 1: ss |
| 0: iss |
| 1: ss |
| 0: ipp |
| 1: pp |
| |
| "No match" is output only if the first match attempt fails. |
| |
| If any of \C, \G, or \L are present in a data line that is successfully |
| matched, the substrings extracted by the convenience functions are output with |
| C, G, or L after the string number instead of a colon. This is in addition to |
| the normal full list. The string length (that is, the return from the |
| extraction function) is given in parentheses after each string for \C and \G. |
| |
| Note that while patterns can be continued over several lines (a plain ">" |
| prompt is used for continuations), data lines may not. However newlines can be |
| included in data by means of the \n escape. |
| |
| If the -p option is given to pcretest, it is equivalent to adding /P to each |
| regular expression: the POSIX wrapper API is used to call PCRE. None of the |
| following flags has any effect in this case. |
| |
| If the option -d is given to pcretest, it is equivalent to adding /D to each |
| regular expression: the internal form is output after compilation. |
| |
| If the option -i is given to pcretest, it is equivalent to adding /I to each |
| regular expression: information about the compiled pattern is given after |
| compilation. |
| |
| If the option -m is given to pcretest, it outputs the size of each compiled |
| pattern after it has been compiled. It is equivalent to adding /M to each |
| regular expression. For compatibility with earlier versions of pcretest, -s is |
| a synonym for -m. |
| |
| If the -t option is given, each compile, study, and match is run 20000 times |
| while being timed, and the resulting time per compile or match is output in |
| milliseconds. Do not set -t with -s, because you will then get the size output |
| 20000 times and the timing will be distorted. If you want to change the number |
| of repetitions used for timing, edit the definition of LOOPREPEAT at the top of |
| pcretest.c |
| |
| Philip Hazel <ph10@cam.ac.uk> |
| January 2000 |