Main Content

regexprep

Replace text in words of documents using regular expression

collapse all in page

Syntax

newDocuments = regexprep(documents,expression,replace)

Description

Text Analytics Toolbox™ provides functions for common text preprocessing steps. For example, to remove punctuation and symbol characters, use erasePunctuation or to remove stem words using the Porter stemmer, use normalizeWords. For more information, see Text Data Preparation.

newDocuments = regexprep(documents,expression,replace) replaces all occurrences of the regular expression expression in the words of documents with the text in replace.

The function matches each word independently. The match does not have to span the whole word.

example

Examples

collapse all

Open Live Script

Replace words that begin with "s", end "e", and have at least one character between them. To match whole words, use "^" to match the start of a word and "$" to match the end of the word.

documents = tokenizedDocument([ ...
    "an example of a short sentence" 
    "a second short sentence"])
documents = 
  2x1 tokenizedDocument:

    6 tokens: an example of a short sentence
    4 tokens: a second short sentence


expression = "^s(\w+)e$";
replace = "thing";
newDocuments = regexprep(documents,expression,replace)
newDocuments = 
  2x1 tokenizedDocument:

    6 tokens: an example of a short thing
    4 tokens: a second short thing

If you do not use "^" and "$", then you can match substrings of the words. Replace all vowels with "_".

expression = "[aeiou]";
replace = "\_";
newDocuments = regexprep(documents,expression,replace)
newDocuments = 
  2x1 tokenizedDocument:

    6 tokens: _n _x_mpl_ _f _ sh_rt s_nt_nc_
    4 tokens: _ s_c_nd sh_rt s_nt_nc_
Open Live Script

Replace variations of the word "walk" by capturing the letters that follow "walk".

documents = tokenizedDocument([
    "I walk"
    "they walked"
    "we are walking"])
documents = 
  3x1 tokenizedDocument:

    2 tokens: I walk
    2 tokens: they walked
    3 tokens: we are walking


expression = "walk(\w*)";
replace = "ascend$1";
newDocuments = regexprep(documents,expression,replace)
newDocuments = 
  3x1 tokenizedDocument:

    2 tokens: I ascend
    2 tokens: they ascended
    3 tokens: we are ascending

Input Arguments

collapse all

Input documents, specified as a tokenizedDocument array.

Regular expression, specified as a character vector, a cell array of character vectors, or a string array. Each expression can contain characters, metacharacters, operators, tokens, and flags that specify patterns to match in str.

The following tables describe the elements of regular expressions.

Metacharacters

Metacharacters represent letters, letter ranges, digits, and space characters. Use them to construct a generalized pattern of characters.

Metacharacter

Description

Example

.

Any single character, including white space

'..ain' matches sequences of five consecutive characters that end with 'ain'.

[c1c2c3]

Any character contained within the square brackets. The following characters are treated literally: $ | . * + ? and - when not used to indicate a range.

'[rp.]ain' matches 'rain' or 'pain' or '.ain'.

[^c1c2c3]

Any character not contained within the square brackets. The following characters are treated literally: $ | . * + ? and - when not used to indicate a range.

'[^*rp]ain' matches all four-letter sequences that end in 'ain', except 'rain' and 'pain' and '*ain'. For example, it matches 'gain', 'lain', or 'vain'.

[c1-c2]

Any character in the range of c1 through c2

'[A-G]' matches a single character in the range of A through G.

\w

Any alphabetic, numeric, or underscore character. For English character sets, \w is equivalent to [a-zA-Z_0-9]

'\w*' identifies a word comprised of any grouping of alphabetic, numeric, or underscore characters.

\W

Any character that is not alphabetic, numeric, or underscore. For English character sets, \W is equivalent to [^a-zA-Z_0-9]

'\W*' identifies a term that is not a word comprised of any grouping of alphabetic, numeric, or underscore characters.

\s

Any white-space character; equivalent to [ \f\n\r\t\v]

'\w*n\s' matches words that end with the letter n, followed by a white-space character.

\S

Any non-white-space character; equivalent to [^ \f\n\r\t\v]

'\d\S' matches a numeric digit followed by any non-white-space character.

\d

Any numeric digit; equivalent to [0-9]

'\d*' matches any number of consecutive digits.

\D

Any nondigit character; equivalent to [^0-9]

'\w*\D\>' matches words that do not end with a numeric digit.

\oN or \o{N}

Character of octal value N

'\o{40}' matches the space character, defined by octal 40.

\xN or \x{N}

Character of hexadecimal value N

'\x2C' matches the comma character, defined by hex 2C.

Character Representation

Operator

Description

\a

Alarm (beep)

\b

Backspace

\f

Form feed

\n

New line

\r

Carriage return

\t

Horizontal tab

\v

Vertical tab

\char

Any character with special meaning in regular expressions that you want to match literally (for example, use \\ to match a single backslash)

Quantifiers

Quantifiers specify the number of times a pattern must occur in the matching text.

Quantifier

Number of Times Expression Occurs

Example

expr*

0 or more times consecutively.

'\w*' matches a word of any length.

expr?

0 times or 1 time.

'\w*(\.m)?' matches words that optionally end with the extension .m.

expr+

1 or more times consecutively.

'<img src="\w+\.gif">' matches an <img> HTML tag when the file name contains one or more characters.

expr{m,n}

At least m times, but no more than n times consecutively.

{0,1} is equivalent to ?.

'\S{4,8}' matches between four and eight non-white-space characters.

expr{m,}

At least m times consecutively.

{0,} and {1,} are equivalent to * and +, respectively.

'<a href="\w{1,}\.html">' matches an <a> HTML tag when the file name contains one or more characters.

expr{n}

Exactly n times consecutively.

Equivalent to {n,n}.

'\d{4}' matches four consecutive digits.

Quantifiers can appear in three modes, described in the following table. q represents any of the quantifiers in the previous table.

Mode

Description

Example

exprq

Greedy expression: match as many characters as possible.

Given the text '<tr><td><p>text</p></td>', the expression '</?t.*>' matches all characters between <tr and /td>:

'<tr><td><p>text</p></td>'

exprq?

Lazy expression: match as few characters as necessary.

Given the text'<tr><td><p>text</p></td>', the expression '</?t.*?>' ends each match at the first occurrence of the closing angle bracket (>):

'<tr>'   '<td>'   '</td>'

exprq+

Possessive expression: match as much as possible, but do not rescan any portions of the text.

Given the text'<tr><td><p>text</p></td>', the expression '</?t.*+>' does not return any matches, because the closing angle bracket is captured using .*, and is not rescanned.

Grouping Operators

Grouping operators allow you to capture tokens, apply one operator to multiple elements, or disable backtracking in a specific group.

Grouping Operator

Description

Example

(expr)

Group elements of the expression and capture tokens.

'Joh?n\s(\w*)' captures a token that contains the last name of any person with the first name John or Jon.

(?:expr)

Group, but do not capture tokens.

'(?:[aeiou][^aeiou]){2}' matches two consecutive patterns of a vowel followed by a nonvowel, such as 'anon'.

Without grouping, '[aeiou][^aeiou]{2}'matches a vowel followed by two nonvowels.

(?>expr)

Group atomically. Do not backtrack within the group to complete the match, and do not capture tokens.

'A(?>.*)Z' does not match 'AtoZ', although 'A(?:.*)Z' does. Using the atomic group, Z is captured using .* and is not rescanned.

(expr1|expr2)

Match expression expr1 or expression expr2.

If there is a match with expr1, then expr2 is ignored.

You can include ?: or ?> after the opening parenthesis to suppress tokens or group atomically.

'(let|tel)\w+' matches words that contain, but do not end, with let or tel.

Anchors

Anchors in the expression match the beginning or end of the input text or word.

Anchor

Matches the...

Example

^expr

Beginning of the input text.

'^M\w*' matches a word starting with M at the beginning of the text.

expr$

End of the input text.

'\w*m$' matches words ending with m at the end of the text.

\<expr

Beginning of a word.

'\<n\w*' matches any words starting with n.

expr\>

End of a word.

'\w*e\>' matches any words ending with e.

Lookaround Assertions

Lookaround assertions look for patterns that immediately precede or follow the intended match, but are not part of the match.

The pointer remains at the current location, and characters that correspond to the test expression are not captured or discarded. Therefore, lookahead assertions can match overlapping character groups.

Lookaround Assertion

Description

Example

expr(?=test)

Look ahead for characters that match test.

'\w*(?=ing)' matches terms that are followed by ing, such as 'Fly' and 'fall' in the input text 'Flying, not falling.'

expr(?!test)

Look ahead for characters that do not match test.

'i(?!ng)' matches instances of the letter i that are not followed by ng.

(?<=test)expr

Look behind for characters that match test.

'(?<=re)\w*' matches terms that follow 're', such as 'new', 'use', and 'cycle' in the input text 'renew, reuse, recycle'

(?<!test)expr

Look behind for characters that do not match test.

'(?<!\d)(\d)(?!\d)' matches single-digit numbers (digits that do not precede or follow other digits).

If you specify a lookahead assertion before an expression, the operation is equivalent to a logical AND.

Operation

Description

Example

(?=test)expr

Match both test and expr.

'(?=[a-z])[^aeiou]' matches consonants.

(?!test)expr

Match expr and do not match test.

'(?![aeiou])[a-z]' matches consonants.

Logical and Conditional Operators

Logical and conditional operators allow you to test the state of a given condition, and then use the outcome to determine which pattern, if any, to match next. These operators support logical OR, and if or if/else conditions.

Conditions can be tokens, lookaround operators, or dynamic expressions of the form (?@cmd). Dynamic expressions must return a logical or numeric value.

Conditional Operator

Description

Example

expr1|expr2

Match expression expr1 or expression expr2.

If there is a match with expr1, then expr2 is ignored.

'(let|tel)\w+' matches words that start with let or tel.

(?(cond)expr)

If condition cond is true, then match expr.

'(?(?@ispc)[A-Z]:\\)' matches a drive name, such as C:\, when run on a Windows® system.

(?(cond)expr1|expr2)

If condition cond is true, then match expr1. Otherwise, match expr2.

'Mr(s?)\..*?(?(1)her|his) \w*' matches text that includes her when the text begins with Mrs, or that includes his when the text begins with Mr.

Token Operators

Tokens are portions of the matched text that you define by enclosing part of the regular expression in parentheses. You can refer to a token by its sequence in the text (an ordinal token), or assign names to tokens for easier code maintenance and readable output.

Ordinal Token Operator

Description

Example

(expr)

Capture in a token the characters that match the enclosed expression.

'Joh?n\s(\w*)' captures a token that contains the last name of any person with the first name John or Jon.

\N

Match the Nth token.

'<(\w+).*>.*</\1>' captures tokens for HTML tags, such as 'title' from the text '<title>Some text</title>'.

(?(N)expr1|expr2)

If the Nth token is found, then match expr1. Otherwise, match expr2.

'Mr(s?)\..*?(?(1)her|his) \w*' matches text that includes her when the text begins with Mrs, or that includes his when the text begins with Mr.

Named Token Operator

Description

Example

(?<name>expr)

Capture in a named token the characters that match the enclosed expression.

'(?<month>\d+)-(?<day>\d+)-(?<yr>\d+)' creates named tokens for the month, day, and year in an input date of the form mm-dd-yy.

\k<name>

Match the token referred to by name.

'<(?<tag>\w+).*>.*</\k<tag>>' captures tokens for HTML tags, such as 'title' from the text '<title>Some text</title>'.

(?(name)expr1|expr2)

If the named token is found, then match expr1. Otherwise, match expr2.

'Mr(?<sex>s?)\..*?(?(sex)her|his) \w*' matches text that includes her when the text begins with Mrs, or that includes his when the text begins with Mr.

Note

If an expression has nested parentheses, MATLAB® captures tokens that correspond to the outermost set of parentheses. For example, given the search pattern '(and(y|rew))', MATLAB creates a token for 'andrew' but not for 'y' or 'rew'.

Dynamic Regular Expressions

Dynamic expressions allow you to execute a MATLAB command or a regular expression to determine the text to match.

The parentheses that enclose dynamic expressions do not create a capturing group.

Operator

Description

Example

(??expr)

Parse expr and include the resulting term in the match expression.

When parsed, expr must correspond to a complete, valid regular expression. Dynamic expressions that use the backslash escape character (\) require two backslashes: one for the initial parsing of expr, and one for the complete match.

'^(\d+)((??\\w{$1}))' determines how many characters to match by reading a digit at the beginning of the match. The dynamic expression is enclosed in a second set of parentheses so that the resulting match is captured in a token. For instance, matching '5XXXXX' captures tokens for '5' and 'XXXXX'.

(??@cmd)

Execute the MATLAB command represented by cmd, and include the output returned by the command in the match expression.

'(.{2,}).?(??@fliplr($1))' finds palindromes that are at least four characters long, such as 'abba'.

(?@cmd)

Execute the MATLAB command represented by cmd, but discard any output the command returns. (Helpful for diagnosing regular expressions.)

'\w*?(\w)(?@disp($1))\1\w*' matches words that include double letters (such as pp), and displays intermediate results.

Within dynamic expressions, use the following operators to define replacement text.

Replacement Operator

Description

$& or $0

Portion of the input text that is currently a match

$`

Portion of the input text that precedes the current match

$'

Portion of the input text that follows the current match (use $'' to represent $')

$N

Nth token

$<name>

Named token

${cmd}

Output returned when MATLAB executes the command, cmd

Comments

Characters

Description

Example

(?#comment)

Insert a comment in the regular expression. The comment text is ignored when matching the input.

'(?# Initial digit)\<\d\w+' includes a comment, and matches words that begin with a number.

Search Flags

Search flags modify the behavior for matching expressions. An alternative to using a search flag within an expression is to pass an option input argument.

Flag

Description

(?-i)

Match letter case (default for regexp and regexprep).

(?i)

Do not match letter case (default for regexpi).

(?s)

Match dot (.) in the pattern with any character (default).

(?-s)

Match dot in the pattern with any character that is not a newline character.

(?-m)

Match the ^ and $ metacharacters at the beginning and end of text (default).

(?m)

Match the ^ and $ metacharacters at the beginning and end of a line.

(?-x)

Include space characters and comments when matching (default).

(?x)

Ignore space characters and comments when matching. Use '\ ' and '\#' to match space and # characters.

The expression that the flag modifies can appear either after the parentheses, such as

(?i)\w*

or inside the parentheses and separated from the flag with a colon (:), such as

(?i:\w*)

The latter syntax allows you to change the behavior for part of a larger expression.

Data Types: char | cell | string

Replacement text, specified as a character vector, a cell array of character vectors, or a string array, as follows:

  • If replace is a single character vector and expression is a cell array of character vectors, then regexprep uses the same replacement text for each expression.

  • If replace is a cell array of N character vectors and expression is a single character vector, then regexprep attempts N matches and replacements.

  • If both replace and expression are cell arrays of character vectors, then they must contain the same number of elements. regexprep pairs each replace element with its corresponding element in expression.

The replacement text can include regular characters, special characters (such as tabs or new lines), or replacement operators, as shown in the following tables.

Replacement Operator

Description

$& or $0

Portion of the input text that is currently a match

$`

Portion of the input text that precedes the current match

$'

Portion of the input text that follows the current match (use $'' to represent $')

$N

Nth token

$<name>

Named token

${cmd}

Output returned when MATLAB executes the command, cmd

Operator

Description

\a

Alarm (beep)

\b

Backspace

\f

Form feed

\n

New line

\r

Carriage return

\t

Horizontal tab

\v

Vertical tab

\char

Any character with special meaning in regular expressions that you want to match literally (for example, use \\ to match a single backslash)

Data Types: char | cell | string

Output Arguments

collapse all

Output documents, returned as a tokenizedDocument array.

Tips

  • Text Analytics Toolbox provides functions for common text preprocessing steps. For example, to remove punctuation and symbol characters, use erasePunctuation or to remove stem words using the Porter stemmer, use normalizeWords. For more information, see Text Data Preparation.

Version History

Introduced in R2017b

See Also

| | | | | | | | |

Topics