Skip to main content

Verified Optimizer

Pattern.Optimize() rewrites a pattern into a simpler one that behaves exactly the same in .NET. It merges single-character branches into a class, factors shared prefixes, drops duplicate branches, normalizes quantifiers, merges runs of the same atom and removes groups that do nothing. Every rewrite is tested before it is kept. For regular patterns it is also proven with language algebra.

Use this page when you need to:

  • clean up a regex that was written by hand, generated by a tool or built from many fluent pieces,
  • see which simplifications are safe for a pattern, and why others are not,
  • understand what "proven equivalent" and "checked on samples" mean in the result,
  • run the optimizer from code, the command line or the playground.

Optimize() and OptimizationResult live in the Masterly.FluentRegex namespace and work on every target the package supports: .NET Standard 2.1, .NET 8 and .NET 10.

Optimizing a pattern​

Call Optimize() on any Pattern. It returns an OptimizationResult that holds the original pattern, the optimized pattern and the list of rewrites that were applied.

using System;
using Masterly.FluentRegex;

Pattern pattern = Pattern.FromRegex("cat|car|cab");

OptimizationResult result = pattern.Optimize();

Console.WriteLine(result.Optimized);          // ca[trb]
Console.WriteLine(result.IsChanged);          // True
Console.WriteLine(result.VerifiedEquivalent); // True
Console.WriteLine(result);
// Optimized 'cat|car|cab' to 'ca[trb]' (proven equivalent):
//   - Merged alternation 'cat|car|cab' into 'ca[trb]'
  • Optimize() never changes the pattern you call it on. Patterns are immutable, so Optimized is a new Pattern.
  • "Behaves exactly the same" means the same leftmost-first matches, the same groups with the same numbers, names and captures, and the same options and timeout. It is a stronger promise than "matches the same strings".
  • Branch order is never changed. The engine tries alternatives from left to right, so reordering them could change which match it finds.

The optimizer works on the regex text, so it does not matter how the pattern was built. Fluent code often adds groups that the optimizer can remove:

Pattern extension = Pattern.With.OneOf("cat", "car", "cab");
Console.WriteLine(extension);                      // (?:cat|car|cab)
Console.WriteLine(extension.Optimize().Optimized); // ca[trb]

Pattern code = Pattern.With.StartOfLine.Digit.Digit.Digit.Literal("-").Digit.Digit.EndOfLine;
Console.WriteLine(code);                           // ^\d\d\d-\d\d$
Console.WriteLine(code.Optimize().Optimized);      // ^\d{3}-\d\d$
  • The group around OneOf is removed because nothing else in the pattern needs it.
  • \d\d stays as it is. \d{2} is not shorter, and a merge is only applied when it shortens the pattern (see Runs of the same atom are merged).

Reading the result​

OptimizationResult is a sealed, read-only class. Its constructor is internal, so you cannot create one yourself; you get it from Optimize().

MemberTypeDescription
OriginalPatternThe pattern you called Optimize() on.
OptimizedPatternThe optimized pattern. When no rewrite applied, this is the same instance as Original.
AppliedRewritesIReadOnlyList<string>One human-readable line per applied rewrite, in the order they were applied. Empty when nothing changed.
IsChangedbooltrue when at least one rewrite was applied. Same as AppliedRewrites.Count > 0.
VerifiedEquivalentbooltrue when the pattern is in the regular subset that language algebra can prove. false when rewrites were only checked on sample inputs. See Proven or checked on samples.
ToString()stringA summary that lists every applied rewrite.

Each entry in AppliedRewrites describes one step:

OptimizationResult result = Pattern.FromRegex("x{1}y{0,1}z{0,}w{1,}").Optimize();

foreach (string rewrite in result.AppliedRewrites)
{
    Console.WriteLine(rewrite);
}
// Removed redundant quantifier '{1}' from 'x{1}'
// Simplified quantifier 'y{0,1}' to 'y?'
// Simplified quantifier 'z{0,}' to 'z*'
// Simplified quantifier 'w{1,}' to 'w+'

ToString() has three forms:

FormWhen
Optimized '<original>' to '<optimized>' (proven equivalent): followed by one - <rewrite> line per rewriteRewrites were applied and the pattern is provable.
Optimized '<original>' to '<optimized>' (checked on samples, not proven): followed by the rewrite linesRewrites were applied, but the pattern is outside the provable subset.
No rewrites applied to '<original>'.Nothing changed.

When there is nothing to improve, you get the original pattern back:

Pattern pattern = Pattern.FromRegex("a|ab");

OptimizationResult result = pattern.Optimize();

Console.WriteLine(result.IsChanged);                           // False
Console.WriteLine(ReferenceEquals(result.Optimized, pattern)); // True
Console.WriteLine(result.AppliedRewrites.Count);               // 0
Console.WriteLine(result.VerifiedEquivalent);                  // True
Console.WriteLine(result);                                     // No rewrites applied to 'a|ab'.
note

VerifiedEquivalent describes the pattern, not the rewrites. It is true for a|ab above even though nothing changed, because the pattern is regular and any rewrite would have had to pass the proof. Check IsChanged to know whether something was rewritten.

The rewrite rules​

The optimizer knows six kinds of rewrite. Each one keeps the order in which the engine tries alternatives.

RewriteBeforeAfter
Single-character branches become a classa|b|c[abc]
Common prefixes are factored outcat|car|cabca[trb]
Later duplicate branches are removedfoo|bar|foofoo|bar
Quantifiers are normalizedx{1}y{0,1}z{0,}w{1,}xy?z*w+
Runs of the same atom are merged\d\d\d-\d\d\d\d\d{3}-\d{4}
Redundant non-capturing groups are unwrapped(?:abc)dabcd

Single-character branches become a class​

Two or more adjacent branches that each match exactly one character are merged into one bracketed class. The order of such branches does not matter, because each one consumes exactly one character.

Console.WriteLine(Pattern.FromRegex("a|b|c").Optimize().Optimized);      // [abc]
Console.WriteLine(Pattern.FromRegex("[ab]|c").Optimize().Optimized);     // [abc]
Console.WriteLine(Pattern.FromRegex(@"\d|x").Optimize().Optimized);      // [\dx]
Console.WriteLine(Pattern.FromRegex(@"\p{L}|0").Optimize().Optimized);   // [\p{L}0]
Console.WriteLine(Pattern.FromRegex(@"\+|-").Optimize().Optimized);      // [+\-]
Console.WriteLine(Pattern.FromRegex("yes|y|n|no").Optimize().Optimized); // yes|[yn]|no
  • Literals, shorthand classes such as \d and \w, Unicode categories such as \p{L} and simple bracketed classes can be merged. Characters are re-escaped for use inside a class, as - is above.
  • Only adjacent branches are merged. In yes|y|n|no, the single-character branches y and n sit between longer branches, so only they become [yn].
  • The merged class must match exactly the union of the branches. Branches that cannot be placed in a class are left alone:
Console.WriteLine(Pattern.FromRegex(".|a").Optimize().IsChanged);              // False
Console.WriteLine(Pattern.FromRegex("[^a]|b").Optimize().IsChanged);           // False
Console.WriteLine(Pattern.FromRegex("[a-z-[aeiou]]|0").Optimize().IsChanged);  // False
Console.WriteLine(Pattern.FromRegex("[a-]|b").Optimize().IsChanged);           // False

These are the dot, negated bracketed classes, classes with subtraction, and classes whose text would change meaning when combined ([a-b] is a range, not a, - and b). Negated shorthand classes are fine, because they keep their meaning inside brackets:

Console.WriteLine(Pattern.FromRegex(@"\D|x").Optimize().Optimized); // [\Dx]

Common prefixes are factored out​

When adjacent branches start with the same single-character atoms, the shared prefix is written once. If every remainder is one character, the remainders become a class. Otherwise they become a non-capturing group.

Console.WriteLine(Pattern.FromRegex("dog|cat|car").Optimize().Optimized);     // dog|ca[tr]
Console.WriteLine(Pattern.FromRegex("abc|abd|xyz").Optimize().Optimized);     // ab[cd]|xyz
Console.WriteLine(Pattern.FromRegex("abcdx1|abcdy2").Optimize().Optimized);   // abcd(?:x1|y2)
Console.WriteLine(Pattern.FromRegex("abcdx1|abcdy2").Optimize().AppliedRewrites[0]);
// Factored common prefix 'abcd' out of 'abcdx1|abcdy2' into 'abcd(?:x1|y2)'

A prefix is factored only when it is safe and useful:

Console.WriteLine(Pattern.FromRegex("cat|dog|car").Optimize().Optimized);     // cat|dog|car
Console.WriteLine(Pattern.FromRegex(@"\bcat\b|\bcar\b").Optimize().IsChanged); // False
Console.WriteLine(Pattern.FromRegex(@"\d+a|\d+b").Optimize().IsChanged);      // False
Console.WriteLine(Pattern.FromRegex("ab|abc").Optimize().IsChanged);         // False
Console.WriteLine(Pattern.FromRegex("Saturday|Sunday").Optimize().IsChanged); // False
  • The branches must be adjacent. cat and car are separated by dog, and moving branches is never allowed.
  • The prefix must consist of single-character atoms: literals and classes. Anchors such as \b and quantified atoms such as \d+ stop the prefix, because they contain choice points that factoring could move.
  • When the remainders cannot form a class, factoring is applied only if the result is not longer than the original. ab|abc would become ab(?:|c), and Saturday|Sunday would become S(?:aturday|unday), so both stay as they are. The .NET engine already finds common prefixes internally, so a longer pattern would help neither readers nor speed.

Later duplicate branches are removed​

A branch that repeats an earlier branch of the same alternation can never match where the first copy failed: it is tried at the same position with the same state. The later copy is removed.

Console.WriteLine(Pattern.FromRegex("foo|bar|foo").Optimize().Optimized); // foo|bar
Console.WriteLine(Pattern.FromRegex("(a|b|a)").Optimize());
// Optimized '(a|b|a)' to '([ab])' (proven equivalent):
//   - Removed duplicate alternation branch 'a'
//   - Merged alternation 'a|b' into '[ab]'
  • Branches are compared by their exact text. In foo|fo{2} the branches match the same string, but their text differs, so nothing is removed. Other rewrites can make two branches identical first: foo|(?:foo) becomes foo after the group is unwrapped.
  • One duplicate is removed per step. foo|bar|foo|bar becomes foo|bar in two steps.
  • A duplicate that contains a capturing or balancing group is kept. Removing it would renumber the groups that follow:
Console.WriteLine(Pattern.FromRegex("(a)|(b)|(a)").Optimize().IsChanged); // False
Console.WriteLine(Pattern.FromRegex("x(a)|x(a)").Optimize().IsChanged);   // False

Quantifiers are normalized​

A quantifier is rewritten into its shortest form.

BeforeAfterRule
x{1}, x{1,1}xExactly once is the same as no quantifier.
x{0,1}x?
x{0,}x*
x{1,}x+
x{2,2}x{2}Equal bounds are written once.
x{0,1}?, x{0,}?, x{1,}?x??, x*?, x+?Lazy quantifiers stay lazy.
x{3}?x{3}A fixed count has nothing to be lazy about.
x{2,}, x{2,5}unchangedAlready the shortest form.
x{0}unchangedA zero-count quantifier is never rewritten.
Console.WriteLine(Pattern.FromRegex("x{1,1}").Optimize().Optimized);  // x
Console.WriteLine(Pattern.FromRegex("x{0,1}?").Optimize().Optimized); // x??
Console.WriteLine(Pattern.FromRegex("x{3}?").Optimize().Optimized);   // x{3}
Console.WriteLine(Pattern.FromRegex("x{0}").Optimize().IsChanged);    // False

Runs of the same atom are merged​

Two or more adjacent copies of the same single-character atom, with or without quantifiers, are merged into one quantified atom. The run tries total lengths in the same order as the merged quantifier, so matches do not change.

Console.WriteLine(Pattern.FromRegex(@"\d\d\d-\d\d\d\d").Optimize().Optimized); // \d{3}-\d{4}
Console.WriteLine(Pattern.FromRegex("[0-9][0-9]").Optimize().Optimized);      // [0-9]{2}
Console.WriteLine(Pattern.FromRegex("aa*b").Optimize().Optimized);            // a+b
Console.WriteLine(Pattern.FromRegex("a*?a").Optimize().Optimized);            // a+?
Console.WriteLine(Pattern.FromRegex("a{2}a{3}").Optimize().Optimized);        // a{5}
Console.WriteLine(Pattern.FromRegex("a{2,3}a{1,2}").Optimize().Optimized);    // a{3,5}
Console.WriteLine(Pattern.FromRegex("a*a*").Optimize().Optimized);            // a*
  • The minimums add up and the maximums add up. Any unbounded part makes the result unbounded.
  • The atom must be a single character: a literal or a class. abab is not merged into (?:ab){2}.
  • The copies must be written the same way. [0-9]\d is not merged, even when both match the same digits.
  • Lazy and greedy parts cannot be mixed. a*?a+ stays as it is. A fixed count such as a or a{2} mixes with either.
  • A merge is applied only when the result is shorter than the run:
Console.WriteLine(Pattern.FromRegex("a+a+").Optimize().IsChanged); // False
Console.WriteLine(Pattern.FromRegex("aa?").Optimize().IsChanged);  // False
Console.WriteLine(Pattern.FromRegex("a?a?").Optimize().IsChanged); // False

a+a+ would become a{2,}, aa? would become a{1,2} and a?a? would become a{0,2}. All three are longer than the original.

Redundant non-capturing groups are unwrapped​

A plain (?:...) group is removed when the parentheses change neither grouping nor options. A group with an inline option such as (?i) directly inside it is never unwrapped, because the option would then leak out of the group.

Console.WriteLine(Pattern.FromRegex("(?:abc)d").Optimize().Optimized);   // abcd
Console.WriteLine(Pattern.FromRegex("(?:a)+").Optimize().Optimized);     // a+
Console.WriteLine(Pattern.FromRegex("(?:)x").Optimize().Optimized);      // x
Console.WriteLine(Pattern.FromRegex("(?:a|bc)").Optimize().Optimized);   // a|bc
Console.WriteLine(Pattern.FromRegex("((?:a|bc))").Optimize().Optimized); // (a|bc)
Console.WriteLine(Pattern.FromRegex("x(?:a|b)").Optimize().Optimized);   // x[ab]

A group is redundant when:

  • it is quantified and its body is a single literal, class or group, as in (?:a)+,
  • its body is an alternation and the group is the whole pattern, a branch of another alternation, or the only content of another group, as in ((?:a|bc)),
  • otherwise, its body is a literal, class, group, sequence, quantifier or nothing at all.

Groups that are kept:

Console.WriteLine(Pattern.FromRegex("(?:ab)+").Optimize().IsChanged);    // False
Console.WriteLine(Pattern.FromRegex("(?:a|bc)d").Optimize().IsChanged);  // False
Console.WriteLine(Pattern.FromRegex("(?i:abc)d").Optimize().IsChanged);  // False
Console.WriteLine(Pattern.FromRegex("(?>ab)c").Optimize().IsChanged);    // False
Console.WriteLine(Pattern.FromRegex(@"(a)\1(?:0)").Optimize().IsChanged); // False
Console.WriteLine(Pattern.FromRegex("(?:(?i)a)b").Optimize().IsChanged);  // False
Console.WriteLine(Pattern.FromRegex("a(?:)b").Optimize().IsChanged);      // False
  • (?:ab)+ repeats two characters. Without the group, + would apply to b only.
  • (?:a|bc)d limits the alternation. Without the group, it would mean a or bcd.
  • (?i:abc) changes options, and (?>ab) is an atomic group. Only plain (?: groups are unwrapped.
  • \1(?:0) would become \10, which .NET reads as a different escape. Groups whose removal would fuse with their neighbours into a different token are kept. The same check stops a(?:{)2} from turning into the quantifier a{2}.
  • (?:(?i)a) holds an inline option, so removing it would make b case-insensitive too.
  • An empty group (?:) is removed only at the start or end of the pattern, as in (?:)x above. Between two neighbours, as in a(?:)b, it is kept by the same fusion guard.

How rewrites combine​

The optimizer applies one rewrite at a time. After each accepted rewrite it parses the new text and looks for more, outermost first, until no safe rewrite is left. So one rewrite often makes the next one possible:

OptimizationResult result = Pattern.FromRegex("^(?:(?:a|b)|c)$").Optimize();

Console.WriteLine(result);
// Optimized '^(?:(?:a|b)|c)$' to '^[abc]$' (proven equivalent):
//   - Unwrapped redundant group '(?:a|b)'
//   - Merged alternation 'a|b|c' into '[abc]'
//   - Unwrapped redundant group '(?:[abc])'

Capturing groups always keep their numbers, names and contents, so code that reads groups keeps working:

using System.Text.RegularExpressions;

Pattern original = Pattern.FromRegex(@"(?<year>\d\d\d\d)-(?<month>\d\d)");

Pattern optimized = original.Optimize().Optimized;
Match match = optimized.Match("Released on 2024-05-17");

Console.WriteLine(optimized);                   // (?<year>\d{4})-(?<month>\d\d)
Console.WriteLine(match.Groups["year"].Value);  // 2024
Console.WriteLine(match.Groups["month"].Value); // 05
Console.WriteLine(match.Index);                 // 12

How equivalence is checked​

The rewrite rules are designed to be safe, but the optimizer does not trust them. Each candidate is applied on its own and kept only when it passes two checks. If either check fails, the candidate is dropped and the optimizer moves on to the next one.

Check 1: match behavior on sample inputs​

The original regex and the rewritten regex run on the same set of inputs. The inputs are built once from the original pattern:

  • the empty string, a space and a newline,
  • 25 matching samples from GenerateMatching (seed 7) and 25 non-matching samples from GenerateNonMatching (seed 11), described in Test Data Generation,
  • for each matching sample, mutated copies: surrounded by x, doubled, with a trailing newline, upper-cased, and with the first or last character removed.

For every input, the two regexes must agree on:

  • the group numbers and group names, in the same order,
  • every successive match (up to the first 20 matches per input): success, index and length,
  • for every group of every match: success, index, length, and the index and length of every capture.

The check runs with a one-second timeout per regex call, without RegexOptions.Compiled. A candidate that times out is rejected. A candidate that is not valid .NET regex syntax is rejected too.

Check 2: proof with language algebra​

When the original pattern is in the regular subset that language algebra supports, the candidate must also be proven to match exactly the same strings as the original. The optimizer builds an automaton for the original once and compares each candidate's automaton with it.

The regular subset covers literals, classes, capturing, named and non-capturing groups, alternation, quantifiers, the anchors ^, \A, $, \Z and \z, and lookaheads at the very start of the pattern. It excludes:

  • backreferences, conditionals and balancing groups,
  • lookbehinds, and lookaheads anywhere other than the start,
  • atomic groups (?>...),
  • the word boundaries \b and \B, and \G,
  • ^ and $ under RegexOptions.Multiline (use \A and \z instead),
  • RegexOptions.RightToLeft.

Language algebra also has size limits: a repeat bound above 1000, or an automaton that grows too large, puts the pattern outside the subset.

Check 1 catches differences in leftmost-first behavior and captures that a proof about matched strings cannot see. Check 2 catches differences on inputs the samples happen to miss. Together they are much stronger than either alone.

Proven or checked on samples​

VerifiedEquivalent tells you which checks ran:

VerifiedEquivalentChecksMeaning
true1 and 2Every applied rewrite is proven to match the same strings, and was seen to behave the same on the samples.
false1 onlyThe pattern uses a construct outside the regular subset, or is too large for language algebra. The same order-preserving rewrites are applied, but they were only checked on samples.

Patterns outside the regular subset are still optimized:

Console.WriteLine(Pattern.FromRegex(@"(cat|car)\1").Optimize());
// Optimized '(cat|car)\1' to '(ca[tr])\1' (checked on samples, not proven):
//   - Merged alternation 'cat|car' into 'ca[tr]'

Console.WriteLine(Pattern.FromRegex("(?<=x)(?:a|b|c)").Optimize().Optimized);                 // (?<=x)[abc]
Console.WriteLine(Pattern.FromRegex("a|b", RegexOptions.RightToLeft).Optimize().Optimized);   // [ab]
Console.WriteLine(Pattern.FromRegex("x{1}a{2000}").Optimize().VerifiedEquivalent);            // False
Console.WriteLine(Pattern.FromRegex(@"(?=\d)(?:a|b)").Optimize().VerifiedEquivalent);         // True
Console.WriteLine(Pattern.FromRegex(@"\b(?:a|b)").Optimize().VerifiedEquivalent);             // False
Console.WriteLine(Pattern.FromRegex("^(?:a|b)$").Optimize().VerifiedEquivalent);              // True
Console.WriteLine(Pattern.FromRegex("^(?:a|b)$", RegexOptions.Multiline).Optimize().VerifiedEquivalent); // False
  • The backreference \1, the lookbehind (?<=x), the RightToLeft option, the repeat {2000}, the word boundary \b and ^/$ under Multiline put those patterns outside the subset. All of them are still optimized.
  • A leading lookahead such as (?=\d) is inside the subset, so that rewrite is proven.
tip

When VerifiedEquivalent is false, add tests for the optimized pattern before you replace the original. Property-based testing and regex coverage are good ways to exercise it.

When a rewrite is skipped​

The optimizer prefers leaving a pattern alone to changing how it behaves. A rewrite is not offered, or is rejected, in these cases:

SituationExampleWhy
The rewrite would change which match is founda|abab? matches the same strings, but on ab it finds ab where a|ab finds a. Branches are never reordered or combined in a way that moves a choice.
Removing a branch would renumber groups(a)|(b)|(a)The third branch is a duplicate, but it contains a capturing group.
A branch of the alternation has inline options(?i)cat|carOptions that change inside a branch make the branches incomparable. No branch of that alternation is merged, factored or removed. Rewrites inside the branches still apply: (?i)cat|car{1} becomes (?i)cat|car.
The result would be longerab|abc, a+a+See Common prefixes are factored out and Runs of the same atom are merged.
The construct is a conditional or a balancing groupx{1}(?(y)a{1}|b)Nothing inside (?(...)...) or (?<-name>...) is rewritten. Rewrites outside still apply: this becomes x(?(y)a{1}|b).
Removing parentheses or a quantifier would fuse tokens(a)\1{1}0Dropping {1} would turn \1 and 0 into the escape \10. The candidate fails Check 1 and is rejected.
The rewritten pattern cannot be provena{999}a{2}a{1001} is over the language algebra repeat limit of 1000, so the proof fails and the merge is rejected.
Console.WriteLine(Pattern.FromRegex("(?i)cat|car").Optimize());       // No rewrites applied to '(?i)cat|car'.
Console.WriteLine(Pattern.FromRegex("a{999}a{2}").Optimize());        // No rewrites applied to 'a{999}a{2}'.
Console.WriteLine(Pattern.FromRegex(@"(a)\1{1}0").Optimize());        // No rewrites applied to '(a)\1{1}0'.
Console.WriteLine(Pattern.FromRegex("x{1}(?(y)a{1}|b)").Optimize());
// Optimized 'x{1}(?(y)a{1}|b)' to 'x(?(y)a{1}|b)' (checked on samples, not proven):
//   - Removed redundant quantifier '{1}' from 'x{1}'

Work limits​

To keep Optimize() bounded on very large patterns, two limits apply:

LimitValueWhat happens when it is reached
Applied rewrites100The optimizer stops and returns the pattern with the first 100 rewrites applied.
Candidate verifications400The optimizer stops and returns the pattern with the rewrites accepted so far.

A candidate text that was already tried is not verified again. When a limit is reached, the result is still correct; it is just not as short as it could be. Call Optimize() again on result.Optimized to continue:

using System.Linq;

// 120 copies of "x{1}y": one quantifier rewrite each
Pattern large = Pattern.FromRegex(string.Concat(Enumerable.Repeat("x{1}y", 120)));

OptimizationResult first = large.Optimize();
OptimizationResult second = first.Optimized.Optimize();

Console.WriteLine(first.AppliedRewrites.Count);  // 100
Console.WriteLine(second.AppliedRewrites.Count); // 20
Console.WriteLine(second.Optimized.ToString() == string.Concat(Enumerable.Repeat("xy", 120))); // True

The limits are fixed. There is no option to raise them.

Options and timeout are preserved​

The optimized pattern keeps the original's RegexOptions and match timeout.

Pattern original = Pattern.FromRegex("cat|car", RegexOptions.IgnoreCase).WithTimeout(TimeSpan.FromSeconds(3));

OptimizationResult result = original.Optimize();

Console.WriteLine(result.Optimized);                // ca[tr]
Console.WriteLine(result.Optimized.Options);        // IgnoreCase
Console.WriteLine(result.Optimized.Timeout);        // 00:00:03
Console.WriteLine(result.Optimized.IsMatch("CAR")); // True
  • Every option is kept, including RegexOptions.Compiled. Only the internal behavior check runs without Compiled.
  • With RegexOptions.IgnorePatternWhitespace, rewritten parts lose their layout whitespace and # comments. The option itself is kept:
Pattern layout = Pattern.FromRegex("c a t | c a r  # pets", RegexOptions.IgnorePatternWhitespace);

OptimizationResult result = layout.Optimize();

Console.WriteLine(result.Optimized);         // ca[tr]
Console.WriteLine(result.Optimized.Options); // IgnorePatternWhitespace
warning

If you keep a pattern with IgnorePatternWhitespace for its comments, those comments are gone from the optimized text. Keep the commented original in source control and treat the optimized text as output.

Optimizing built-in patterns​

Common Patterns are already tight, but a few still contain groups added while composing them. IPv4() drops four redundant groups, and the result is proven to accept exactly the same addresses:

Pattern ipv4 = CommonPatterns.IPv4();

OptimizationResult result = ipv4.Optimize();

Console.WriteLine(ipv4.ToString().Length);             // 140
Console.WriteLine(result.Optimized.ToString().Length); // 124
Console.WriteLine(result.AppliedRewrites.Count);       // 4
Console.WriteLine(result.Optimized.IsEquivalentTo(ipv4)); // True
Console.WriteLine(result.Optimized);
// ^(25[0-5]|2[0-4]\d|[01]?\d\d?)\.(25[0-5]|2[0-4]\d|[01]?\d\d?)\.(25[0-5]|2[0-4]\d|[01]?\d\d?)\.(25[0-5]|2[0-4]\d|[01]?\d\d?)$

IsEquivalentTo is described in Pattern Algebra. Patterns such as CommonPatterns.Email() and CommonPatterns.Guid() have nothing to optimize and come back unchanged.

From the command line and the playground​

The fluentregex tool has an optimize command that prints the same summary as ToString():

fluentregex optimize "cat|car|cab"
# Optimized 'cat|car|cab' to 'ca[trb]' (proven equivalent):
#   - Merged alternation 'cat|car|cab' into 'ca[trb]'

fluentregex optimize "Cat|Car" --options IgnoreCase
# Optimized 'Cat|Car' to 'Ca[tr]' (proven equivalent):
#   - Merged alternation 'Cat|Car' into 'Ca[tr]'

fluentregex optimize "\d\d\d" --ascii-digits
# Optimized '[0-9][0-9][0-9]' to '[0-9]{3}' (proven equivalent):
#   - Merged repeated atoms '[0-9][0-9][0-9]' into '[0-9]{3}'
  • --options takes a comma-separated list of RegexOptions names, ignoring case.
  • --ascii-digits rewrites \d to [0-9] before optimizing.
  • The exit code is 0 whether or not a rewrite applied, and 2 for an invalid regex or bad arguments.

See Command-Line Tool for installation and the other commands. The Playground has an Optimize tab that shows the optimized pattern, the applied rewrites and whether equivalence was proven.

Keeping patterns optimal in tests​

Because Optimize() is deterministic, you can use it in a unit test to fail the build when a pattern can be simplified:

using Masterly.FluentRegex;
using Xunit;

public class PhonePatternTests
{
    [Fact]
    public void PhonePattern_IsAlreadyOptimal()
    {
        Pattern phone = Pattern.FromRegex(@"^\d{3}-\d{4}$");

        OptimizationResult result = phone.Optimize();

        Assert.False(result.IsChanged, result.ToString());
    }
}
  • The failure message is the optimizer's summary, so it tells you exactly which rewrite to apply.
  • The same check works for patterns built with the fluent API. For more test helpers, see xUnit Integration.

Errors​

Optimize() itself takes no arguments and is designed not to throw for valid patterns:

  • An invalid regex is rejected earlier, by Pattern.FromRegex, with an ArgumentException whose message starts with The regular expression is invalid:. For string literals, the compile-time checks report this as FRX006 before you run the code.
  • A candidate that is not valid regex, times out during the behavior check, or cannot be proven is silently rejected. It never surfaces as an exception.
  • If the library's own regex parser cannot read the pattern, Optimize() throws InvalidOperationException with a message starting with The pattern is not valid regex syntax:. This is not expected for any pattern that .NET accepts.
// userRegex comes from your application, for example "(cat|car"
try
{
    Pattern pattern = Pattern.FromRegex(userRegex);
    Console.WriteLine(pattern.Optimize().Optimized);
}
catch (ArgumentException ex)
{
    Console.WriteLine(ex.Message);
    // The regular expression is invalid: Invalid pattern '(cat|car' at offset 8. Not enough )'s. (Parameter 'regex')
}

Tips and pitfalls​

  • Optimize at design time, not per request. Optimize() generates samples, builds automata and runs up to 400 verifications. Run it once, then use the optimized pattern or paste its text into your code. Do not call it in a hot path.
  • Do not expect a big speed-up. The .NET engine already applies many of these simplifications internally when it compiles a regex. The main gain is a shorter pattern that is easier to read and review. Measure with your own inputs before you assume a pattern got faster; the Backtracking Profiler helps.
  • The optimizer does not fix ReDoS. It only applies rewrites that keep behavior identical, so a pattern that backtracks badly will still backtrack badly. Use ReDoS Safety for that.
  • The optimizer does not change what a pattern means. If a|ab should really find ab, that is a bug in the pattern. Fix it yourself; the optimizer will not.
  • Check VerifiedEquivalent before trusting a rewrite blindly. false means the rewrites passed hundreds of sample comparisons, not a proof.
  • One call is usually enough. When a rewrite makes another rewrite possible, the same call applies it. Only when a very large pattern hits the work limits do you need to call Optimize() again on the result.
  • Group numbers never change. You can swap the optimized pattern in without touching code that reads Groups[1] or Groups["name"].