Simplify regular expression? (Converting Unicode fractions to TeX)

Question

Background

I'm converting Unicode text to TeX for typesetting. In the input, I'm allowing simple fractions like ½ and ⅔ using single Unicode characters and complex fractions like ¹²³/₄₅₆ using superscripted and subscripted numerals. First I convert the simple fractions (½ becomes \frac{1}{2}) and superscripted and subscripted numerals (¹²³ becomes $^{123}$ and ₄₅₆ becomes $_{456}$) using a character lookup table, then I make a second pass to collapse runs of numerals and combine numerator and denominator into a fraction (so ¹²³/₄₅₆ then becomes \frac{123}{456}). Finally, I make a third pass to insert a ⅙-em thin space between an integer and a fraction (so that, for example, 2¼ displays as 2 ¼).

Question(s)

Code works great, but I'm wondering how to simplify the regular expressions. There are three transformations.

• In the first transformation, is there a way to avoid the alternation operator (|) and simply match on either \^ or \_? I don't see a way to do it using ([\^\_]) and \1.
• Also in the first transformation, is there a way to avoid the nested substitutions?
• Is there some completely other solution to this that would work even better? (By better I don't mean faster but easier to understand.)

Here are the guts:

# First collapse runs of superscripted and/or subscripted numerals.
$text =~ s{
    (
        (?: \$ \^ [0-9] \$ ){2,}         # Match superscripted numerals
      | (?: \$ \_ [0-9] \$ ){2,}         # Match subscripted numerals
    )
}{
    my $x = $1;
    $x =~ s{\$}{}g;                      # Remove '$'s
    $x =~ s{(?<!^)[\^\_]}{}g;            # Remove redundant '^'s and '_'s
    $x =~ s{^(.)(.*)$}{\$${1}{$2}\$};    # Wrap in '{}' and replace '$'.
    $x
}xeg;

# Now combine complete fractions.
$text =~ s{
    \$ \^ \{? ([0-9]+) \}? \$            # $1 = numerator
    (?: / | \x{2044} )                   # Slash or Fraction Slash
    \$ \_ \{? ([0-9]+) \}? \$            # $2 = denominator
}{
    "\\frac{$1}{$2}"
}xeg;

# Finally, add a thin space between a whole number and a fraction.
# (But do not add space between a whole number and an exponent.)
$text =~ s{(?<=[0-9])(?=\\frac\{.*?\}\{.*?\})}{\\,}g;

Note that this (probably) cannot be done in a single translation step, because it must also collapse superscripts like ¹²³ without an accompanying fraction, for if ¹²³ is left as $^1$$^2$$$^3$ (instead of collapsing it to $^{123}$), then it will contain undesirable tiny spaces between the numerals.

Additionally, it must convert 2¼ (add thin space) to 2 ¼ when it appears in the input using either the single character ¼ or the pair of superscript/subscript characters ¹ and ₄. Hence the very separate translation steps.

Test cases

Use ⅔ cup chopped garlic and 1½ cups chopped onion.
➥ Use \frac{2}{3} cup chopped garlic and 1\frac{1}{2} cups chopped onion.
➥ Use \frac{2}{3} cup chopped garlic and 1\,\frac{1}{2} cups chopped onion.

This DeLorean DMC-12 runs on ²³⁹Pu and uses 1.21×10⁹ W.
➥ This DeLorean DMC-12 runs on $^2$$^3$$^9$Pu and uses 1.21$\times$10$^9$ W.
➥ This DeLorean DMC-12 runs on $^{239}$Pu and uses 1.21$\times$10$^9$ W.

A googol is 10¹⁰⁰
➥ A googol is 10$^1$$^0$$^0$
➥ A googol is 10$^{100}$

1 ns = ¹/₁₀₀₀ µs
➥ 1 ns = $^1$/$_1$$_0$$_0$$_0$ µs
➥ 1 ns = \frac{1}{1000} µs

π ≅ ²²/₇ = 3¹/₇
➥ π ≅ $^2$$^2$/$_7$ = 3$^1$/$_7$
➥ π ≅ \frac{22}{7} = 3\,\frac{1}{7}

Complete working example

Here is the complete program, with test cases:

#!/usr/bin/perl -w
use strict;
use utf8;
binmode STDOUT, ":utf8";

my $unicode_to_tex = {
    # Unicode     TeX

    "\x{2070}" => "\$^0\$",            # ⁰ (superscript 0)
    "\x{00B9}" => "\$^1\$",            # ¹ (superscript 1)
    "\x{00B2}" => "\$^2\$",            # ² (superscript 2)
    "\x{00B3}" => "\$^3\$",            # ³ (superscript 3)
    "\x{2074}" => "\$^4\$",            # ⁴ (superscript 4)
    "\x{2075}" => "\$^5\$",            # ⁵ (superscript 5)
    "\x{2076}" => "\$^6\$",            # ⁶ (superscript 6)
    "\x{2077}" => "\$^7\$",            # ⁷ (superscript 7)
    "\x{2078}" => "\$^8\$",            # ⁸ (superscript 8)
    "\x{2079}" => "\$^9\$",            # ⁹ (superscript 9)

    "\x{2080}" => "\$_0\$",            # ₀ (subscript 0)
    "\x{2081}" => "\$_1\$",            # ₁ (subscript 1)
    "\x{2082}" => "\$_2\$",            # ₂ (subscript 2)
    "\x{2083}" => "\$_3\$",            # ₃ (subscript 3)
    "\x{2084}" => "\$_4\$",            # ₄ (subscript 4)
    "\x{2085}" => "\$_5\$",            # ₅ (subscript 5)
    "\x{2086}" => "\$_6\$",            # ₆ (subscript 6)
    "\x{2087}" => "\$_7\$",            # ₇ (subscript 7)
    "\x{2088}" => "\$_8\$",            # ₈ (subscript 8)
    "\x{2089}" => "\$_9\$",            # ₉ (subscript 9)

    "\x{00BD}" => "\\frac{1}{2}",      # ½ (fraction one half)
    "\x{2153}" => "\\frac{1}{3}",      # ⅓ (fraction one third)
    "\x{2154}" => "\\frac{2}{3}",      # ⅔ (fraction two thirds)
    "\x{00BC}" => "\\frac{1}{4}",      # ¼ (fraction one fourth)
    "\x{00BE}" => "\\frac{3}{4}",      # ¾ (fraction three fourths)
    "\x{2155}" => "\\frac{1}{5}",      # ⅕ (fraction one fifth)
    "\x{2156}" => "\\frac{2}{5}",      # ⅖ (fraction two fifths)
    "\x{2157}" => "\\frac{3}{5}",      # ⅗ (fraction three fifths)
    "\x{2158}" => "\\frac{4}{5}",      # ⅘ (fraction four fifths)
    "\x{2159}" => "\\frac{1}{6}",      # ⅙ (fraction one sixth)
    "\x{215A}" => "\\frac{5}{6}",      # ⅚ (fraction five sixths)
    "\x{215B}" => "\\frac{1}{8}",      # ⅛ (fraction one eighth)
    "\x{215C}" => "\\frac{3}{8}",      # ⅜ (fraction three eighths)
    "\x{215D}" => "\\frac{5}{8}",      # ⅝ (fraction five eighths)
    "\x{215E}" => "\\frac{7}{8}",      # ⅞ (fraction seven eighths)

    "\x{00D7}" => "\$\\times\$",       # × (multiplication sign)
};

sub convert($) {
    my ($text) = @_;
    $text =~ s{(.)}{$unicode_to_tex->{$1}||$1}eg;
    return $text;
}

sub combine($) {
    my ($text) = @_;

    # First collapse runs of superscripted and/or subscripted numerals.
    $text =~ s{
        (
            (?: \$ \^ [0-9] \$ ){2,}         # Match superscripted numerals
          | (?: \$ \_ [0-9] \$ ){2,}         # Match subscripted numerals
        )
    }{
        my $x = $1;
        $x =~ s{\$}{}g;                      # Remove '$'s
        $x =~ s{(?<!^)[\^\_]}{}g;            # Remove redundant '^'s and '_'s
        $x =~ s{^(.)(.*)$}{\$${1}{$2}\$};    # Wrap in '{}' and replace '$'.
        $x
    }xeg;

    # Now combine complete fractions.
    $text =~ s{
        \$ \^ \{? ([0-9]+) \}? \$            # $1 = numerator
        (?: / | \x{2044} )                   # Slash or Fraction Slash
        \$ \_ \{? ([0-9]+) \}? \$            # $2 = denominator
    }{
        "\\frac{$1}{$2}"
    }xeg;

    # Finally, add a thin space between a whole number and a fraction.
    # (But do not add space between a whole number and an exponent.)
    $text =~ s{(?<=[0-9])(?=\\frac\{.*?\}\{.*?\})}{\\,}g;

    return $text;
}

while (defined(my $text = <DATA>)) {
    print $text;
    $text = convert($text);
    print $text;
    $text = combine($text);
    print $text;
    print "\n";
}

__DATA__
Use ⅔ cup chopped garlic and 1½ cups chopped onion.
This DeLorean DMC-12 runs on ²³⁹Pu and uses 1.21×10⁹ W.
A googol is 10¹⁰⁰
1 ns = ¹/₁₀₀₀ µs
π ≅ ²²/₇ = 3¹/₇

Answer 1

(Disclaimer: I don't know perl, just regular expressions)

First question

is there a way to avoid the alternation operator (|) and simply match on either \^ or \_?

    (?: \$ \^ [0-9] \$ ){2,}         # Match superscripted numerals
  | (?: \$ \_ [0-9] \$ ){2,}         # Match subscripted numerals

Try doing this:

    (?: \$ [_^] [0-9] \$ ){2,}         # Match (super|sub)scripted numerals

(edit: this above regex actually wouldn't work for this case, see comments below for another suggestion)

The item [_^] is a Character Class - same as [0-9]. Perl syntax might require the escape characters, in which case it would look like [\^\_] - but generally in regexes you don't need to escape inside a character class. As you mentioned, in this case if the ^ character is first then it is a negated character class - and it tends to be up to the developer (my preference is to escape only when necessary).

Here's some more information on character classes: http://www.regular-expressions.info/charclass.html

Second question

in the first transformation, is there a way to avoid the nested substitutions?

I assume you mean this:

$x =~ s{\$}{}g;                      # Remove '$'s
$x =~ s{(?<!^)[\^\_]}{}g;            # Remove redundant '^'s and '_'s
$x =~ s{^(.)(.*)$}{\$${1}{$2}\$};    # Wrap in '{}' and replace '$'.

With regular expressions, I've found that the most compact ones aren't necessarily the easiest ones to read. The way you have it split up here and documented seems helpful if it doesn't cause a performance hit. That said, you could probably combine the first two lines like this:

$x =~ s{\$|(?<!^)[_^]}{}g;           # Remove '$'s and redundant ^ and _ chars

Third question

Is there some completely other solution to this that would work even better?

I can only think of one way to do part of this differently... In the beginning, it looks like you can safely (maybe?) hunt down and kill ALL $$^ instances, e.g.:

This DeLorean DMC-12 runs on $^2$$^3$$^9$Pu and uses 1.21$\times$10$^9$ W. This DeLorean DMC-12 runs on $^239$Pu and uses 1.21$\times$10$^9$ W.

Then you could search for instances of (<?=\$^)(.*)(?=\$) and wrap those with the {} characters:

This DeLorean DMC-12 runs on $^{239}$Pu and uses 1.21$\times$10$^9$ W.

You'll have to make the best judgement as to whether or not $$^ occurs validly another way, however, in which case this wouldn't help.

You could also write it as a humongous sed command if you're looking to spend a lot of time.