Regex to match several slightly different strings

Question

I work with a charity that does a number of jobs, more than 100, less than 1000 a year. Jobs are identified with a year, and an up to 3 digit serial number, so 2021#123 for instance.

I'm writing a Python report program to pull jobs out of our database, and I'd like be be able to specify ranges of jobs rather like a print dialogue can specify ranges of pages, so a dash separated range 2021#123-45, or a comma separated list 2021#003,5,07,89,131 as well. I want to be able to parse all of these representations out of text strings.

I'm only interested in the syntax at the moment, the semantics of whether a range is valid is handled later. Even though I have very limited experience of regexes, I felt a one-liner would beat a whole bunch of python ifs.

The valid strings I want to match are therefore of the form

• 20y#n OR
• 20y#n,q,p,r OR
• 20y#n-m

where y is any two digits, and m,n,p,q,r are 1 to 3 digits.

This is a question about the regex. The only python consideration is the use of r'…' for a raw string.

I struggled and failed to write a regex that captures any of these forms by trying to match the year#ddd, and then OR the different endings. I think I fell foul of the single job form not having an ending at all. I've therefore ended up with this rather agricultural effort, that works, but looks ugly. I didn't put the comments in just for this post; I put them in knowing that if I needed to change things next month, I'd need them.

job_patt = r'((?:20\d\d#\d{1,3}-\d{1,3})|(?:20\d\d#\d{1,3}(?:,\d{1,3})+)|(?:20\d\d#\d{1,3}))'
# cap group  (                                                                             )
# non-caps    (?:                      ) (?:                           ) (?:              )
# OR the non-caps, in this L->R order   |                               |
# get year#      20\d\d#                    20\d\d#                         20\d\d#
# get 1 to 3 digits     \d{1,3}                    \d{1,3}                         \d{1,3}
# get one instance of range    -\d{1,3}
# get one or more repeats of list         ',ddd'          (?:,\d{1,3})+

What could be done to shorten or tidy up this regex?

Answer 1

This isn't so much a Python question as it is a RegEx question. The Python tag is almost irrelevant.

'+' -vs- '*'

You regex includes two alternates:

• (?:20\d\d#\d{1,3}(?:,\d{1,3})+)
• (?:20\d\d#\d{1,3})

The difference in these expressions is the first one include one or more repeats of ,\d{1,3}, where as the second does not include any repeats of that ...

... in other words, you have 20\d\d#\d{1,3} followed by 0 or more repeats of ,\d{1,3}.

So we can replace the + modifier with the * modifier, and the pair of alternates becomes simply

(?:20\d\d#\d{1,3}(?:,\d{1,3})*)

Common prefix

All three (or rather, both now that the last two have been combined into one) regex's start with 20\d\d#. You can factor this out of both patterns.

20\d\d# (?: (?: \d{1,3}-\d{1,3} | \d{1,3}(?:,\d{1,3})* ) )

Capture Groups

There is no point creating a capture group for the entire regex expression. The entire match is always captured, as group 0.

Reworked Regex & Example

The regex can be rewritten as:

r"20\d\d#(?:(?:\d{1,3}-\d{1,3}|\d{1,3}(?:,\d{1,3})*))"

which is shorter, but is still quite unreadable. It is better for future programmers at the charity (as well as for yourself) to document the regex using the re.VERBOSE option. I find it clearer than comments below the regular expression that try to document the parts of the regular expression through vertically lined up notes.

Note: Since comments follow the octothorpe (#) character in re.VERBOSE mode, we need to escape it (\#) to indicate that it is actually a character we want matched, and not the start of a comment.

import re

JOB_PATTERN = re.compile(r"""
                          20\d\d                     # A 4-digit year
                          \#                         # followed by a hash mark
                          (?:                        # followed by either ...
                             (?:\d{1,3}-\d{1,3})        # a range of serial #'s
                             |                          # or
                             (?:\d{1,3}(?:,\d{1,3})*)   # a list of serial #'s
                          )""", re.VERBOSE)


test_cases = ["junk2021#123-45bar",
              "baz2021#003,5,07,89,131foo"]

for test_case in test_cases:
    m = JOB_PATTERN.search(test_case)
    if m:
        print("Found:", m[0])

Output:

Found: 2021#123-45
Found: 2021#003,5,07,89,131

Update

Based on the OP's response to RootTwo's comment, named capturing groups are actually desired. Here is an updated example with RootTwo's enhancement:

import re

JOB_PATTERN = re.compile(
    r"""(?P<year>              # Capturedas "year" ...
           20\d\d                 # a 4-digit year
        )
        \#                     # followed by a hash mark
        (?:                    # followed by either ...
           (?P<range>             # captured as "range" ...
              \d{1,3}-\d{1,3}        # a range of serial #'s
           )
           |                      # or
           (?P<list>              # captured as "list" ...
               \d{1,3}               # a serial number
               (?:,\d{1,3})*         # or list of comma-seperated serial #'s
           )
        )
        """, re.VERBOSE)

test_cases = ["junk2021#123-45bar",
              "baz2021#003,5,07,89,131foo"]

for test_case in test_cases:
    m = JOB_PATTERN.search(test_case)
    if m:
        if m.lastgroup == "range":
            start, end = m['range'].split('-')
            print("Year:", m['year'], " from", start, "to", end)
        else:
            serial_numbers = m['list'].split(',')
            print("Year:", m['year'], " serial #':", serial_numbers)

Output:

Year: 2021  from 123 to 45
Year: 2021  serial #': ['003', '5', '07', '89', '131']