Functions to convert camelCase strings to snake_case

Question

I have written two functions, one actual and one helper, to convert camelCase strings into snake_case strings (I call it joint-lower case). Any ideas for improvement are welcome.

def _cc2jl(string):
    """Camel case to joint-lower helper."""

    for index, current in enumerate(string):

        if is_upper(current):
            if index > 0:
                previous = string[index-1]

                if is_upper(previous):
                    try:
                        next = string[index+1]
                    except IndexError:
                        yield current.lower()
                        continue
                    else:
                        if is_upper(next):
                            yield current.lower()
                            continue

                yield '_' + current.lower()
                continue

            yield current.lower()
            continue

        yield current


def cc2jl(string):
    """Converts a camel case string to joint-lower."""

    return ''.join(_cc2jl(string))

Expected behaviour:

>>> cc2jl('m')
'm'
>>> cc2jl('AA')
'aa'
>>> cc2jl('MySQLDatabase')
'my_sql_database'
>>> cc2jl('GarbageCollection')
'garbage_collection'
>>> cc2jl('AAAAAAAAAAAAAAAAA')
'aaaaaaaaaaaaaaaaa'

Answer 1

I think regex would be the easiest to do this with. You just need to find each word in the string.

1. The name starts with an uppercase letter. [A-Z]. However due to the way the rest of the regex works, we can change this to ., so that we match all words, even ones that start with _.

2. The word will either contain uppercase or lowercase letters, both with other characters - excluding _.

   • Uppercase:

     1. The word won't be lowercase or have a _. [^a-z_]+
     2. You don't want the last uppercase letter or a _. (?=[A-Z_])
     3. You want the above if it's the last in the string. (?=[A-Z_]|$)

   • Lowercase:

     1. The word won't be uppercase or have a _. [^A-Z_]+

And so you can use:

(.(?:[^a-z_]+(?=[A-Z_]|$)|[^A-Z_]+))

You then want to apply the following to these:

1. Prepend the name with a single _, unless:

   • It's the first word in the name
   • The word already starts with a _

2. Convert the word to lowercase

Making:

def _jl_match(match):
    group = match.group()
    prefix = bool(match.start() and not group.startswith('_'))
    return '_' * prefix + group.lower()

And so I'd use:

def _jl_match(match):
    group = match.group()
    prefix = bool(match.start() and not group.startswith('_'))
    return '_' * prefix + group.lower()

REGEX = r'(.(?:[^a-z_]+(?=[A-Z_]|$)|[^A-Z_]+))'

def _cc2jl(string):
    return re.subn(REGEX, _jl_match, string)[0]

Answer 2

A couple of things:

1. Why 2 functions?
2. Now that we established (1), you can get rid of the yields. yield is useful when you want to process stuff in chunks and do not want to wait for the entire population to be created first. This is not the case here.
3. There are some continues that do make sense and some that are redundant. Yours are of the latter type.
4. Nested if statements without corresponding else clauses can be merged with and if readability is not reduced.

Putting all that together, we get the following:

def cc2jl(my_str):
  """Camel case to joint-lower"""

  r = my_str[0].lower()
  for i, letter in enumerate(my_str[1:], 1):
    if letter.isupper():
      try:
        if my_str[i-1].islower() or my_str[i+1].islower():
          r += '_'
      except IndexError:
        pass
    r += letter.lower()
  return r

print(cc2jl('m'))                  #-> m
print(cc2jl('AA'))                 #-> aa
print(cc2jl('MySQLDatabase'))      #-> my_sql_database
print(cc2jl('GarbageCollection'))  #-> garbage_collection
print(cc2jl('AAAAAAAAAAAAAAAAA'))  #-> aaaaaaaaaaaaaaaaa

---

Finally, some thoughts on avoiding the try block that bothers me personally.

We can leverage from the short-circuiting used in Python's logical expressions and (re-)write the following:

def cc2jl(my_str):
  """Camel case to joint-lower"""

  r = my_str[0].lower()
  for i, letter in enumerate(my_str[1:], 1):
    if letter.isupper():
      if my_str[i-1].islower() or (i != len(my_str)-1 and my_str[i+1].islower()):
        r += '_'
    r += letter.lower()
  return r

Note the i != len(my_str)-1 on the left side of the and. If this returns False the my_str[i+1] is not evaluated at all and thus cannot raise the IndexError.

Answer 3

First a note on naming: cc2jl is a very cryptic name. Give the public function a clearer name, like to_snake_case or something similar (note that the function does not do anything to a string that already is in snake_case).

---

While I quite like the fact that you used a generator approach, which guarantees that you make only on pass over the string, this seems like the perfect place for a regular expression. Some quick googling finds this rather simple function:

import re

def convert(name):
    s1 = re.sub('(.)([A-Z][a-z]+)', r'\1_\2', name)
    return re.sub('([a-z0-9])([A-Z])', r'\1_\2', s1).lower()

This is definitely shorter, even though that cryptic regex is not very easy to understand. But following your code, while written quite straightforward, is also not easy, especially with the deep nesting.

It works by first splitting runs of upper-case letters followed by one or more lowercase letters so that the last upper-case letter goes to the lower-case letter run. So "ABCdef" becomes "AB_Cdef".

Then it separates runs of not-upper-case letters, followed by a single upper-case letter with a "_", so "abcD" becomes "abc_D".

To get a bit more performance out of this, you should pre-compile the regexes:

first_cap_re = re.compile('(.)([A-Z][a-z]+)')
all_cap_re = re.compile('([a-z0-9])([A-Z])')
def convert(name):
    s1 = first_cap_re.sub(r'\1_\2', name)
    return all_cap_re.sub(r'\1_\2', s1).lower()

---

Performance wise, all algorithms are similar:

Not quite surprising, they all give a linear behavior (as a function of string length).

My regex version needs to make two passes over the string and is therefore consistently slower. The function written by @Ev.Kounis in his answer outperforms both of ours, but the regex approach by @Peilonrayz is even faster, because it manages to do only one pass, but with a regex.

Note that all functions are very fast, so as long as you need this less than a few thousand times per second, any is fine.

---

The test strings were generated with this code:

import random
import string

strings = [''.join(random.choice(string.ascii_letters) for _ in range(n))
           for n in range(10, 2000, 20)]

In Python 3 it could have been:

strings = [''.join(random.choices(string.ascii_letters, k=k))
           for k in range(10, 2000, 20)]

Answer 4

It can be a bit shorter:

def cc2jl(s):
  return "".join(["_"+l if i and l.isupper() and not s[i-1:i+2].isupper() else l for i, l in enumerate(s)]).lower()

Regular Expression Alternative:

rx = re.compile(r"(?<=.)(((?<![A-Z])[A-Z])|([A-Z](?=[a-z])))")
def cc2jl(s):
    return rx.sub("_\\1", s).lower()