Golang - Splitting a string by a separator not prefixed by an escape string

Question

I need to split a string by a separator but don't want to split if the separator is prefixed by the escape string.

For example:

"This works\\ but it\\ isn't pretty." with " " as the separator and "\\" as the escape string should produce the following: []string{"This", "works but", "it isn't", "pretty."}

I wrote the following code for it:

func SplitWithEscaping(s string, separator string, escapeString string) []string {
    untreated := strings.Split(s, separator)
    toReturn := make([]string, 0, len(untreated))
    for i := 0; i < len(untreated); i++ {
        next, ii := t(untreated, i, separator, escapeString)
        i = ii - 1
        toReturn = append(toReturn, strings.ReplaceAll(next, escapeString+separator, separator))
    }
    return toReturn
}

func t(stringSlice []string, i int, seperator, escapeString string) (string, int) {
    if !strings.HasSuffix(stringSlice[i], escapeString) {
        return stringSlice[i], i + 1
    }
    next, ii := t(stringSlice, i+1, seperator, escapeString)
    return stringSlice[i] + seperator + next, ii
}

This is the playground link for my working code: https://play.golang.org/p/jfHFt9_vtE7

How can I make my code prettier but also more performant?

Answer 1

One approach that's simple, but is a bit of a dirty trick: first replace sequences of escape+separator with a string that's never going to occur in your text (for example a NUL byte "\x00"), then do the split, then do the reverse replace on each token. For example (Go Playground link):

func SplitWithEscaping(s, separator, escape string) []string {
    s = strings.ReplaceAll(s, escape+separator, "\x00")
    tokens := strings.Split(s, separator)
    for i, token := range tokens {
        tokens[i] = strings.ReplaceAll(token, "\x00", separator)
    }
    return tokens
}

Your approach of splitting and then re-joining if the last character was an escape works, but it's a bit tricky, and it does more work than necessary. You may also want to name the t function a bit more meaningfully.

An alternative approach would be more of a tokenizer, where you loop through all the bytes in the string, looking for escape and space. Here's the code for that, but note that I've made the separator and escape a single byte to simplify it -- I'm guessing they will be in most cases anyway (Go Playground link):

func SplitWithEscaping(s string, separator, escape byte) []string {
    var token []byte
    var tokens []string
    for i := 0; i < len(s); i++ {
        if s[i] == separator {
            tokens = append(tokens, string(token))
            token = token[:0]
        } else if s[i] == escape && i+1 < len(s) {
            i++
            token = append(token, s[i])
        } else {
            token = append(token, s[i])
        }
    }
    tokens = append(tokens, string(token))
    return tokens
}

The other approach that came to mind is regexes, but you have to be able to say "match space, but only if it's not preceded by this escape", which I think you can only do with lookbehind expressions like ?<, and Go's regexp package doesn't support those.

Answer 2

You can omit the string replacements with the following code. It should be quite a bit faster, especially without escapes in s.

func SplitWithEscaping(s string, separator string, escapeString string) []string {
    a := strings.Split(s, separator)

    for i := len(a) - 2; i >= 0; i-- {
        if strings.HasSuffix(a[i], escapeString) {
            a[i] = a[i][:len(a[i])-len(escapeString)] + separator + a[i+1]
            a = append(a[:i+1], a[i+2:]...)
        }
    }
    return a
}