String unescaping in Dyalog APL

Question

I am adapting the following existing python parser:

def parse_hexchar(c):
    if (c >= '0') and (c <= '9'):
        return ord(c) - ord('0')
    elif (c >= 'a') and (c <= 'f'):
        return ord(c) - ord('a') + 10
    elif (c >= 'A') and (c <= 'F'):
        return ord(c) - ord('A') + 10
    else:
        return -1

def unescape_string (buf):
    dst = ""
    src = 0
    l = len(buf)
    while src < l:
        c = buf[src]
        if c == '\\' and ((src+1) < l):
            c = buf[src+1]
            if c == 'n':
                dst += '\n'
                src += 2
                continue
            elif c == 't':
                dst += '\t'
                src += 2
                continue
            elif c == 'r':
                dst += '\r'
                src += 2
                continue
            elif c == '"':
                dst += '"'
                src += 2
                continue
            elif c == '\n':
                src += 2
                # skip until next non whitespace character
                while src < l:
                    c = buf[src]
                    if c in ' \t':
                        src += 1
                    else:
                        break
                continue
            elif c == 'x' and ((src+3) < l):
                c0 = parse_hexchar(buf[src+2])
                c1 = parse_hexchar(buf[src+3])
                if (c0 >= 0) and (c1 >= 0):
                    dst += chr((c0 << 4) | c1)
                    src += 3
                    continue
        dst += c
        src += 1
    return dst

(its input has already been extracted at an earlier stage that guarantees it doesn't have any unescaped quotes etc, so it is doing less validation than you may naively expect.)

My current translation to APL runs as follows

      I←{⍺[⍵]} ⋄ sub←{⍵⍵(⍺⍺I⍳)@(∊∘⍵⍵)⍵} 
      ht nl cr←⎕ucs 9 10 13 ⋄ wesc←(ht nl cr)sub'tnr'
      segment←{⍵⊆⍨(⌈\(⍳⍴⍵)+(2×1⌽'x'=⍵)(+×⊢)>/∘⌽¨,\'\'=⍵)}
      splice←{⍵/⍨⊃>⍥(⌈\⍳∘⍴×⊢)/((~⍵∊,¨' ',ht) (⍵∊⊂'\',nl))}
      hex←{16⊥¯1+(16↑⎕d,⎕c⎕a)⍳⎕c⍵}
      unescape←{∊⎕ucs∘(hex 2∘↓)¨@(4=≢¨)(¯1↑wesc)¨@(2=≢¨)splice segment ⍵}

Test case:

      unescape '\n,normal escapes: \(\)\"\''\\\\ character escapes: \n,\t,\x22stuff\x22\r,escaped line-\',nl,'   breaks'

,normal escapes: ()"'\\ character escapes: 
,   ,"stuff"                               
,escaped line-break

which works well enough, but feels unwieldy, especially splice.

Suggestions? Comments? Concerns?

Answer 1

Edit: Added Take 2

Here are a couple Hsu-like linear dataflow approach:

Take 1

⎕IO←0  ⍝ ⎕IO delenda est
I←{⍺[⍵]} ⋄ HT LF CR←⎕UCS 9 10 13

s←'\n,normal escapes: \(\)\"\''\\\\ character escapes: \n,\t,\x22stuff\x22\r,escaped line-\',LF,'   breaks'
t←(≢s)⍴0 ⋄ t[⍸m]←1 1 1 1 2 3 0I('tnr"x',LF)⍳s/⍨m←2⊣/0,'\'=s ⍝ Escape types
s[⍸m]←HT LF CR '"'I'tnr"'⍳s/⍨m←1=t ⋄ s t/⍨←⊂1⌽~m            ⍝ Char escapes
s[⍸m]←⎕UCS{16⊥'0123456789ABCDEF'⍳2↑1↓⍵}¨s⊂⍨m←t=2            ⍝ Hex escapes
 s t/⍨←⊂~(1∘⌽∨¯1∘⌽∨¯2∘⌽)m
s←∊{⍵↓⍨⍸<\~HT' '∊⍨⍵}⍥(2↓⊢)¨(1 0,1⌽t=3)⊂'\',LF,s             ⍝ Line cont.

This has the advantage of being very direct. None of the APL here is fancy. Data flows in from the top and proceeds line-by-line through to the end. Spiritually, we're performing parsing passes on the input. Here's another version that might fit better into a larger system, if perhaps the specific logic might change, or there is other processing on strings that you want to do:

Take 2

⎕IO←0  ⍝ ⎕IO delenda est
I←{⍺[⍵]} ⋄ HT LF CR←⎕UCS 9 10 13 ⋄ XD←⎕C'0123456789ABCDEF'

⍝ Input
s←'\n,normal escapes: \(\)\"\''\\\\ character escapes: \n,\t,\x22stuff\x22\r,'
s,←'escaped line-\',LF,'   breaks'

⍝ Escape data
t←m\1 1 1 1 2 3 0I('tnr"x',LF)⍳s/⍨m←2⊣/0,'\'=s   ⍝ Type
x←{2↑1↓⍵}¨s⊂⍨t=2                                 ⍝ Hex code
l←{⍸<\~HT' '∊⍨⍵}⍤(1↓⊢)¨s⊂⍨t=3                    ⍝ Fold length

⍝ Hex escapes
x/⍨←m←{∧/⍵∊XD}¨x ⋄ t[(~m)/⍸t=2]←0

⍝ Stripped escape sequences
s t/⍨←⊂1⌽0=t                                     ⍝ Backslashes
s t/⍨←⊂(¯2∘⌽∧¯1∘⌽)t≠2                            ⍝ Hex digits
s/⍨←t≠3 ⋄ t/⍨←¯1⌽t≠3                             ⍝ Line continuation break

⍝ Reified escapes
s[⍸m]←HT LF CR '"'I'tnr"'⍳s/⍨m←1=t               ⍝ Simple escapes
s[⍸t=2]←⎕UCS{16⊥XD⍳⎕C⍵}¨x                        ⍝ Hex escapes
s←∊(1,l)↓¨(1,t=3)⊂LF,s                           ⍝ Line continuations

Answer 2

First of all, your code is quite good in my opinion.

• I highly recommend adopting a naming convention that allows the reader immediate understanding of the syntactic class of each name. E.g. by making functions have uppercase initials and dyadic operators have underscores at both ends (_sub_).

• It is recommended that multiple assignments be parenthesised: (ht nl cr)←

• But strands otherwise don't need parenthesising: ht nl cr _sub_'tnr'

• Make sure to use ≢ instead of ⍴ when appropriate.

• splice has an unnecessary parenthesis and reduction (and therefore disclose) on a fixed pair: {⍵/⍨(~⍵∊,¨' ',ht)>⍥(⌈\⍳∘⍴×⊢)(⍵∊⊂'\',nl)}

• Making hex a train prevents recalculation of the hex digits upon every invocation: 16⊥¯1+(16↑⎕d,⎕c⎕a)⍳⎕

• I prefer naming function for their result rather than their argument, so I'd rename hex as dec.

• I'd either make ⎕ucs∘(hex 2∘↓) explicit as {⎕ucs hex 2↓⍵} or at least put ⎕ucs∘ inside the parentheses: (⎕ucs∘hex 2∘↓)

• That said, you can take advantage of @'s left argument to completely eliminate the inner trains there: 2⎕ucs∘hex⍤↓¨@(4=≢¨)¯1↑∘wesc¨@(2=≢¨)

• And that means unescape makes a good train: ∊2⎕ucs∘hex⍤↓¨@(4=≢¨)¯1↑∘wesc¨@(2=≢¨)splice∘segment

In summary:

I←{⍺[⍵]} ⋄ _sub_←{⍵⍵(⍺⍺I⍳)@(∊∘⍵⍵)⍵} 
(ht nl cr)←⎕ucs 9 10 13 ⋄ Wesc←ht nl cr _sub_'tnr'
Segment←{⍵⊆⍨(⌈\(⍳≢⍵)+(2×1⌽'x'=⍵)(+×⊢)>/∘⌽¨,\'\'=⍵)}
Splice←{⍵/⍨(~⍵∊,¨' ',ht)>⍥(⌈\⍳∘⍴×⊢)(⍵∊⊂'\',nl)}
Dec←{16⊥¯1+(16↑⎕d,⎕a)⍳⍥⎕c⍵}
Unescape←∊2⎕ucs∘Dec⍤↓¨@(4=≢¨)¯1↑∘Wesc¨@(2=≢¨)Splice∘Segment

---

Be aware that Dyalog supports many things you can take advantage of, including JSON5 and PCRE:

      Unescape←(⎕JSON⍠'Dialect' 'JSON5')'^\s+'⎕R''
      Unescape '"\n,normal escapes: \(\)\"\''\\\\ character escapes: \n,\t,\x22stuff\x22\r,escaped line-\',nl,'   breaks"'

,normal escapes: ()"'\\ character escapes: 
,   ,"stuff"
,escaped line-breaks