6
\$\begingroup\$

I am adapting the following existing python parser:

def parse_hexchar(c):
    if (c >= '0') and (c <= '9'):
        return ord(c) - ord('0')
    elif (c >= 'a') and (c <= 'f'):
        return ord(c) - ord('a') + 10
    elif (c >= 'A') and (c <= 'F'):
        return ord(c) - ord('A') + 10
    else:
        return -1

def unescape_string (buf):
    dst = ""
    src = 0
    l = len(buf)
    while src < l:
        c = buf[src]
        if c == '\\' and ((src+1) < l):
            c = buf[src+1]
            if c == 'n':
                dst += '\n'
                src += 2
                continue
            elif c == 't':
                dst += '\t'
                src += 2
                continue
            elif c == 'r':
                dst += '\r'
                src += 2
                continue
            elif c == '"':
                dst += '"'
                src += 2
                continue
            elif c == '\n':
                src += 2
                # skip until next non whitespace character
                while src < l:
                    c = buf[src]
                    if c in ' \t':
                        src += 1
                    else:
                        break
                continue
            elif c == 'x' and ((src+3) < l):
                c0 = parse_hexchar(buf[src+2])
                c1 = parse_hexchar(buf[src+3])
                if (c0 >= 0) and (c1 >= 0):
                    dst += chr((c0 << 4) | c1)
                    src += 3
                    continue
        dst += c
        src += 1
    return dst

(its input has already been extracted at an earlier stage that guarantees it doesn't have any unescaped quotes etc, so it is doing less validation than you may naively expect.)

My current translation to APL runs as follows

      I←{⍺[⍵]} ⋄ sub←{⍵⍵(⍺⍺I⍳)@(∊∘⍵⍵)⍵} 
      ht nl cr←⎕ucs 9 10 13 ⋄ wesc←(ht nl cr)sub'tnr'
      segment←{⍵⊆⍨(⌈\(⍳⍴⍵)+(2×1⌽'x'=⍵)(+×⊢)>/∘⌽¨,\'\'=⍵)}
      splice←{⍵/⍨⊃>⍥(⌈\⍳∘⍴×⊢)/((~⍵∊,¨' ',ht) (⍵∊⊂'\',nl))}
      hex←{16⊥¯1+(16↑⎕d,⎕c⎕a)⍳⎕c⍵}
      unescape←{∊⎕ucs∘(hex 2∘↓)¨@(4=≢¨)(¯1↑wesc)¨@(2=≢¨)splice segment ⍵}

Test case:

      unescape '\n,normal escapes: \(\)\"\''\\\\ character escapes: \n,\t,\x22stuff\x22\r,escaped line-\',nl,'   breaks'

,normal escapes: ()"'\\ character escapes: 
,   ,"stuff"                               
,escaped line-break

which works well enough, but feels unwieldy, especially splice.

Suggestions? Comments? Concerns?

\$\endgroup\$
3
  • \$\begingroup\$ What does the example show exactly? \$\endgroup\$ May 21, 2023 at 7:53
  • \$\begingroup\$ Also, if the code in question is an existing parser (by Leonard Ritter) then where is your code? I'm confused. \$\endgroup\$ May 21, 2023 at 7:57
  • \$\begingroup\$ @QuasiStellar the python code, in the first code block, is by Leonard Ritter. The second code block is APL, by me, aiming to replicate the functionality of the python code. Each name←{function code} is akin to a python def name(args): ... It shows, in JSON terms, the string "\\n,normal escapes: \\(\\)\\\"\\'\\\\\\\\ character escapes: \\n,\\t,\\x22stuff\\x22\\r,escaped line-\\\n breaks" being transformed to "\n,normal escapes: ()\"'\\\\ character escapes: \n,\t,\"stuff\"\r,escaped line-breaks" The python is provided as reference for what the second, APL, code block should do. \$\endgroup\$
    – ohAitch
    May 21, 2023 at 11:48

2 Answers 2

5
\$\begingroup\$

Edit: Added Take 2

Here are a couple Hsu-like linear dataflow approach:

Take 1

⎕IO←0  ⍝ ⎕IO delenda est
I←{⍺[⍵]} ⋄ HT LF CR←⎕UCS 9 10 13

s←'\n,normal escapes: \(\)\"\''\\\\ character escapes: \n,\t,\x22stuff\x22\r,escaped line-\',LF,'   breaks'
t←(≢s)⍴0 ⋄ t[⍸m]←1 1 1 1 2 3 0I('tnr"x',LF)⍳s/⍨m←2⊣/0,'\'=s ⍝ Escape types
s[⍸m]←HT LF CR '"'I'tnr"'⍳s/⍨m←1=t ⋄ s t/⍨←⊂1⌽~m            ⍝ Char escapes
s[⍸m]←⎕UCS{16⊥'0123456789ABCDEF'⍳2↑1↓⍵}¨s⊂⍨m←t=2            ⍝ Hex escapes
 s t/⍨←⊂~(1∘⌽∨¯1∘⌽∨¯2∘⌽)m
s←∊{⍵↓⍨⍸<\~HT' '∊⍨⍵}⍥(2↓⊢)¨(1 0,1⌽t=3)⊂'\',LF,s             ⍝ Line cont.

This has the advantage of being very direct. None of the APL here is fancy. Data flows in from the top and proceeds line-by-line through to the end. Spiritually, we're performing parsing passes on the input. Here's another version that might fit better into a larger system, if perhaps the specific logic might change, or there is other processing on strings that you want to do:

Take 2

⎕IO←0  ⍝ ⎕IO delenda est
I←{⍺[⍵]} ⋄ HT LF CR←⎕UCS 9 10 13 ⋄ XD←⎕C'0123456789ABCDEF'

⍝ Input
s←'\n,normal escapes: \(\)\"\''\\\\ character escapes: \n,\t,\x22stuff\x22\r,'
s,←'escaped line-\',LF,'   breaks'

⍝ Escape data
t←m\1 1 1 1 2 3 0I('tnr"x',LF)⍳s/⍨m←2⊣/0,'\'=s   ⍝ Type
x←{2↑1↓⍵}¨s⊂⍨t=2                                 ⍝ Hex code
l←{⍸<\~HT' '∊⍨⍵}⍤(1↓⊢)¨s⊂⍨t=3                    ⍝ Fold length

⍝ Hex escapes
x/⍨←m←{∧/⍵∊XD}¨x ⋄ t[(~m)/⍸t=2]←0

⍝ Stripped escape sequences
s t/⍨←⊂1⌽0=t                                     ⍝ Backslashes
s t/⍨←⊂(¯2∘⌽∧¯1∘⌽)t≠2                            ⍝ Hex digits
s/⍨←t≠3 ⋄ t/⍨←¯1⌽t≠3                             ⍝ Line continuation break

⍝ Reified escapes
s[⍸m]←HT LF CR '"'I'tnr"'⍳s/⍨m←1=t               ⍝ Simple escapes
s[⍸t=2]←⎕UCS{16⊥XD⍳⎕C⍵}¨x                        ⍝ Hex escapes
s←∊(1,l)↓¨(1,t=3)⊂LF,s                           ⍝ Line continuations
\$\endgroup\$
6
  • \$\begingroup\$ Isn't s[⍸m] just (m/s)? \$\endgroup\$
    – Adám
    May 22, 2023 at 6:32
  • \$\begingroup\$ +/16 1× should be 16⊥ \$\endgroup\$
    – Adám
    May 22, 2023 at 6:32
  • \$\begingroup\$ Fixed the to . Cheers. \$\endgroup\$
    – B. Wilson
    May 22, 2023 at 9:12
  • \$\begingroup\$ Regarding s[⍸m], it's mostly an aesthetic choice in this case. However, in the real world stuff I'm playing on at the moment, indexed assignment tends to signal to me that we're updating individual elements of our data structure, whereas selective assignment tends to be a more specific tool used for changing the structure itself. I just went with what felt semantically "right" in this case. \$\endgroup\$
    – B. Wilson
    May 22, 2023 at 9:22
  • \$\begingroup\$ Neither selective nor indexed assignment can change the outer structure or shape. \$\endgroup\$
    – Adám
    May 22, 2023 at 9:39
3
\$\begingroup\$

First of all, your code is quite good in my opinion.

  • I highly recommend adopting a naming convention that allows the reader immediate understanding of the syntactic class of each name. E.g. by making functions have uppercase initials and dyadic operators have underscores at both ends (_sub_).

  • It is recommended that multiple assignments be parenthesised: (ht nl cr)←

  • But strands otherwise don't need parenthesising: ht nl cr _sub_'tnr'

  • Make sure to use instead of when appropriate.

  • splice has an unnecessary parenthesis and reduction (and therefore disclose) on a fixed pair: {⍵/⍨(~⍵∊,¨' ',ht)>⍥(⌈\⍳∘⍴×⊢)(⍵∊⊂'\',nl)}

  • Making hex a train prevents recalculation of the hex digits upon every invocation: 16⊥¯1+(16↑⎕d,⎕c⎕a)⍳⎕

  • I prefer naming function for their result rather than their argument, so I'd rename hex as dec.

  • I'd either make ⎕ucs∘(hex 2∘↓) explicit as {⎕ucs hex 2↓⍵} or at least put ⎕ucs∘ inside the parentheses: (⎕ucs∘hex 2∘↓)

  • That said, you can take advantage of @'s left argument to completely eliminate the inner trains there: 2⎕ucs∘hex⍤↓¨@(4=≢¨)¯1↑∘wesc¨@(2=≢¨)

  • And that means unescape makes a good train: ∊2⎕ucs∘hex⍤↓¨@(4=≢¨)¯1↑∘wesc¨@(2=≢¨)splice∘segment

In summary:

I←{⍺[⍵]} ⋄ _sub_←{⍵⍵(⍺⍺I⍳)@(∊∘⍵⍵)⍵} 
(ht nl cr)←⎕ucs 9 10 13 ⋄ Wesc←ht nl cr _sub_'tnr'
Segment←{⍵⊆⍨(⌈\(⍳≢⍵)+(2×1⌽'x'=⍵)(+×⊢)>/∘⌽¨,\'\'=⍵)}
Splice←{⍵/⍨(~⍵∊,¨' ',ht)>⍥(⌈\⍳∘⍴×⊢)(⍵∊⊂'\',nl)}
Dec←{16⊥¯1+(16↑⎕d,⎕a)⍳⍥⎕c⍵}
Unescape←∊2⎕ucs∘Dec⍤↓¨@(4=≢¨)¯1↑∘Wesc¨@(2=≢¨)Splice∘Segment

Be aware that Dyalog supports many things you can take advantage of, including JSON5 and PCRE:

      Unescape←(⎕JSON⍠'Dialect' 'JSON5')'^\s+'⎕R''
      Unescape '"\n,normal escapes: \(\)\"\''\\\\ character escapes: \n,\t,\x22stuff\x22\r,escaped line-\',nl,'   breaks"'

,normal escapes: ()"'\\ character escapes: 
,   ,"stuff"
,escaped line-breaks
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.