3
\$\begingroup\$

I'm learning Python, and found a fun little video on YouTube called "Learn Python through public data hacking". In essence it uses the CTA unofficial API to do some parsing of XML data. In taking things further than inline code, I'm chunking behaviors into modules and classes to further understand how Python works.

The XML from the API looks something like this:

<buses rt="22">
<time>1:40 AM</time>
    <bus>
        <id>4194</id>
        <rt>22</rt>
        <d>East Bound</d>
        <dd>Northbound</dd>
        <dn>E</dn>
        <lat>41.88327006970422</lat>
        <lon>-87.62828115689553</lon>
        <pid>5421</pid>
        <pd>Northbound</pd>
        <run>P238</run>
        <fs>Howard</fs>
        <op>49875</op>
        <dip>5314</dip>
        <bid>7323287</bid>
        <wid1>0P</wid1>
        <wid2>238</wid2>                
    </bus>
    <!-- bus... n -->
</buses>

I'm using the python-requests module for the HTTP side, and once the XML is downloaded, I parse each <bus> node into a Bus class, assign each of the child nodes into a dictionary, and make that visible with a get(prop) statement, so I can just call bus.get('lat') to retrieve the latitude, etc. However, in order to do the correct calculations on it (i.e. arithmetic) each node's value needs to be returned as the correct type. By default, they're all read as strings.

Considering that Python doesn't have a "switch" statement like most other languages, someone at SO said to use a dictionary. Is this the/a correct way of doing something like this? Or is there some nifty builtin that I don't know of?

def dyncast(value):
    _type = type(value)
    types = {
        "FloatType": (r'^[\d]{2}\.[\d]+$', lambda f : float(f)),
        "IntType" : (r'^[\d]+$', lambda i : int(i)),
        "StrType" : (r'^[a-zA-z]+$', lambda s : str(s))
    }
    for typeval in types:
        pattern = types[typeval][0]
        fn = types[typeval][1]
        match = re.match(pattern, value)

        # if it matches a regex and has a group(), return the
        # lambda calculation (typecast)
        if match and match.group():
            return fn(value)

    # return straight up if no matches
    return value

# Called via:
for elm in node:
    self._data[elm.tag] = dyncast(elm.text)
# where node is the bus node, and elm are the child nodes

It seems to work well, as I get returns such as:

# print bus.get('fs'), type(bus.get('fs'))
Harrison <type 'str'>

# print bus.get('lat'), type(bus.get('lat'))
41.9027030677 <type 'float'>
\$\endgroup\$
4
\$\begingroup\$

Makes sense to me. A couple small things:

lambda f: float(f) should be equivalent to just float if I'm not mistaken. You can simplify the loop a bit too since you're not actually using the dictionary keys.

types = [
  (r'(regex)', float),
  (r'(regex)', int),
  (r'(regex)', str),
]
for pattern, fn in types:
  match = ...
\$\endgroup\$
  • \$\begingroup\$ "lambda f: float(f) should be equivalent to just float" ⟶ It's known as η-conversion. \$\endgroup\$ – Gareth Rees Nov 21 '13 at 18:19
2
\$\begingroup\$
  • It is not necessary to match StrType as your fallback is anyway to return the original string.
  • The float regex does not accept a sign, as in <lon>-87.62828115689553</lon>, or exponential notation.
  • You can use just if match: instead of if match and match.group(): as the regexes do not match empty strings.
  • MatrixFrog's approach for streamlining the code is good.
\$\endgroup\$
  • \$\begingroup\$ Thanks. Yea I'll go in and modify the regex to match all possibilities. This was just a milestone point to and wanted to make sure it wasn't overkill. \$\endgroup\$ – Phix Nov 22 '13 at 18:15
1
\$\begingroup\$

There is no need to redefine types each time the function is called. Also the use of the name types isn't declarative enough - the reader can't gain much info from the name.

The name dyncast I guess stands for dynamic cast? be more declarative in the function name. E.g, unserialise.

There may be gains to be made by compiling the regex once.

The code defines _type, but don't use it.

The code iterates over a dict in an inefficient way: instead its better to use the .iteritems() method instead of iterating the keys and then getting the value from the dict. However, saying that - you don't need a dict at all - this can be achieved with a humble tuple. Its a good habit to use better performing data types.

match evaluates to truthy or falsey so there is no need to do if match and match.groups...

The regex patterns can miss values that should be matches - e.g, float of 1.1 or 123.4 would not be matched. Moreover, you don't need to match for string because dyncast returns the unchanged value if not matched (which is a string).

So given all the above, instead consider

PATTERN_TO_TYPE = (
    (compile(r'^[\d]+\.[\d]+$'), float),
    (compile(r'^[\d]+$'), int),
}

def unserialise(value):
    for regex, fn in PATTERN_TO_TYPE:
        if regex.match(value):
            return fn(value)
    return value
\$\endgroup\$
  • \$\begingroup\$ Thanks for the cleanup! As for the match statement I was getting errors about None Type, the and check removed. I'll revisit. Some of the vars were leftover from debug. \$\endgroup\$ – Phix Nov 22 '13 at 18:17
  • \$\begingroup\$ for debugging - use pdb - its an interactive debugger where you set a breakpoint at the line you are interested in with from pdb import set_trace; set_trace(). then when you run your script you will be given an interactive shell of the script "paused" at that line so you can introspect. Aso, post the error in a comment here, maybe I can help \$\endgroup\$ – rikAtee Nov 24 '13 at 13:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.