3
\$\begingroup\$

I am parsing large XML (DB dump). So, the content looks like many different objects with their properties and dependencies (foreign keys). My way to do it - using lxml.etree (iterparse):

    def insert_db(content):
        for item in xml_parser(content):
            if isinstance(item, DPoint):
                # manipulations for inserting DPoint object
                pass
            elif isinstance(item, ApotHpot):
                # manipulations for inserting ApotHpot object
                pass
                ...


    def xml_parser(xmlfile):
        context = ET.iterparse(xmlfile, events=('start', 'end'))
        context = iter(context)
        event, root = next(context)

        d_pt, a_pt, n_pt = None, None, None
        for event, elem in context:
            if event == 'start':
                if elem.tag == 'ApotHpot':
                    a_pt = Object(None, None)
                    a_pt.id = elem.attrib.get('id')
                    n_pt = Object(None, None)
                    designated_pt = Object(None, None)
                    # d_pt = DPoint(None, None, None, None, ...)
                    # n_pt = NDB(None, None, None, None, None, ...)
                elif elem.tag == 'DPoint':
                    d_pt = Object(None, None)
                    d_pt.id = elem.attrib.get('id')
                    n_pt = Object(None, None)
                    a_pt = Object(None, None)
            else:
                if elem.tag == 'identifier':
                    if d_pt.id:
                        d_pt.uuid = elem.text
                    elif a_pt.id:
                        airport.uuid = elem.text
                    elif n_pt.id:
                        n_pt.uuid = elem.text
            ...
            # and so on for the properties
            # yield an object when closing tag:
                    elif elem.tag == 'DPoint':
                        yield d_pt
                    elif elem.tag == 'ApotHpot':
                        yield a_pt
                    ...
                    root.clear()

The question is that I have a doubt about the way to create class:

There are variants:

1. Define only one (common) class (as class Object in code above with only 2 arguments (all of the objects have these properties) 'uuid' and 'id'), when properties appeared (tag in XML doc), they will add dynamically to the class instance

a)

class Object:
    def __init__(self, uuid, id):
        self.uuid = uuid
        self.id = id

then call some_variable = Object(None, None) or

b)

class Object:
    def __init__(self):
        self.uuid = None
        self.id = None

then call some_variable = Object()

2. Define class for every object in DB Dump with all predefined attributes, e.g.

a)

class NDB:
    def __init__(self, uuid, id, srid, coords, interpretation, name...):
        self.uuid = uuid
        self.id = id
        self.srid = srid
        self.coords = coords
        self.interpretation = interpretation
        self.name = name
        ...

call some_variable = NDB(None, None, None, None, None, ...)

b)

class NDB:
    def __init__(self):
        self.uuid = None
        self.id = None
        self.srid = None
        self.coords = None
        self.interpretation = None
        self.name = None
        ...

call some_variable = NDB()

and so on for all of the objects.

Firstly I used the 2a) style, it seemed to me that it is the right approach to describe all of the objects. But the more I deepen to the XML doc the more objects I have to implement to its own class - when parsed all the doc I get 23 instances to be described in its own class with its own attributes. Therefore instead of this I wondered that the common class with dynamically attributes (or all attributes -> None as 2b)) is suitable for my case.

I wonder if somebody help me to resolve my doubts\give some links\recommend the best practice!

ADD

Piece of a raw data is here

As a result I would like to get the objects (refer to the data: AportHport, VR, DE, Navd, Rway, RwayDirection, RwayCentrelinePoint, Glide, Loc) with all attributes (some attributes have the same name for the objects) from XML for def insert_db(). Set of attributes varies (some of them were null from dumped DB) so when inserting in DB I obviously get class.attribute for fill my DB table. But I have to check if object hasattr(cls, attribute) or if it has not (in case I choose the 2b))

\$\endgroup\$
  • 1
    \$\begingroup\$ Provide a testable XML fragment and respective expected result for that \$\endgroup\$ – RomanPerekhrest Oct 17 at 13:58
  • 1
    \$\begingroup\$ @RomanPerekhrest, added link and expected result!) \$\endgroup\$ – Jane Oct 17 at 15:28
  • 1
    \$\begingroup\$ Link-only content generally doesn't work well for StackExchange sites. Please copy and paste a representative section of your XML into your question. Also, as an aside: single-file content like that is better-suited to gist than a github repository. \$\endgroup\$ – Reinderien Oct 23 at 2:05
  • \$\begingroup\$ @Reinderien, Wow, thank you! I'll keep the gist in mind ) \$\endgroup\$ – Jane Oct 23 at 9:50
  • \$\begingroup\$ Still waiting on that XML snippet. Otherwise: you've elided so much code that it makes it difficult to do a review. Please include all of your source. \$\endgroup\$ – Reinderien Oct 24 at 1:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.