Questions tagged [lxml]

lxml is a full-featured, high performance Python library for processing XML and HTML.

Filter by
Sorted by
Tagged with
4
votes
1answer
187 views

Python lxml transform xml to dictionary

I need to download a fairly large xml (up to 2 GB) then load it into memory or possibly an empty db on a stand alone machine on a semi-regular basis. I need to grab a large amount of very specific ...
2
votes
1answer
102 views

How to divide this Python code into different functions?

I'm making a code that, starting from an xml file: stores the index of child elements of a tag and the child elements as key, values in a dictionary; deletes keys whose values contain a certain ...
1
vote
1answer
114 views

Parsing xml files, ideas to design my code

I begun to work for a company which do not have enough time to practice code review. As a beginner programmer I would like to improve my skills on real working cases. I have been asked to facilitate ...
3
votes
0answers
54 views

OOP parsing for DB dump XML Python

I am parsing large XML (DB dump). So, the content looks like many different objects with their properties and dependencies (foreign keys). My way to do it - using lxml.etree (iterparse): ...
3
votes
1answer
549 views

Nested for-loops in xml parsing

I wrote program that reads and processes xml files submitted by external users. The part of the code in question iterates over xml node's children, appends a dictionary - a future row in a table, and ...
2
votes
1answer
38 views

Flattening XML and selecting nodes based on input to convert to CSV

I have some XML that I want to flatten based on its XPath. The task is to acquire certain nodes that are passed as input. That said, I have to get to parent nodes and look for nodes that might be part ...
1
vote
1answer
87 views

Python xml-parsing script

I have a Python code that deals with parsing xml-feeds of footwear online stores. I want to let the code look more professional for the sake of optimal reusability for other programmers. It includes ...
3
votes
0answers
42 views

Applying a transformation to all the text nodes inside a given node using lxml.html?

I would like to apply a transformation, say uppercase, to all the text nodes inside a given node. I came up with the following code which works fine for my sample ...
8
votes
1answer
4k views

Python CSV to XML converter

I'm creating an application that reads in data from a CSV file and creates an XML file using LXML. The below code works as expected. However, before I develop it further I would like to refactor it ...
2
votes
1answer
2k views

Parsing scraped data from html table

I've written a simple python web scraper that parses text from an html table and stores the scraped data in List of dictionaries. The code works and doesn't seem to have any glaring issues performance-...
2
votes
1answer
327 views

Parsing XML documents with LXML

I'm new to LXML, and parsing XML documents in general. I have written a small section of code that appears to do what I need it too do, but it feels like I have overly complicated it. Is there any way ...
3
votes
1answer
3k views

Finding data on XML using Python's LXML

Using Python's LXML I must read an XML file and print from each "basic" and "expert" tag, the name and email text from it. I've done a script that works but I don't think is the best way of doing this....
4
votes
1answer
200 views

Richelieu - product scraper (follow-up)

Intro This is a follow-up question to my earlier post. Although @alecxe had very nice remarks in hist last answer, I decided to get all the products a bit different. Instead of searching for a space (...
5
votes
1answer
198 views

Richelieu - product scraper

I wanted to see how would I deal with a large amount of data being scraped and written into a CSV file, so I decided to get the info out of a random website. First off, I found a way to search for ...
7
votes
2answers
723 views

E-mail crawler for yellowpages

I've written a script in python to scrape e-mail addresses from different pizza shops located in los-angeles available in yellowpage traversing multiple pages. It is able to go one-layer deep and dig ...
2
votes
1answer
251 views

URL List generator

The below code will read a downloaded HTML file. Uses BeautifulSoup and lxml to parse the document and extract all the URLs ...
5
votes
1answer
128 views

Scraping required fields exhausting a full website

I've written some script in python using lxml library which is capable of dealing with a webpage with complicatedly layed-out next page links. My scraper is able to parse all the next page links ...
2
votes
1answer
175 views

Making two classes work ambidextrously

I've written some script in python following OOP guidelines. I've created two classes in my script in such a way so that both of them are able to make requests and produce flawless results. It can ...
6
votes
1answer
2k views

LXML parsing for an HTML file

I wrote this code to parse an HTML file which is basically a report generated by a software. This file is 40+ MB in size. The code is 100% perfect. The only problem that I am facing is the parsing ...
3
votes
1answer
444 views

Parsing a full site using python

I've created a crawler which is scraping name, phone number and web address of each profile from houzz website. Hope I did it the right way. Here is what I've written: ...
6
votes
2answers
1k views

Extracting a div from parsed HTML

It seems that lxml/etree are generally imported as from lxml import etree -- why is that? It keeps the code tidier, and while the potential namespace ambiguity ...