Questions tagged [lxml]

lxml is a full-featured, high performance Python library for processing XML and HTML.

Filter by
Sorted by
Tagged with
8
votes
1answer
2k views

Python CSV to XML converter

I'm creating an application that reads in data from a CSV file and creates an XML file using LXML. The below code works as expected. However, before I develop it further I would like to refactor it ...
7
votes
2answers
556 views

E-mail crawler for yellowpages

I've written a script in python to scrape e-mail addresses from different pizza shops located in los-angeles available in yellowpage traversing multiple pages. It is able to go one-layer deep and dig ...
6
votes
1answer
2k views

LXML parsing for an HTML file

I wrote this code to parse an HTML file which is basically a report generated by a software. This file is 40+ MB in size. The code is 100% perfect. The only problem that I am facing is the parsing ...
6
votes
2answers
941 views

Extracting a div from parsed HTML

It seems that lxml/etree are generally imported as from lxml import etree -- why is that? It keeps the code tidier, and while the potential namespace ambiguity ...
5
votes
1answer
191 views

Richelieu - product scraper

I wanted to see how would I deal with a large amount of data being scraped and written into a CSV file, so I decided to get the info out of a random website. First off, I found a way to search for ...
5
votes
1answer
124 views

Scraping required fields exhausting a full website

I've written some script in python using lxml library which is capable of dealing with a webpage with complicatedly layed-out next page links. My scraper is able to parse all the next page links ...
3
votes
1answer
163 views

Richelieu - product scraper (follow-up)

Intro This is a follow-up question to my earlier post. Although @alecxe had very nice remarks in hist last answer, I decided to get all the products a bit different. Instead of searching for a ...
3
votes
1answer
399 views

Parsing a full site using python

I've created a crawler which is scraping name, phone number and web address of each profile from houzz website. Hope I did it the right way. Here is what I've written: ...
3
votes
1answer
66 views

Nested for-loops in xml parsing

I wrote program that reads and processes xml files submitted by external users. The part of the code in question iterates over xml node's children, appends a dictionary - a future row in a table, and ...
3
votes
1answer
2k views

Finding data on XML using Python's LXML

Using Python's LXML I must read an XML file and print from each "basic" and "expert" tag, the name and email text from it. I've done a script that works but I don't think is the best way of doing this....
3
votes
0answers
37 views

OOP parsing for DB dump XML Python

I am parsing large XML (DB dump). So, the content looks like many different objects with their properties and dependencies (foreign keys). My way to do it - using lxml.etree (iterparse): ...
3
votes
0answers
41 views

Applying a transformation to all the text nodes inside a given node using lxml.html?

I would like to apply a transformation, say uppercase, to all the text nodes inside a given node. I came up with the following code which works fine for my sample ...
2
votes
1answer
31 views

Flattening XML and selecting nodes based on input to convert to CSV

I have some XML that I want to flatten based on its XPath. The task is to acquire certain nodes that are passed as input. That said, I have to get to parent nodes and look for nodes that might be part ...
2
votes
1answer
174 views

Making two classes work ambidextrously

I've written some script in python following OOP guidelines. I've created two classes in my script in such a way so that both of them are able to make requests and produce flawless results. It can ...
2
votes
1answer
226 views

URL List generator

The below code will read a downloaded HTML file. Uses BeautifulSoup and lxml to parse the document and extract all the URLs ...
2
votes
1answer
305 views

Parsing XML documents with LXML

I'm new to LXML, and parsing XML documents in general. I have written a small section of code that appears to do what I need it too do, but it feels like I have overly complicated it. Is there any way ...
1
vote
1answer
73 views

Parsing xml files, ideas to design my code

I begun to work for a company which do not have enough time to practice code review. As a beginner programmer I would like to improve my skills on real working cases. I have been asked to facilitate ...
1
vote
1answer
67 views

Python xml-parsing script

I have a Python code that deals with parsing xml-feeds of footwear online stores. I want to let the code look more professional for the sake of optimal reusability for other programmers. It includes ...
1
vote
1answer
1k views

Parsing scraped data from html table

I've written a simple python web scraper that parses text from an html table and stores the scraped data in List of dictionaries. The code works and doesn't seem to have any glaring issues performance-...