Questions tagged [xpath]

The primary purpose of XPath is to address parts of an XML document. It also provides basic facilities for manipulation of strings, numbers and booleans. XPath uses a compact, non-XML syntax. XPath operates on the abstract, logical structure of an XML document, rather than its surface syntax.

Filter by
Sorted by
Tagged with
-1
votes
0answers
32 views

VB6 XML Shredder speedup [on hold]

I'm working with a legacy VB6 XML shredder that extracts information from proprietary XML files and loads the information into a SQL Server database. It's working, but it's slow. I'm looking for ...
2
votes
1answer
26 views

Beautifulsoup and lxml (xpath) too slow with respect to regex when parsing HTML

I agree that using regex to parse HTML is not a good way, in particular I am worried about their fragility with respect to change in the HTML. The problem is that any alternatives are really too slow....
-2
votes
0answers
14 views

Accessing span.name.actor-name using xpath

I am trying to extract the text between the span tag : <span class=name actor-name>Name1</span>. The code has multiple such tags. Here the intended ...
1
vote
0answers
48 views

Retrieve data from big xml files

I have written a code to retrieve data from xml file. It works but it is really slow. I takes about a minute. This is ok for 1 or some more xml files. But If you have to do it for 1000 it's not a good ...
2
votes
1answer
3k views

Scrapy crawler to parse data recursively

I've written a script in python scrapy to parse "name" and "price" of different products from a website. Firstly, it scrapes the links of different categories from the upper sided bar located in the ...
5
votes
1answer
2k views

Extracting necessary records from LinkedIn

I wanted to create a scraper in python which can fetch required data from LinkedIn. I tried with python in many different ways but I could not make it until I used selenium in combination with. ...
2
votes
1answer
4k views

Scraping content from a javascript enabled website with load more button

The script I've written is able to scrape name, address, phone and web address from a webpage using python and selenium. The main barrier I had to face was to exhaust the load more button to get the ...
2
votes
2answers
197 views

Web-crawler for yellowpage designed using python

I've written some code to scrape name, address, and phone number from yellowpage using python. This scraper has got input parameter. If the input is properly filled in and the filled in url exists in ...
2
votes
1answer
452 views

Scraping data unveiling a button from craigslist

I've written some code to parse the names and phone numbers from craigslist. It starts from the link in "m_url" then goes one layer deep to parse the name and then again another layer deep to parse ...
2
votes
1answer
2k views

Scraping table contents from a webpage using vba with selenium

My script is able to harvest full contents of a table from a webpage with javascript encrypted using vba in combination with selenium. The table has got a drop-down option from where the full contents ...
3
votes
1answer
984 views

Web scraper for parsing names and email addresses from Yellowpage

I've written a script using python to parse the names and email addresses of different pizza shops in USA. I am very new in writing classes using python so I'm not very sure I didn't do anything wrong ...
9
votes
2answers
5k views

Evaluating an XPath with document.evaluate() to get an array of nodes

The Problem Statement: Filter all nodes with an existing attribute that starts with a specific string (temp for example purposes). Print an array of node string ...
5
votes
1answer
163 views

Crawler for scraping and approving friend requests in Facebook

I've made a Facebook scraper using VBA in combination with selenium which is able to parse the name from friend request list. It is necessary to do one thing manually, though. After running this ...
6
votes
1answer
662 views

Scraper for a lazy-loading webpage

I've written a script in VBA in combination with selenium which is able to scrape all the 1000 links from a lazy-loading webpage which displays it's content 20 at a time and doesn't display the full ...
8
votes
1answer
8k views

Scraping the full content from a lazy-loading webpage

I've written a script in python in combination with selenium which is able to scrape 1000 links from a webpage in which lazy-loading method is applied for that reason it displays it's content 20 at a ...
1
vote
1answer
375 views

is it good practice to use relative path in xpath more than once [closed]

I am using Selenium Webdriver-Java to automate a web application. Suppose the html is somewhat like below: ...
2
votes
1answer
2k views

Verify if the element is readonly or disabled

I have written a function where it checks if the field is read-only or disabled. However, first it check if that element field is present This is my working function, but i feel like this is too ...
13
votes
1answer
1k views

Functions to hide and reclaim first visible publication on a page using Selenium

I have the following functions in JavaScript: ...
7
votes
2answers
3k views

SQL query to compare two XML columns

I need to compare two XML columns. I currently have about 1000 rows of data that this is running against and it takes about 20min. Is there anyway I can improve this query to run faster. ...
1
vote
1answer
81 views

Retrieving remote pages and parsing html

This is my code: ...
1
vote
0answers
1k views

Scrape google apps page and store application details in database

Below is a python script which scrapes specific google apps url for example https://play.google.com/store/apps/details?id=com.mojang.minecraftpe and save the ...
2
votes
1answer
3k views

Distinct List of attribute values from XML, using a pre-defined set of attribute names

I have an XML stream that looks like the following (attribute and node quantity , and node depth reduced for demonstration purposes.) ...
1
vote
1answer
905 views

XML editing and merging

I have only started working with XML files and PowerShell. I feel that while there are many ways to accomplish the same thing that I might be doing the following inefficiently. This is all based on ...
8
votes
3answers
251 views

XML splitting query very slow

I have a query to split a delimited string into to multiple columns. The query is extremely slow and takes more than 5 minutes to handle 80444 rows. How can I improve execution time? ...
8
votes
1answer
9k views

Converting partial XML (Node List) to a String

I have a need to extract a part of XML tree (everything under root) and convert it to a string. (The result string will be later pasted inside ...
1
vote
1answer
2k views

Use DOM and XPath to make some changes in HTML document

I intend to make some changes in an HTML document, like remove, replace, append some nodes. I have several arrays with same structure like the following example ...
3
votes
1answer
534 views

Parsing large CSV in Perl

I am getting Out of Memory errors when I try to parse through a large CSV file (2.5Gb). My computer has 32Gb of Memory but Perl uses all of it up. The CSV has 2 ...
3
votes
1answer
4k views

Get InnerHTML, OuterHTML, and plain text of an element by ID or class

I do a fair amount of scraping but I am by no way a good PHP programmer. I always struggle to get the innerhtml of elements using PHP, domdocument and XPath. I have cobbled together a couple of ...
3
votes
3answers
112 views

Merging website info into an array

I'm scraping a title and a link from a website and placing them in a single array. The $i variable seems a bit wrong to me, or is it perfectly fine? ...
1
vote
1answer
732 views

Performance problem when getting text from a context node with VTD-XML

I have a structure like this: ...
3
votes
2answers
846 views

Using Scrapy/Xpath to scrape ESPN for football (soccer) commentaries

The class takes an input of a game-id from the ESPN soccer website. The code then has to grab the commentary, process that and also grab the player names/ids and create a small dictionary of those. I ...
1
vote
1answer
565 views

Handling XPATH input from a user

I want to design API which could handle XPATH input from a user. I currently model the XPATH input in the following way: ...
6
votes
3answers
1k views

Returning a large number of values from a thread

I have an application in which I am querying an XML web service continuously every 2 seconds in a thread. The returned XML is very big and I am retrieving lots of stuff from them using XPath and ...
2
votes
1answer
5k views

Parsing and data presentation in Django

For some time now, I've been using a pattern which I think could be done more elegantly. I have a lot of XML files that need to be represented in a Django views. My current approach is like this: I ...