Questions tagged [selenium]
Selenium is a tools suite to automate web browsers, and web application testing
167
questions
-1
votes
2
answers
50
views
Application install code- Need to remove Nested Loops and make code more compact and readable. Code installs application and its version as per req
This code installs the application, according to the type of application selected like alpha, beta, and released, also it checks for latest version in each type.
...
- 19
2
votes
0
answers
77
views
Python script that refreshes network connection at random intervals
This is a followup to Python script that reboots the router every 600 seconds.
Recently censorship in China has increased because of the meeting of heads of states, and my previous script has been ...
- 2,954
12
votes
4
answers
1k
views
Python script that reboots the router every 600 seconds
This is a Python script I wrote to reboot the router every 600 seconds.
Why did I do it? Well, in case you don't know, I live in China, and I use VPNs to browse the international internet.
My ISP ...
- 2,954
0
votes
1
answer
70
views
Poetry Web Scraping in Python [closed]
I have a script that obtains urls that lead to a specific poem. This code current works and uses multiprocessing pools. I currently am getting restricted or blocked by some way from the website that I ...
- 3
5
votes
1
answer
62
views
Waits for one of two xpaths to be present, one inside of an iframe, one outside of it
I'm using selenium for testing, and I've written code that waits for one of two locations before it continues. This is really easy when both locations are within the same frame, because then I can ...
- 153
1
vote
1
answer
48
views
Python script for get minimum product price having maximum reviews
Technical task.
user visits amazon.com website
user fills out a search field with the product name and activates
search ==>
a page with search results is displayed.
user looks for the product ...
- 21
0
votes
1
answer
106
views
Selenium/Beautifulsoup web scrape of Grainger
This is my first time writing a web scrape using Selenium and Beautifulsoup. The website I'm scraping is https://www.grainger.com/ and I have it pulling a specific set of SKUs stored in an Excel file. ...
0
votes
3
answers
1k
views
To automate scheduling at the Italian Consulate
I'm a beginner developer in Python and I need some tips to improve my code, in performance (speed) and hardware usage. Is it possible to improve?
And I have a question: is it possible to keep the ...
user267072
2
votes
1
answer
48
views
scraping pollution data tables from government website
i made this python program to scrape and save the daily maximum pollution values in mexico city, the data will be used in machine learning but i wonder what could be improved:
...
3
votes
1
answer
163
views
Scrape PokeDex and display in tkinter
Hi I am new here and I just completed my first working version of a pokedex app with a GUI using tkinter. I used selenium to scrape the data from pokemondb.net, and then used pandas to clean up the ...
- 33
0
votes
0
answers
60
views
Unit test for a registration form using C# and Selenium
I would like to know if the below code written in C# Selenium to do Unit Test is efficient and readable. Is there any particular way to do Unit Test. Else suggest me where I can learn to do better ...
- 1
3
votes
1
answer
140
views
Crawler/scraper for soccer match results
I wrote this code some time ago as part of the web-scraping learning. Every now and then I find mistakes in it, as well as I have doubts. Feedback please, is this code compliant with the common best ...
1
vote
1
answer
58
views
Volunteer registration website Selenium testing
I have a volunteer registration website that I wrote some end-to-end tests for in Selenium. Selenium is of course software that lets you automate clicking around a browser, which makes it great for ...
- 1,385
1
vote
0
answers
168
views
Web scraper for pokemon prices using Selenium
I made this scraper to pull the prices of pokemon cards off tcgcollector, using a csv file from the same site. Because for some reason they don't export price. I'm looking for any kind of noobie ...
- 11
1
vote
1
answer
302
views
Selenium web scraper to scrape Apple tweets of the last 3 years
I want to use my Web Scraper to get all tweets about Apple up to a date I specify. At the moment I'm scraping the tweets of today, or the last few days. However, my goal is to scrape all tweets from ...
- 11
3
votes
0
answers
79
views
Scraping housing dataset row by row from Estate Agent's website
I have written web scraping code with selenium with python3 for a dynamic site (it was my first project that is why I started with selenium). In my code I extracted the values and created DataFrame.
...
- 31
4
votes
2
answers
258
views
Scraping campsite availability from a webpage using vba with selenium
I wrote some code to extract the information from a table, but it takes an extremely long time.
The table is in the format of a calendar.
I need the information on an Excel sheet with column 1 as the ...
- 41
1
vote
1
answer
265
views
Web-scrape bonds information from nasdaqomxnordic
I'm a newbie getting into web scrapers. I've made something that works, it takes 3.2 hours to complete job and randomly have 10 lines blank each time I run this job. Help is much appreciated!
...
- 13
4
votes
2
answers
527
views
Web scraping data.cdc.gov for COVID-19 Data with Selenium in Python
I'm attempting to scrape data.cdc.gov for their COVID-19 information on cases and deaths.
The problem that I'm having is that the code seems to be very inefficient. It takes an extremely long time for ...
- 55
1
vote
1
answer
89
views
Web Scraping DOL with Selenium
This is my code to extract data from the DOL website for a project I'm doing for my portfolio. The code does extract all of the data that I need, however it runs really slowly. I think it took like 15 ...
- 55
2
votes
0
answers
407
views
My first vba-selenium code
I have been obtaining financial information on an annuity I have which is invested in multiple funds which are proprietary to the insurance company.
The asset value is not available other than on ...
- 121
1
vote
2
answers
100
views
WaitFor method for website automation testing with Selenium WebDriver
I am trying to use Selenium for website automation testing tasks and I am new to Selenium testing framework. The the situation I faced is to wait the website components loading and then do the related ...
- 3,918
4
votes
1
answer
898
views
Scraping a dynamic website with Scrapy (or Requests) and Selenium
I am trying to use Scrapy for one of the sites I've scraped before using Selenium over here.
Because the search field for this site is dynamically generated and requires the user to hover the cursor ...
- 417
2
votes
1
answer
635
views
Extract flight arrival data from web
I'm trying to scrape arrival data from Flight Radar 24
My script takes extremely long time to scrape the data. Is there any way I can speed up the scraping process?
Here's my script:
...
- 51
1
vote
1
answer
43
views
Simple login test, translated from java to python
I'm reading on how to write proper testing suites here. So I'm trying to follow the selenium example in the docs which is in Java; I'm trying to translate it to Python since my app is written in ...
- 241
1
vote
0
answers
20
views
Organizing things together to form a minimum viable Scraper App (part 2)
This is a follow-up of my question over here.
Response to @Reinderien's answer:
I have corrected the more trivial issues highlighted in @Reinderien's answer below as follows. ...
- 417
1
vote
0
answers
76
views
Optimising this web-scraping code
A member of SO who I immensely respect just told me that the code below makes him uncomfortable.
...
- 179
2
votes
0
answers
2k
views
Python script to scrape google maps without API
I wrote a python script to scrape google maps for my app. I really want my code to be readable and have tried to follow PEP-8 wherever I could, so I have come to you all for guidance.
It uses selenium ...
- 21
2
votes
1
answer
385
views
Looping through search results in Selenium
This is a follow-up of my question over here.
I am extending the functionality of @Reinderien's suggested code to enable the automatic looping through of pages in the search result.
I've mainly ...
- 417
0
votes
1
answer
165
views
Unix batch job execution from BDD automation framework
As part of my testing, I have to connect to Unix server using putty and execute few unix jobs.
Feature file
Steps for test scenario:
...
- 89
3
votes
1
answer
267
views
Looping through keywords and multiple pages in Selenium
This is a follow-up of my question over here.
I adopted @Reinderien's suggested script to a second website below:
fudan.py
...
- 417
1
vote
1
answer
90
views
How to make use of a class of descriptors to save code and other issues
This is a follow-up to my question over here.
Having worked through most of the issues highlighted by @Reinderien in his detailed answer, I somehow got the code to work...
Some quick observations:
<...
- 417
3
votes
1
answer
131
views
Getting xpath indices to count forward to preserve table structure
Scraping tables from the web get complicated when there are 2 or more values in a cell. In order to preserve the table structure, I have devised a way to count the row-number index of its xpath, ...
- 417
-2
votes
1
answer
102
views
Can I refactor my code anymore? [closed]
I have written one method; however I feel I can refactor it even more but not sure what
Can someone help me here?
...
- 11
1
vote
0
answers
245
views
A thread-safe WebDriver pool to submit browsing jobs to
For a project, I need to make use of Selenium WebDrivers, and since they're so expensive to start, I decided to write a pool to manage them.
...
- 16.2k
2
votes
2
answers
248
views
Scrape and parse dictionary website and get meaning of each word in a text file
In this program, I have done automation using Selenium, to find english meaning of each sanskrit word from a dictionary website.
The function eng_meaning accepts ...
- 2,449
1
vote
0
answers
263
views
Selenium Webscraper for Odd data analyses
I have written the following code to achieve the following:
Regularly scrape all live matches on the betting site oddsportal.com
Pull odds data into data frame
Evaluate data frame for two odd ...
- 11
2
votes
0
answers
58
views
Web Scraping data from website using Selenium in python
ApplicationDetail.txt:
URL: https://www.abc.com
applicationNo : 123456
Application Class:
...
- 21
2
votes
0
answers
547
views
Discord Bot Python. Selenium screenshots
I wrote my own Discord Bot which is taking screenshot of a specific website. This scrip is very simple and thought it should work fast, but it isn't. I read a lot, and I think I'm not able to improve ...
- 21
2
votes
1
answer
2k
views
How to make selenium in python faster
I'm making a bot for PlayStation 5.
And restock is coming soon so I want to make my bot faster.
The approximate duration is 4.90 seconds.
Is there any way to make this faster?
I found that I should ...
4
votes
0
answers
86
views
Python Selenium First Project - loop through pages and check/uncheck tick boxes
New to Python/Selenium and learning on the fly with my first basic selenium script.
Request user input - activate or deactivate.
Open's website, loops through shop URL as well as activate/deactivate ...
- 41
0
votes
1
answer
83
views
Reading configuration variables in a Python project
I am new to python, in .NET we use web.config to store environment specific variables... I want to achieve the same for my python program (scraper)
I have created .env file at the root of the project ...
- 573
5
votes
1
answer
242
views
Automating TypeRacer using selenium
This is a personal project which I created solely to torture my friends ;)
I'm pretty sure you've heard about typeracer.
And if you have friends like mine who are faster than you, you'd surely be ...
- 2,720
0
votes
2
answers
159
views
open multiple tabs using selenium
I am using the following script to read search terms from a file and then search those terms in google (each search result in it's own tab).
...
- 461
2
votes
0
answers
84
views
C# Scraper - EventHandler
It might help to check my previous question out, its part of the same application. The only minor changes are with the entity class names. They now each have their own entities, and I use interfaces ...
- 301
7
votes
1
answer
198
views
Booking an East London Tennis Court
Description
I'm not sure if it Covid-19 but lately it is impossible to book a tennis court in my area on time. It's always full or maybe just don't check enough :)
To beat the queue and get notified ...
- 11.5k
5
votes
1
answer
411
views
Web Scraping using Selenium and Python
This is my first program code ever, and it actually works.
My goal is to scrape information from the website and store it in a database.
It is a site that has historical data on sporting events and ...
user230470
5
votes
1
answer
1k
views
Game scraper for Steam
I made a scraper for Steam that get different info about a Steam game, such as the price, the specs and the supported platforms. The reason I made this was because I have a super slow laptop, so ...
- 153
3
votes
1
answer
128
views
Project : My Anime Finder
This is a project that finds a specific anime (show) and checks to see if there is a new episode of it .
this is also the first time that i used def like this and ...
- 369
6
votes
2
answers
408
views
Web scraper for driver's license test times
I have created a small selenium script that checks for available times to write a test for a drivers license. The program runs every minute and takes approx 50 seconds to run. I have noticed that it's ...
- 229