Questions tagged [selenium]

Selenium is a tools suite to automate web browsers, and web application testing

Filter by
Sorted by
Tagged with
5
votes
1answer
2k views

Scraping all closed captions (subtitles) of a YouTube's creator's video library

A script that scrapes all YouTube's videos closed captions(subtitles) of a single creator's video library. It scrapes either auto-generated or manually written subtitles. The script accepts a link to ...
4
votes
1answer
48 views

Python script for parsing links

I wrote a test that parses links from a web page, and in a loop, clicks on each of them, and then returns to the main page. But, each iteration is accompanied by parsing the entire web page again and ...
1
vote
1answer
41 views

Selenium - Java data driven Framework suggestion

Dropdown Screenshot I have 3 dropdowns with multiple components in the UI. Have incorporated data driven java selenium framework, data is fetched from excel sheet: Excel sheet- where data is fetched ...
3
votes
1answer
132 views

Initializing a controller that uses Selenium

I've outsourced the setup functions from my class to another file, I outsourced initDriver from the class PortalController to ...
2
votes
1answer
74 views

Submit random choices on Google form

I've written a Python script that randomly selects radio buttons for each question, and submits the google form. My main questions for improvement are: Algorithm: I collect all the buttons into a 2D ...
3
votes
1answer
72 views

Get result from form website operation

I want to get a result from the website when I submit a form. There are several things that can go wrong. In a simple scenario I want to return: true - when ...
2
votes
0answers
509 views

Grabbing information traversing multiple pages

I've written a script in python in combination with selenium to parse different information from a webpage and store the collected data in a csv file. The data are rightly coming through. The email ...
3
votes
1answer
4k views

Extracting all the data from a table out of a javascript encrypted website

I've written a script to harvest all the data out of a table from a webpage using python in combination with selenium. It takes a while to parse them all. There are seven steps to hurdle to get to the ...
5
votes
1answer
2k views

Selecting filter checkboxes using Selenium

I am using Selenium Webdriver via Java. My method clicks on a filter check box (eg. make, model, etc.), that is passed in, from a modal. When running my method in my test class, via ChromeDriver, it ...
3
votes
1answer
522 views

Python script to download adult comics from 8muses

I made a simple python script using the BeautifulSoup and Selenium to automatically download adult comics from 8muses. I've used selenium for the reason that the website uses javascript to load the ...
0
votes
0answers
24 views

Select a value other then currently selected drop down value

Scenario:Select a any other value other then currently selected from a drop down. Please help/review the code and tell me whether this is a efficient solution? ...
1
vote
0answers
43 views

Behat Page Object Model

I have a working solution on a training exercise that I wish to improve as I feel there is a better way to utilize the form and get to the fieldsets etc using page model. HTML: ...
2
votes
0answers
241 views

Python, Selenium, Pandas project; for populating web form page on loop

I have inherited this nightmare Python, Selenium, and Pandas project. I am learning Python as I go; and have been cleaning up and refactoring the code where I see necessary - It was worst if you can ...
1
vote
1answer
40 views

Select depart date, 5 days from today and return date, 6 days from today in Selenium

I am automating a scenario of flight booking on Cleartrip.com(https://www.cleartrip.com/flights) in which I have to enter depart date, 5 days from today and return date, 6 days from today. I have ...
4
votes
3answers
1k views

Fill out webform with Selenium

I am automating a webform on a webpage. I am still learning Selenium on the road. I know a few bad/good practices but a code review is always a good thing to do. What should I improve on in my code? ...
1
vote
0answers
84 views

Automating A Workflow Using Selenium - Best Way To Fetch Elements & Make Code More Generic

I'm a newbie to Selenium and I'm trying to get better at it by taking up interview assignments. This YouTube link describes the workflow that needs to be automated. The following points are worth ...
4
votes
1answer
90 views

Uploading songs from website to database and then to Spotify

My Python program parses songs from a website, corrects song titles and artists with the Last.fm API, searches for the spotify uri using the Spotify API, stores all the information in a SQLite ...
0
votes
1answer
68 views

Factory to instantiate and configure Selenium WebDriver

I, also, need to know whether this is a valid implementation of the factory pattern. The amount of effort which goes into configuring DriverFactory to simply get a ...
0
votes
2answers
140 views

2048 Webgame bot

So there is this game 2048. You can play it here I wrote a python script which "plays" the game on the website by clicking the arrow keys. I would like to know what can be improved. Is it a good ...
10
votes
1answer
6k views

Simple yet Hackable WhatsApp API [UNOFFICIAL]

There is no official WhatsApp API. Here is a simple python class which satisfies the need. ...
4
votes
1answer
88 views

Python Selenium webscraper for job listing website

I'm a newbie getting into web scrapers. I've made something that works, but it takes hours and hours to get everything I need. I read something about using parallel processes to process the URLs but I ...
1
vote
1answer
93 views

Automate booking on a travel site

Given below is my task Write a selenium test that makes a booking from Bangalore to New Delhi for today's date with a return of tomorrow using makemytrip.com. Select the cheapest flight and ...
3
votes
1answer
426 views

Measure website home page total network size in bytes

We have a requirement to check the size of some of our website home pages, in MB. This is to ensure that they are not growing too large, and that large images haven't been uploaded. I couldn't find ...
9
votes
1answer
8k views

Scraping the full content from a lazy-loading webpage

I've written a script in python in combination with selenium which is able to scrape 1000 links from a webpage in which lazy-loading method is applied for that reason it displays it's content 20 at a ...
5
votes
1answer
2k views

Extracting necessary records from LinkedIn

I wanted to create a scraper in python which can fetch required data from LinkedIn. I tried with python in many different ways but I could not make it until I used selenium in combination with. ...
-1
votes
2answers
61 views

Finding length of cells in a row

I'm facing the following problem: I have a row of cells. Some cells are available, some are not. Every cell is associated with a cell_id. I want to determine the ...
1
vote
0answers
104 views

Extracting data from a used car sales site

I am developing code for extracting data from a used car sales site. There are 4 sites in total. In 3 of them I use requests and beautifulsoup. The time taken to extract data from these sites was ...
2
votes
1answer
184 views

Program to automate skipping users on a chat website until the username meets certain conditions (v.2.0)

This is a python program that automates skipping through users on a chat website called talkwithstrangers using the Selenium Webdriver and BeautifulSoup modules of Python. The first part uses ...
0
votes
1answer
146 views

Program to automate clicking on the new button on a chat website until the username matches certain conditions

This is a python program that automates skipping through users on a chat website called talkwithstrangers using the Selenium Webdriver and ...
6
votes
2answers
518 views

Scraping NHL Individual stats

With the idea of gathering the biggest hockey's individual stats dataset ever, I've started with the big league, the NHL. Using Selenium, I've been able to scrape this NHL's statistics page. To ...
3
votes
0answers
130 views

Two methods with long switch statements [closed]

The code uses a string key to find the relevant element on the DOM, then the second switch uses the string key to query the DB with certain filter and where clause (or not in case of a date filter). ...
5
votes
1answer
4k views

Parsing a slow-loading webpage with scrapy in combination with selenium

I've written a scraper in Python scrapy in combination with selenium to scrape 1000 company names and their revenue from a website. The site has got lazy-loading method enabled so it is not possible ...
4
votes
2answers
139 views

Selenium framework for an insurance platform

I have recently started to create my first Selenium Framework to help test the product at my work. I have managed to implement the first three pages and would like some feedback on and selenium or C# ...
2
votes
1answer
2k views

Testing multiple logins using Selenium listed in a property file

Excuse please review my code. I am working in automation testing. I did a automation testing for my login page. I am using selenium with page object pattern. I wrote here my code. I am using java ...
2
votes
1answer
439 views

Firefox profile specs to speed up web scraping

I am scraping a pool of URLs with a python/selenium framework using Firefox and Geckodriver. For each URL, all I am doing is to find one element by class name and ...
3
votes
1answer
62 views

Selenium script that books a room

I have a lot to learn in Python, and now I am adventuring in automating tasks with Selenium. Specifically, to just quickly explain the purpose of the script I have created, I have made a script that ...
3
votes
1answer
131 views

Making a reservation using Selenium/PhantomJS

I have this small script that needs to make a reservation at a DMV office. The problem is that the reservation window opens only every few hours for a few minutes (it might show up in one of these ...
4
votes
2answers
1k views

Scraper to deal with some complicated site with ads

I've written a script in Python in combination with Selenium to scrape data from a slightly complicated web page. When the web page is made to open, it comes up with an ad sitting on the data making ...
2
votes
0answers
612 views

Optimization Resource File for Robot Framework

I'm looking for optimization between these files, and simplification. If something already exists to do what I have a keyword do, I'd like to know about it. Loops.py is located in one of my other ...
1
vote
1answer
515 views

Click through a table of hyperlinks

My goal was to click every single hyperlink under the "Shortcuts" header on here. The code looks like it works to me, but if you have ideas that could help me improve/change I would love to hear them!...
2
votes
1answer
840 views

A universal email crawler

I've written a script in python using requests module in combination with selenium along with regex to parse email address (if any exists) from any website. I tried to create it in such a way so that ...
1
vote
1answer
1k views

Python selenium object repository

I was trying to create the selenium web element object repository in python. I have come up with three ways to do so. However, I am new to python so I am not sure which one is better than other. ...
1
vote
0answers
811 views

Click the center of an image if that image appears on the screen

To be clear, this code is built to be used with the Snipping Tool and Robot Framework. Use the snipping tool to snip an image of something on the screen. A strange button with weird locators, ...
1
vote
2answers
98 views

Selecting multiple filters based on the true/false conditions

I wrote a method to select the filters (two or more) based on the true/false conditions. Here is the method I used for selecting the multiple filters: ...
3
votes
1answer
2k views

Scraping restaurant names and addresses from a website using Scrapy and Selenium

I have written some Python code that uses Scrapy and Selenium to scrape restaurant names and addresses from a website. I needed to use Selenium because the button to show more restaurants on a page is ...
1
vote
1answer
1k views

Making a dynamic parser to collect data from a javascript enabled webpage

I've written a scraper in VBA in combination with selenium to parse product names and prices from a javascript enabled webpage. The thing is that I just used selenium to get the page source (as it was ...
2
votes
1answer
391 views

Scraping names from multipages without clicking on the next button

I've written a script in python in association with selenium to harvest some coffee shop names from yellowpages. Although, the webpage is not javascript injected one, I used selenium for the purpose ...
3
votes
1answer
130 views

Extracting some names from a webpage

I've written some code in python in combination with selenium to collect photo shoot spaces in different locations in Paris. My scraper is harvesting the names successfully at this moment. However, ...
2
votes
1answer
565 views

Scraping data from tables connected to each dot on a map

I've written a script in python with selenium to scrape different table data lie within different dots on a map in a certain website. Each table is connected to each dot. The table is activated once ...
1
vote
1answer
1k views

Python + selenium scraper to grab results using reverse search

I've written some code in python in combination with selenium to scrape populated result from a website after performing a reverse search. My scraper opens that site clicks on the "search by address"...