Questions tagged [selenium]

Selenium is a tools suite to automate web browsers, and web application testing

Filter by
Sorted by
Tagged with
13
votes
1answer
1k views

Functions to hide and reclaim first visible publication on a page using Selenium

I have the following functions in JavaScript: ...
13
votes
3answers
2k views

AniPop - The anime downloader

Note: The topics of performance and Selenium/BS4 have not yet been addressed, so this question can still receive a better answer! Chat Room: https://chat.stackexchange.com/rooms/100275/anipop-...
11
votes
1answer
12k views

Scraping the full content from a lazy-loading webpage

I've written a script in python in combination with selenium which is able to scrape 1000 links from a webpage in which lazy-loading method is applied for that reason it displays it's content 20 at a ...
10
votes
3answers
3k views

Scraping the date of most recent post from various social media services

Task I have a large spreadsheet where each line should include: The URL of a social media account A field indicating whether the account is "active" A name and UID number for each account I have to ...
10
votes
2answers
160 views

Get Wikimedia attributions for images

We are using images from Wikimedia commons for some of Cardshifter's game artwork. I selected some art for each card to make a .jpg file to use in the game clients, ...
10
votes
1answer
7k views

Simple yet Hackable WhatsApp API [UNOFFICIAL]

There is no official WhatsApp API. Here is a simple python class which satisfies the need. ...
8
votes
1answer
6k views

Scraping scores from flashscore.com

I built a bot with Python to scrap scores on flashscore.com but the data scrap from the site loads into its listbox very slowly. I am curious about the speed of selenium so I made a button that prints ...
7
votes
1answer
1k views

Scraper for a lazy-loading webpage

I've written a script in VBA in combination with selenium which is able to scrape all the 1000 links from a lazy-loading webpage which displays it's content 20 at a time and doesn't display the full ...
6
votes
2answers
885 views

Scraping NHL Individual stats

With the idea of gathering the biggest hockey's individual stats dataset ever, I've started with the big league, the NHL. Using Selenium, I've been able to scrape this NHL's statistics page. To ...
6
votes
2answers
182 views

Finding solutions on GitHub and Stack Overflow

The first part of the code opens Chrome and navigates to GitHub. If I don't find my solution, I open a new tab with ctrlt. Then my code goes to Stack Overflow and searches there for solutions. ...
6
votes
2answers
1k views

Selenium-based link checker for a shopping site

I just started learning Python and I wrote my first useful script for work. I did a bunch of the basic tutorials and really enjoyed learning Python so far. I am looking for any advice on how to make ...
6
votes
2answers
5k views

Selenium Framework using a lot of statics

I am writing a Selenium framework that should be really easy for a tester with relatively little Java knowledge to use and write tests for. In order to keep the framework as user friendly as possible, ...
6
votes
1answer
5k views

Retrieving value from HTML table

Currently all this does is retrieve data from specific columns, the final result will be that I will check the data to make sure that it passes the test cases, but I want to look at this foreach right ...
6
votes
0answers
50 views

Booking an East London Tennis Court

Description I'm not sure if it Covid-19 but lately it is impossible to book a tennis court in my area on time. It's always full or maybe just don't check enough :) To beat the queue and get notified ...
6
votes
2answers
72 views

Web scraper for driver's license test times

I have created a small selenium script that checks for available times to write a test for a drivers license. The program runs every minute and takes approx 50 seconds to run. I have noticed that it's ...
5
votes
2answers
5k views

Checking if text is NOT displayed

I'm currently writing unit tests in Selenium (C#). I need to do two things in this particular test: Check if 2 errors (with html container) are displayed and check if another (given) error is not ...
5
votes
2answers
87 views

Crawling a court website and downloading records

I am new to Python and I am using Python 3.8 and Selenium. This is a follow up of my previous question. Since I received several useful suggestions I have re-written the following code. The code works ...
5
votes
1answer
3k views

Selecting filter checkboxes using Selenium

I am using Selenium Webdriver via Java. My method clicks on a filter check box (eg. make, model, etc.), that is passed in, from a modal. When running my method in my test class, via ChromeDriver, it ...
5
votes
1answer
682 views

Scraping Instagram - Download posts, photos - videos

Python script that can downloads public and private profiles images and videos, like Gallery with photos or videos. It saves the data in the folder. How it works: Log in in instragram using selenium ...
5
votes
1answer
6k views

Parsing a slow-loading webpage with scrapy in combination with selenium

I've written a scraper in Python scrapy in combination with selenium to scrape 1000 company names and their revenue from a website. The site has got lazy-loading method enabled so it is not possible ...
5
votes
1answer
2k views

Extracting necessary records from LinkedIn

I wanted to create a scraper in python which can fetch required data from LinkedIn. I tried with python in many different ways but I could not make it until I used selenium in combination with. ...
5
votes
2answers
4k views

Parser for Facebook friend list

I have written some code in python in combination with selenium to parse all the names from facebook friend list. It was hard to manage the pop up notification and the process of scrolling to the end ...
5
votes
1answer
144 views

Game scraper for Steam

I made a scraper for Steam that get different info about a Steam game, such as the price, the specs and the supported platforms. The reason I made this was because I have a super slow laptop, so ...
5
votes
1answer
3k views

Scraping all closed captions (subtitles) of a YouTube's creator's video library

A script that scrapes all YouTube's videos closed captions(subtitles) of a single creator's video library. It scrapes either auto-generated or manually written subtitles. The script accepts a link to ...
5
votes
1answer
2k views

Using Page Object Model in Selenium tests

I am very new to Selenium, and for my current project, I had the task of creating the automated test scripts for our website. I am the only one on my team who had any experience with Selenium, but not ...
5
votes
1answer
7k views

Ruby selenium script waiting for an element to disappear

I'm a Java developer, I'm new to Ruby and I'm worried that I'm forcing or not-so-well using the goodness of the Ruby syntax. What do you think about the Exception catching and how to print it in ...
5
votes
1answer
114 views

Using Python & Selenium to set Gnome desktop to Google's Chromecast wallpapers

I created a simple script that is used to implement Google's Chromecast wallpaper-images as the wallpaper of my Gnome desktop. The script extract the images from 2 different sources: A static ...
5
votes
1answer
11k views

Copy table using Selenium and Python

I have some Python code that copies a table from a website using Selenium,to create a csv, but as I haven't used Selenium much, I have that irksome feeling in the back of my mind that there must be a ...
5
votes
1answer
37 views

Automating TypeRacer using selenium

This is a personal project which I created solely to torture my friends ;) I'm pretty sure you've heard about typeracer. And if you have friends like mine who are faster than you, you'd surely be ...
5
votes
1answer
187 views

Crawler for scraping and approving friend requests in Facebook

I've made a Facebook scraper using VBA in combination with selenium which is able to parse the name from friend request list. It is necessary to do one thing manually, though. After running this ...
4
votes
4answers
2k views

Using if else statements correctly

I have the following code, but I'm looking for a way to make the code read easier. ...
4
votes
3answers
327 views

Crawl a website and download records

I am new to Python. Amateur, enthusiast and learner. I have developed code which visits one particular website. Crawls through it to reach a certain repository, downloads the records from there to the ...
4
votes
2answers
2k views

Scraper to deal with some complicated site with ads

I've written a script in Python in combination with Selenium to scrape data from a slightly complicated web page. When the web page is made to open, it comes up with an ad sitting on the data making ...
4
votes
2answers
368 views

Python Stock Checker

...
4
votes
1answer
108 views

Uploading songs from website to database and then to Spotify

My Python program parses songs from a website, corrects song titles and artists with the Last.fm API, searches for the spotify uri using the Spotify API, stores all the information in a SQLite ...
4
votes
2answers
549 views

Improving performance in a Webdriver method

I'm testing an application which is not exactly an e-commerce application but which behaves enough like one that you can think of it as an e-commerce application for the purposes of understanding this ...
4
votes
1answer
59 views

Web Scraping using Selenium and Python

This is my first program code ever, and it actually works. My goal is to scrape information from the website and store it in a database. It is a site that has historical data on sporting events and ...
4
votes
2answers
149 views

Selenium framework for an insurance platform

I have recently started to create my first Selenium Framework to help test the product at my work. I have managed to implement the first three pages and would like some feedback on and selenium or C# ...
4
votes
3answers
1k views

Fill out webform with Selenium

I am automating a webform on a webpage. I am still learning Selenium on the road. I know a few bad/good practices but a code review is always a good thing to do. What should I improve on in my code? ...
4
votes
1answer
214 views

Instagram scraper Posts (Videos and Photos)

I wrote this code which has the ability to download images and videos from a specific Instagram profile. Using multiprocessing and threading I managed to speed up the extraction of data. My goal is ...
4
votes
1answer
50 views

Python script for parsing links

I wrote a test that parses links from a web page, and in a loop, clicks on each of them, and then returns to the main page. But, each iteration is accompanied by parsing the entire web page again and ...
4
votes
1answer
906 views

GUI for Selenium web testing

I am wanting to build web-based testing tools for test website environments utilizing the Selenium Web Driver. I am building a GUI which will contain the following: Drop-down for selecting the website,...
4
votes
1answer
97 views

Webscraping With Selenium - a Course Downloader and Sorter

I've been working on a program that automates the painful task of downloading each file from a website for courses one by one and sorting them using python. This is my first python webscraping project ...
4
votes
1answer
114 views

Python Selenium webscraper for job listing website

I'm a newbie getting into web scrapers. I've made something that works, but it takes hours and hours to get everything I need. I read something about using parallel processes to process the URLs but I ...
4
votes
1answer
1k views

Selenium WebDriver test for Slack

I submitted this script as part of a interview process. It was rejected and no reasons were given. I'm wondering what parts of my script were unacceptable. The script works, and satisfies the question,...
4
votes
0answers
124 views

The anime downloader [duplicate]

NOTE: Here's the latest version of this program, since this question idled out. This is a recreational script made to update my home server w/ the latest anime from HorribleSubs. I'd like to know if ...
3
votes
1answer
167 views

Extracting some names from a webpage

I've written some code in python in combination with selenium to collect photo shoot spaces in different locations in Paris. My scraper is harvesting the names successfully at this moment. However, ...
3
votes
1answer
146 views

Making a reservation using Selenium/PhantomJS

I have this small script that needs to make a reservation at a DMV office. The problem is that the reservation window opens only every few hours for a few minutes (it might show up in one of these ...
3
votes
1answer
948 views

Scraping Instagram with selenium, extract URLs, download posts

I made a very simple Instagram Bot that can download images and videos of the user, like Gallery with photos or videos. It saves the data in the folder. How it works: Creating directory for saving ...
3
votes
1answer
149 views

Elegant Method for Sleeping with Selenium Webscraper

My goal is to scrape the no. 1 suggestion of the suggestion list on https://finance.yahoo.com for any entry. If there is no suggestion the code should write ...