Questions tagged [selenium]

Selenium is a tools suite to automate web browsers, and web application testing

Filter by
Sorted by
Tagged with
4
votes
1answer
56 views

Web Scraping using Selenium and Python

This is my first program code ever, and it actually works. My goal is to scrape information from the website and store it in a database. It is a site that has historical data on sporting events and ...
5
votes
1answer
137 views

Game scraper for Steam

I made a scraper for Steam that get different info about a Steam game, such as the price, the specs and the supported platforms. The reason I made this was because I have a super slow laptop, so ...
3
votes
1answer
81 views

Project : My Anime Finder

This is a project that finds a specific anime (show) and checks to see if there is a new episode of it . this is also the first time that i used def like this and ...
6
votes
2answers
68 views

Web scraper for driver's license test times

I have created a small selenium script that checks for available times to write a test for a drivers license. The program runs every minute and takes approx 50 seconds to run. I have noticed that it's ...
6
votes
2answers
176 views

Finding solutions on GitHub and Stack Overflow

The first part of the code opens Chrome and navigates to GitHub. If I don't find my solution, I open a new tab with ctrlt. Then my code goes to Stack Overflow and searches there for solutions. ...
5
votes
2answers
87 views

Crawling a court website and downloading records

I am new to Python and I am using Python 3.8 and Selenium. This is a follow up of my previous question. Since I received several useful suggestions I have re-written the following code. The code works ...
3
votes
1answer
66 views

Web Scraping Dynamically Generated Content Python

This is a small web scraping project I made in 2 hours that targets the website remote.co . I am looking forward for improvements in my code. I know about the inconsistency with the WebDriverWait and ...
4
votes
3answers
323 views

Crawl a website and download records

I am new to Python. Amateur, enthusiast and learner. I have developed code which visits one particular website. Crawls through it to reach a certain repository, downloads the records from there to the ...
4
votes
1answer
95 views

Webscraping With Selenium - a Course Downloader and Sorter

I've been working on a program that automates the painful task of downloading each file from a website for courses one by one and sorting them using python. This is my first python webscraping project ...
2
votes
1answer
230 views

Instagram Scraping Using Selenium - Download Posts - Photos - Videos

Python script that can downloads public and private profiles images and videos, like Gallery with photos or videos. It saves the data in the folder. How it works: Log in in instragram using selenium ...
5
votes
1answer
622 views

Scraping Instagram - Download posts, photos - videos

Python script that can downloads public and private profiles images and videos, like Gallery with photos or videos. It saves the data in the folder. How it works: Log in in instragram using selenium ...
2
votes
1answer
87 views

Download pictures (or videos) from Instagram using Selenium

Python script that can downloads public and private profiles images and videos, like Gallery with photos or videos. It saves the data in the folder. How it works: Log in in instragram using selenium ...
2
votes
1answer
147 views

Instagram Scraping Posts Using Selenium

Python script that can download images and videos of the user, like Gallery with photos or videos. It saves the data in the folder. How it works: Log in in instragram using selenium and navigate to ...
2
votes
1answer
160 views

Instagram Scraping Using Selenium

Python script that can download images and videos of the user, like Gallery with photos or videos. It saves the data in the folder. How it works: Log in in instragram using selenium and navigate to ...
1
vote
1answer
170 views

Instagram Bot, selenium, web scraping

I made some changes in my code from the previous post. The changes that I made: I put all the functions to the class All the global arrays I moved them to class too Created ...
2
votes
1answer
261 views

Web scraping using selenium, multiprocessing, InstagramBot

An Instagram Bot which downloads the posts from profile I have to mention my previous posts: Instagram scraper Posts (Videos and Photos) Scraping Instagram with selenium, extract URLs, download ...
3
votes
1answer
808 views

Scraping Instagram with selenium, extract URLs, download posts

I made a very simple Instagram Bot that can download images and videos of the user, like Gallery with photos or videos. It saves the data in the folder. How it works: Creating directory for saving ...
4
votes
1answer
205 views

Instagram scraper Posts (Videos and Photos)

I wrote this code which has the ability to download images and videos from a specific Instagram profile. Using multiprocessing and threading I managed to speed up the extraction of data. My goal is ...
3
votes
1answer
147 views

Elegant Method for Sleeping with Selenium Webscraper

My goal is to scrape the no. 1 suggestion of the suggestion list on https://finance.yahoo.com for any entry. If there is no suggestion the code should write ...
1
vote
0answers
32 views

Getting all descendants IWebElements (using XPath) is too slow in Selenium

Well, I'm working with Selenium to scrap a webpage that has obfuscated class names (example <div class="example-gHJkLM">...)... So I needed to create an ...
2
votes
0answers
51 views

Web scraping code using selenium and Python

I have a code to scrap data from webpage using selenium and python. The code selects a URL the database and search it using selenium chrome driver and fetches the result and insert into a postgreSQL ...
5
votes
1answer
112 views

Using Python & Selenium to set Gnome desktop to Google's Chromecast wallpapers

I created a simple script that is used to implement Google's Chromecast wallpaper-images as the wallpaper of my Gnome desktop. The script extract the images from 2 different sources: A static ...
13
votes
3answers
2k views

AniPop - The anime downloader

Note: The topics of performance and Selenium/BS4 have not yet been addressed, so this question can still receive a better answer! Chat Room: https://chat.stackexchange.com/rooms/100275/anipop-...
4
votes
0answers
124 views

The anime downloader [duplicate]

NOTE: Here's the latest version of this program, since this question idled out. This is a recreational script made to update my home server w/ the latest anime from HorribleSubs. I'd like to know if ...
4
votes
1answer
50 views

Python script for parsing links

I wrote a test that parses links from a web page, and in a loop, clicks on each of them, and then returns to the main page. But, each iteration is accompanied by parsing the entire web page again and ...
1
vote
1answer
50 views

Selenium - Java data driven Framework suggestion

Dropdown Screenshot I have 3 dropdowns with multiple components in the UI. Have incorporated data driven java selenium framework, data is fetched from excel sheet: Excel sheet- where data is fetched ...
3
votes
1answer
74 views

Get result from form website operation

I want to get a result from the website when I submit a form. There are several things that can go wrong. In a simple scenario I want to return: true - when ...
3
votes
1answer
1k views

Python script to download adult comics from 8muses

I made a simple python script using the BeautifulSoup and Selenium to automatically download adult comics from 8muses. I've used selenium for the reason that the website uses javascript to load the ...
2
votes
1answer
432 views

Submit random choices on Google form

I've written a Python script that randomly selects radio buttons for each question, and submits the google form. My main questions for improvement are: Algorithm: I collect all the buttons into a 2D ...
1
vote
0answers
46 views

Behat Page Object Model

I have a working solution on a training exercise that I wish to improve as I feel there is a better way to utilize the form and get to the fieldsets etc using page model. HTML: ...
2
votes
0answers
508 views

Python, Selenium, Pandas project; for populating web form page on loop

I have inherited this nightmare Python, Selenium, and Pandas project. I am learning Python as I go; and have been cleaning up and refactoring the code where I see necessary - It was worst if you can ...
1
vote
1answer
152 views

Select depart date, 5 days from today and return date, 6 days from today in Selenium

I am automating a scenario of flight booking on Cleartrip.com(https://www.cleartrip.com/flights) in which I have to enter depart date, 5 days from today and return date, 6 days from today. I have ...
1
vote
0answers
86 views

Automating A Workflow Using Selenium - Best Way To Fetch Elements & Make Code More Generic

I'm a newbie to Selenium and I'm trying to get better at it by taking up interview assignments. This YouTube link describes the workflow that needs to be automated. The following points are worth ...
4
votes
1answer
108 views

Uploading songs from website to database and then to Spotify

My Python program parses songs from a website, corrects song titles and artists with the Last.fm API, searches for the spotify uri using the Spotify API, stores all the information in a SQLite ...
0
votes
1answer
105 views

Factory to instantiate and configure Selenium WebDriver

I, also, need to know whether this is a valid implementation of the factory pattern. The amount of effort which goes into configuring DriverFactory to simply get a ...
0
votes
2answers
254 views

2048 Webgame bot

So there is this game 2048. You can play it here I wrote a python script which "plays" the game on the website by clicking the arrow keys. I would like to know what can be improved. Is it a good ...
4
votes
1answer
114 views

Python Selenium webscraper for job listing website

I'm a newbie getting into web scrapers. I've made something that works, but it takes hours and hours to get everything I need. I read something about using parallel processes to process the URLs but I ...
1
vote
1answer
268 views

Automate booking on a travel site

Given below is my task Write a selenium test that makes a booking from Bangalore to New Delhi for today's date with a return of tomorrow using makemytrip.com. Select the cheapest flight and ...
3
votes
1answer
737 views

Measure website home page total network size in bytes

We have a requirement to check the size of some of our website home pages, in MB. This is to ensure that they are not growing too large, and that large images haven't been uploaded. I couldn't find ...
2
votes
0answers
829 views

Grabbing information traversing multiple pages

I've written a script in python in combination with selenium to parse different information from a webpage and store the collected data in a csv file. The data are rightly coming through. The email ...
-1
votes
2answers
66 views

Finding length of cells in a row

I'm facing the following problem: I have a row of cells. Some cells are available, some are not. Every cell is associated with a cell_id. I want to determine the ...
1
vote
0answers
169 views

Extracting data from a used car sales site

I am developing code for extracting data from a used car sales site. There are 4 sites in total. In 3 of them I use requests and beautifulsoup. The time taken to extract data from these sites was ...
2
votes
1answer
197 views

Program to automate skipping users on a chat website until the username meets certain conditions (v.2.0)

This is a python program that automates skipping through users on a chat website called talkwithstrangers using the Selenium Webdriver and BeautifulSoup modules of Python. The first part uses ...
6
votes
2answers
878 views

Scraping NHL Individual stats

With the idea of gathering the biggest hockey's individual stats dataset ever, I've started with the big league, the NHL. Using Selenium, I've been able to scrape this NHL's statistics page. To ...
0
votes
1answer
156 views

Program to automate clicking on the new button on a chat website until the username matches certain conditions

This is a python program that automates skipping through users on a chat website called talkwithstrangers using the Selenium Webdriver and ...
3
votes
0answers
131 views

Two methods with long switch statements [closed]

The code uses a string key to find the relevant element on the DOM, then the second switch uses the string key to query the DB with certain filter and where clause (or not in case of a date filter). ...
4
votes
2answers
149 views

Selenium framework for an insurance platform

I have recently started to create my first Selenium Framework to help test the product at my work. I have managed to implement the first three pages and would like some feedback on and selenium or C# ...
2
votes
1answer
674 views

Firefox profile specs to speed up web scraping

I am scraping a pool of URLs with a python/selenium framework using Firefox and Geckodriver. For each URL, all I am doing is to find one element by class name and ...
3
votes
1answer
63 views

Selenium script that books a room

I have a lot to learn in Python, and now I am adventuring in automating tasks with Selenium. Specifically, to just quickly explain the purpose of the script I have created, I have made a script that ...
3
votes
1answer
145 views

Making a reservation using Selenium/PhantomJS

I have this small script that needs to make a reservation at a DMV office. The problem is that the reservation window opens only every few hours for a few minutes (it might show up in one of these ...