Questions tagged [selenium]

Selenium is a tools suite to automate web browsers, and web application testing

Filter by
Sorted by
Tagged with
1
vote
0answers
35 views

Python selenium web scraping

Written following code to extract table form portal. The code works fine (except the net connectivity). But 1. I want to make it more simple, more flat. 2. If I can use explicit wait to avoid stale ...
2
votes
0answers
28 views

Web scraper for driver's license test times

I have created a small selenium script that checks for available times to write a test for a drivers license. The program runs every minute and takes approx 50 seconds to run. I have noticed that it's ...
6
votes
2answers
164 views

Finding solutions on GitHub and Stack Overflow

The first part of the code opens Chrome and navigates to GitHub. If I don't find my solution, I open a new tab with ctrlt. Then my code goes to Stack Overflow and searches there for solutions. ...
5
votes
2answers
86 views

Crawling a court website and downloading records

I am new to Python and I am using Python 3.8 and Selenium. This is a follow up of my previous question. Since I received several useful suggestions I have re-written the following code. The code works ...
3
votes
1answer
58 views

Web Scraping Dynamically Generated Content Python

This is a small web scraping project I made in 2 hours that targets the website remote.co . I am looking forward for improvements in my code. I know about the inconsistency with the WebDriverWait and ...
4
votes
3answers
318 views

Crawl a website and download records

I am new to Python. Amateur, enthusiast and learner. I have developed code which visits one particular website. Crawls through it to reach a certain repository, downloads the records from there to the ...
3
votes
1answer
69 views

Webscraping With Selenium - a Course Downloader and Sorter

I've been working on a program that automates the painful task of downloading each file from a website for courses one by one and sorting them using python. This is my first python webscraping project ...
2
votes
1answer
128 views

Instagram Scraping Using Selenium - Download Posts - Photos - Videos

Python script that can downloads public and private profiles images and videos, like Gallery with photos or videos. It saves the data in the folder. How it works: Log in in instragram using selenium ...
5
votes
1answer
333 views

Scraping Instagram - Download posts, photos - videos

Python script that can downloads public and private profiles images and videos, like Gallery with photos or videos. It saves the data in the folder. How it works: Log in in instragram using selenium ...
2
votes
1answer
63 views

Download pictures (or videos) from Instagram using Selenium

Python script that can downloads public and private profiles images and videos, like Gallery with photos or videos. It saves the data in the folder. How it works: Log in in instragram using selenium ...
5
votes
1answer
7k views

Ruby selenium script waiting for an element to disappear

I'm a Java developer, I'm new to Ruby and I'm worried that I'm forcing or not-so-well using the goodness of the Ruby syntax. What do you think about the Exception catching and how to print it in ...
2
votes
1answer
187 views

Web scraping using selenium, multiprocessing, InstagramBot

An Instagram Bot which downloads the posts from profile I have to mention my previous posts: Instagram scraper Posts (Videos and Photos) Scraping Instagram with selenium, extract URLs, download ...
2
votes
1answer
99 views

Instagram Scraping Posts Using Selenium

Python script that can download images and videos of the user, like Gallery with photos or videos. It saves the data in the folder. How it works: Log in in instragram using selenium and navigate to ...
1
vote
1answer
128 views

Instagram Scraping Using Selenium

Python script that can download images and videos of the user, like Gallery with photos or videos. It saves the data in the folder. How it works: Log in in instragram using selenium and navigate to ...
1
vote
1answer
152 views

Instagram Bot, selenium, web scraping

I made some changes in my code from the previous post. The changes that I made: I put all the functions to the class All the global arrays I moved them to class too Created ...
3
votes
1answer
462 views

Scraping Instagram with selenium, extract URLs, download posts

I made a very simple Instagram Bot that can download images and videos of the user, like Gallery with photos or videos. It saves the data in the folder. How it works: Creating directory for saving ...
4
votes
1answer
179 views

Instagram scraper Posts (Videos and Photos)

I wrote this code which has the ability to download images and videos from a specific Instagram profile. Using multiprocessing and threading I managed to speed up the extraction of data. My goal is ...
3
votes
1answer
141 views

Elegant Method for Sleeping with Selenium Webscraper

My goal is to scrape the no. 1 suggestion of the suggestion list on https://finance.yahoo.com for any entry. If there is no suggestion the code should write ...
1
vote
0answers
32 views

Getting all descendants IWebElements (using XPath) is too slow in Selenium

Well, I'm working with Selenium to scrap a webpage that has obfuscated class names (example <div class="example-gHJkLM">...)... So I needed to create an ...
2
votes
0answers
49 views

Web scraping code using selenium and Python

I have a code to scrap data from webpage using selenium and python. The code selects a URL the database and search it using selenium chrome driver and fetches the result and insert into a postgreSQL ...
5
votes
1answer
100 views

Using Python & Selenium to set Gnome desktop to Google's Chromecast wallpapers

I created a simple script that is used to implement Google's Chromecast wallpaper-images as the wallpaper of my Gnome desktop. The script extract the images from 2 different sources: A static ...
13
votes
3answers
2k views

AniPop - The anime downloader

Note: The topics of performance and Selenium/BS4 have not yet been addressed, so this question can still receive a better answer! Chat Room: https://chat.stackexchange.com/rooms/100275/anipop-...
4
votes
0answers
124 views

The anime downloader [duplicate]

NOTE: Here's the latest version of this program, since this question idled out. This is a recreational script made to update my home server w/ the latest anime from HorribleSubs. I'd like to know if ...
5
votes
1answer
3k views

Scraping all closed captions (subtitles) of a YouTube's creator's video library

A script that scrapes all YouTube's videos closed captions(subtitles) of a single creator's video library. It scrapes either auto-generated or manually written subtitles. The script accepts a link to ...
4
votes
1answer
50 views

Python script for parsing links

I wrote a test that parses links from a web page, and in a loop, clicks on each of them, and then returns to the main page. But, each iteration is accompanied by parsing the entire web page again and ...
1
vote
1answer
50 views

Selenium - Java data driven Framework suggestion

Dropdown Screenshot I have 3 dropdowns with multiple components in the UI. Have incorporated data driven java selenium framework, data is fetched from excel sheet: Excel sheet- where data is fetched ...
3
votes
1answer
132 views

Initializing a controller that uses Selenium

I've outsourced the setup functions from my class to another file, I outsourced initDriver from the class PortalController to ...
2
votes
1answer
407 views

Submit random choices on Google form

I've written a Python script that randomly selects radio buttons for each question, and submits the google form. My main questions for improvement are: Algorithm: I collect all the buttons into a 2D ...
3
votes
1answer
74 views

Get result from form website operation

I want to get a result from the website when I submit a form. There are several things that can go wrong. In a simple scenario I want to return: true - when ...
2
votes
0answers
796 views

Grabbing information traversing multiple pages

I've written a script in python in combination with selenium to parse different information from a webpage and store the collected data in a csv file. The data are rightly coming through. The email ...
3
votes
1answer
4k views

Extracting all the data from a table out of a javascript encrypted website

I've written a script to harvest all the data out of a table from a webpage using python in combination with selenium. It takes a while to parse them all. There are seven steps to hurdle to get to the ...
5
votes
1answer
3k views

Selecting filter checkboxes using Selenium

I am using Selenium Webdriver via Java. My method clicks on a filter check box (eg. make, model, etc.), that is passed in, from a modal. When running my method in my test class, via ChromeDriver, it ...
3
votes
1answer
1k views

Python script to download adult comics from 8muses

I made a simple python script using the BeautifulSoup and Selenium to automatically download adult comics from 8muses. I've used selenium for the reason that the website uses javascript to load the ...
1
vote
0answers
46 views

Behat Page Object Model

I have a working solution on a training exercise that I wish to improve as I feel there is a better way to utilize the form and get to the fieldsets etc using page model. HTML: ...
2
votes
0answers
480 views

Python, Selenium, Pandas project; for populating web form page on loop

I have inherited this nightmare Python, Selenium, and Pandas project. I am learning Python as I go; and have been cleaning up and refactoring the code where I see necessary - It was worst if you can ...
1
vote
1answer
124 views

Select depart date, 5 days from today and return date, 6 days from today in Selenium

I am automating a scenario of flight booking on Cleartrip.com(https://www.cleartrip.com/flights) in which I have to enter depart date, 5 days from today and return date, 6 days from today. I have ...
4
votes
3answers
1k views

Fill out webform with Selenium

I am automating a webform on a webpage. I am still learning Selenium on the road. I know a few bad/good practices but a code review is always a good thing to do. What should I improve on in my code? ...
1
vote
0answers
86 views

Automating A Workflow Using Selenium - Best Way To Fetch Elements & Make Code More Generic

I'm a newbie to Selenium and I'm trying to get better at it by taking up interview assignments. This YouTube link describes the workflow that needs to be automated. The following points are worth ...
4
votes
1answer
106 views

Uploading songs from website to database and then to Spotify

My Python program parses songs from a website, corrects song titles and artists with the Last.fm API, searches for the spotify uri using the Spotify API, stores all the information in a SQLite ...
0
votes
1answer
102 views

Factory to instantiate and configure Selenium WebDriver

I, also, need to know whether this is a valid implementation of the factory pattern. The amount of effort which goes into configuring DriverFactory to simply get a ...
0
votes
2answers
241 views

2048 Webgame bot

So there is this game 2048. You can play it here I wrote a python script which "plays" the game on the website by clicking the arrow keys. I would like to know what can be improved. Is it a good ...
10
votes
1answer
7k views

Simple yet Hackable WhatsApp API [UNOFFICIAL]

There is no official WhatsApp API. Here is a simple python class which satisfies the need. ...
4
votes
1answer
113 views

Python Selenium webscraper for job listing website

I'm a newbie getting into web scrapers. I've made something that works, but it takes hours and hours to get everything I need. I read something about using parallel processes to process the URLs but I ...
1
vote
1answer
202 views

Automate booking on a travel site

Given below is my task Write a selenium test that makes a booking from Bangalore to New Delhi for today's date with a return of tomorrow using makemytrip.com. Select the cheapest flight and ...
3
votes
1answer
685 views

Measure website home page total network size in bytes

We have a requirement to check the size of some of our website home pages, in MB. This is to ensure that they are not growing too large, and that large images haven't been uploaded. I couldn't find ...
11
votes
1answer
11k views

Scraping the full content from a lazy-loading webpage

I've written a script in python in combination with selenium which is able to scrape 1000 links from a webpage in which lazy-loading method is applied for that reason it displays it's content 20 at a ...
5
votes
1answer
2k views

Extracting necessary records from LinkedIn

I wanted to create a scraper in python which can fetch required data from LinkedIn. I tried with python in many different ways but I could not make it until I used selenium in combination with. ...
-1
votes
2answers
65 views

Finding length of cells in a row

I'm facing the following problem: I have a row of cells. Some cells are available, some are not. Every cell is associated with a cell_id. I want to determine the ...
1
vote
0answers
157 views

Extracting data from a used car sales site

I am developing code for extracting data from a used car sales site. There are 4 sites in total. In 3 of them I use requests and beautifulsoup. The time taken to extract data from these sites was ...
2
votes
1answer
197 views

Program to automate skipping users on a chat website until the username meets certain conditions (v.2.0)

This is a python program that automates skipping through users on a chat website called talkwithstrangers using the Selenium Webdriver and BeautifulSoup modules of Python. The first part uses ...