Questions tagged [selenium]

Selenium is a tools suite to automate web browsers, and web application testing

Filter by
Sorted by
Tagged with
2
votes
1answer
26 views

Instagram Scraping Using Selenium

Python script that can download images and videos of the user, like Gallery with photos or videos. It saves the data in the folder. How it works: Log in in instragram using selenium and navigate to ...
-2
votes
0answers
14 views

Python Selenium - How to dissect each web element [closed]

I've inherited some working code using selenium in python. I am trying to break this down bit by bit to really understand what is happening. But I'm struggling to do so. ...
1
vote
1answer
111 views

Instagram Bot, selenium, web scraping

I made some changes in my code from the previous post. The changes that I made: I put all the functions to the class All the global arrays I moved them to class too Created ...
2
votes
1answer
55 views

Web scraping using selenium, multiprocessing, InstagramBot

An Instagram Bot which downloads the posts from profile I have to mention my previous posts: Instagram scraper Posts (Videos and Photos) Scraping Instagram with selenium, extract URLs, download ...
3
votes
1answer
69 views

Scraping Instagram with selenium, extract URLs, download posts

I made a very simple Instagram Bot that can download images and videos of the user, like Gallery with photos or videos. It saves the data in the folder. How it works: Creating directory for saving ...
4
votes
1answer
117 views

Instagram scraper Posts (Videos and Photos)

I wrote this code which has the ability to download images and videos from a specific Instagram profile. Using multiprocessing and threading I managed to speed up the extraction of data. My goal is ...
3
votes
1answer
118 views

Elegant Method for Sleeping with Selenium Webscraper

My goal is to scrape the no. 1 suggestion of the suggestion list on https://finance.yahoo.com for any entry. If there is no suggestion the code should write ...
1
vote
0answers
28 views

Getting all descendants IWebElements (using XPath) is too slow in Selenium

Well, I'm working with Selenium to scrap a webpage that has obfuscated class names (example <div class="example-gHJkLM">...)... So I needed to create an ...
2
votes
0answers
43 views

Web scraping code using selenium and Python

I have a code to scrap data from webpage using selenium and python. The code selects a URL the database and search it using selenium chrome driver and fetches the result and insert into a postgreSQL ...
5
votes
1answer
77 views

Using Python & Selenium to set Gnome desktop to Google's Chromecast wallpapers

I created a simple script that is used to implement Google's Chromecast wallpaper-images as the wallpaper of my Gnome desktop. The script extract the images from 2 different sources: A static ...
13
votes
3answers
2k views

AniPop - The anime downloader

Note: The topics of performance and Selenium/BS4 have not yet been addressed, so this question can still receive a better answer! Chat Room: https://chat.stackexchange.com/rooms/100275/anipop-...
4
votes
0answers
122 views

The anime downloader [duplicate]

NOTE: Here's the latest version of this program, since this question idled out. This is a recreational script made to update my home server w/ the latest anime from HorribleSubs. I'd like to know if ...
4
votes
1answer
50 views

Python script for parsing links

I wrote a test that parses links from a web page, and in a loop, clicks on each of them, and then returns to the main page. But, each iteration is accompanied by parsing the entire web page again and ...
1
vote
1answer
48 views

Selenium - Java data driven Framework suggestion

Dropdown Screenshot I have 3 dropdowns with multiple components in the UI. Have incorporated data driven java selenium framework, data is fetched from excel sheet: Excel sheet- where data is fetched ...
3
votes
1answer
74 views

Get result from form website operation

I want to get a result from the website when I submit a form. There are several things that can go wrong. In a simple scenario I want to return: true - when ...
3
votes
1answer
1k views

Python script to download adult comics from 8muses

I made a simple python script using the BeautifulSoup and Selenium to automatically download adult comics from 8muses. I've used selenium for the reason that the website uses javascript to load the ...
0
votes
0answers
24 views

Select a value other then currently selected drop down value

Scenario:Select a any other value other then currently selected from a drop down. Please help/review the code and tell me whether this is a efficient solution? ...
2
votes
1answer
157 views

Submit random choices on Google form

I've written a Python script that randomly selects radio buttons for each question, and submits the google form. My main questions for improvement are: Algorithm: I collect all the buttons into a 2D ...
1
vote
0answers
45 views

Behat Page Object Model

I have a working solution on a training exercise that I wish to improve as I feel there is a better way to utilize the form and get to the fieldsets etc using page model. HTML: ...
2
votes
0answers
414 views

Python, Selenium, Pandas project; for populating web form page on loop

I have inherited this nightmare Python, Selenium, and Pandas project. I am learning Python as I go; and have been cleaning up and refactoring the code where I see necessary - It was worst if you can ...
1
vote
1answer
87 views

Select depart date, 5 days from today and return date, 6 days from today in Selenium

I am automating a scenario of flight booking on Cleartrip.com(https://www.cleartrip.com/flights) in which I have to enter depart date, 5 days from today and return date, 6 days from today. I have ...
1
vote
0answers
86 views

Automating A Workflow Using Selenium - Best Way To Fetch Elements & Make Code More Generic

I'm a newbie to Selenium and I'm trying to get better at it by taking up interview assignments. This YouTube link describes the workflow that needs to be automated. The following points are worth ...
4
votes
1answer
102 views

Uploading songs from website to database and then to Spotify

My Python program parses songs from a website, corrects song titles and artists with the Last.fm API, searches for the spotify uri using the Spotify API, stores all the information in a SQLite ...
0
votes
1answer
98 views

Factory to instantiate and configure Selenium WebDriver

I, also, need to know whether this is a valid implementation of the factory pattern. The amount of effort which goes into configuring DriverFactory to simply get a ...
0
votes
2answers
204 views

2048 Webgame bot

So there is this game 2048. You can play it here I wrote a python script which "plays" the game on the website by clicking the arrow keys. I would like to know what can be improved. Is it a good ...
4
votes
1answer
108 views

Python Selenium webscraper for job listing website

I'm a newbie getting into web scrapers. I've made something that works, but it takes hours and hours to get everything I need. I read something about using parallel processes to process the URLs but I ...
1
vote
1answer
148 views

Automate booking on a travel site

Given below is my task Write a selenium test that makes a booking from Bangalore to New Delhi for today's date with a return of tomorrow using makemytrip.com. Select the cheapest flight and ...
3
votes
1answer
586 views

Measure website home page total network size in bytes

We have a requirement to check the size of some of our website home pages, in MB. This is to ensure that they are not growing too large, and that large images haven't been uploaded. I couldn't find ...
2
votes
0answers
686 views

Grabbing information traversing multiple pages

I've written a script in python in combination with selenium to parse different information from a webpage and store the collected data in a csv file. The data are rightly coming through. The email ...
-1
votes
2answers
62 views

Finding length of cells in a row

I'm facing the following problem: I have a row of cells. Some cells are available, some are not. Every cell is associated with a cell_id. I want to determine the ...
1
vote
0answers
140 views

Extracting data from a used car sales site

I am developing code for extracting data from a used car sales site. There are 4 sites in total. In 3 of them I use requests and beautifulsoup. The time taken to extract data from these sites was ...
2
votes
1answer
190 views

Program to automate skipping users on a chat website until the username meets certain conditions (v.2.0)

This is a python program that automates skipping through users on a chat website called talkwithstrangers using the Selenium Webdriver and BeautifulSoup modules of Python. The first part uses ...
6
votes
2answers
771 views

Scraping NHL Individual stats

With the idea of gathering the biggest hockey's individual stats dataset ever, I've started with the big league, the NHL. Using Selenium, I've been able to scrape this NHL's statistics page. To ...
0
votes
1answer
152 views

Program to automate clicking on the new button on a chat website until the username matches certain conditions

This is a python program that automates skipping through users on a chat website called talkwithstrangers using the Selenium Webdriver and ...
3
votes
0answers
131 views

Two methods with long switch statements [closed]

The code uses a string key to find the relevant element on the DOM, then the second switch uses the string key to query the DB with certain filter and where clause (or not in case of a date filter). ...
4
votes
2answers
146 views

Selenium framework for an insurance platform

I have recently started to create my first Selenium Framework to help test the product at my work. I have managed to implement the first three pages and would like some feedback on and selenium or C# ...
2
votes
1answer
597 views

Firefox profile specs to speed up web scraping

I am scraping a pool of URLs with a python/selenium framework using Firefox and Geckodriver. For each URL, all I am doing is to find one element by class name and ...
3
votes
1answer
63 views

Selenium script that books a room

I have a lot to learn in Python, and now I am adventuring in automating tasks with Selenium. Specifically, to just quickly explain the purpose of the script I have created, I have made a script that ...
3
votes
1answer
139 views

Making a reservation using Selenium/PhantomJS

I have this small script that needs to make a reservation at a DMV office. The problem is that the reservation window opens only every few hours for a few minutes (it might show up in one of these ...
4
votes
2answers
2k views

Scraper to deal with some complicated site with ads

I've written a script in Python in combination with Selenium to scrape data from a slightly complicated web page. When the web page is made to open, it comes up with an ad sitting on the data making ...
2
votes
0answers
734 views

Optimization Resource File for Robot Framework

I'm looking for optimization between these files, and simplification. If something already exists to do what I have a keyword do, I'd like to know about it. Loops.py is located in one of my other ...
2
votes
1answer
902 views

A universal email crawler

I've written a script in python using requests module in combination with selenium along with regex to parse email address (if any exists) from any website. I tried to create it in such a way so that ...
5
votes
1answer
5k views

Parsing a slow-loading webpage with scrapy in combination with selenium

I've written a scraper in Python scrapy in combination with selenium to scrape 1000 company names and their revenue from a website. The site has got lazy-loading method enabled so it is not possible ...
1
vote
0answers
869 views

Click the center of an image if that image appears on the screen

To be clear, this code is built to be used with the Snipping Tool and Robot Framework. Use the snipping tool to snip an image of something on the screen. A strange button with weird locators, ...
1
vote
2answers
101 views

Selecting multiple filters based on the true/false conditions

I wrote a method to select the filters (two or more) based on the true/false conditions. Here is the method I used for selecting the multiple filters: ...
10
votes
1answer
6k views

Simple yet Hackable WhatsApp API [UNOFFICIAL]

There is no official WhatsApp API. Here is a simple python class which satisfies the need. ...
1
vote
1answer
2k views

Making a dynamic parser to collect data from a javascript enabled webpage

I've written a scraper in VBA in combination with selenium to parse product names and prices from a javascript enabled webpage. The thing is that I just used selenium to get the page source (as it was ...
1
vote
1answer
1k views

Python selenium object repository

I was trying to create the selenium web element object repository in python. I have come up with three ways to do so. However, I am new to python so I am not sure which one is better than other. ...
2
votes
1answer
427 views

Scraping names from multipages without clicking on the next button

I've written a script in python in association with selenium to harvest some coffee shop names from yellowpages. Although, the webpage is not javascript injected one, I used selenium for the purpose ...
3
votes
1answer
146 views

Extracting some names from a webpage

I've written some code in python in combination with selenium to collect photo shoot spaces in different locations in Paris. My scraper is harvesting the names successfully at this moment. However, ...