Questions tagged [python-requests]
USE ONLY FOR THE PYTHON REQUESTS LIBRARY. Requests is a full-featured Python HTTP library with an easy-to-use, logical API.
36
questions
2
votes
1
answer
129
views
Python script to POST requests to a networking device
Request to review my code please and see if I am following the best practices.
This is sending a POST operation via a Python script using the requests library to a networking cloud orchestration ...
1
vote
1
answer
81
views
Printing a JSON/HTTP response from a Cisco endpoint
Please check over my Python code for a HTTP GET operation using the requests library, and provide any potential pointers for improvement.
...
1
vote
1
answer
34
views
Iterating over tables with nested <th> tags with BS4
The following code iterates over a table with nested th headings.
The decision has been made to just go with the main heading and ignore sub-headings, but nested ...
- 321
3
votes
1
answer
51
views
Acupuncture database builder
The following code builds a rudimentary acupuncture database by collecting data from the web.
I would like to hear suggestions about improvements to the database structure, code organization, web-...
- 321
0
votes
0
answers
26
views
Python scraper that gets product information from e-commerce website
I have built a scraper that scrapes data from e-commerce website and saves it to json using my own module. Although, I made the scraper multi-threaded, it starts slow, then speeds up and uses a lot of ...
- 101
4
votes
2
answers
56
views
Search for providers of TV Shows
All my code is below. It uses an API to search for a show and then outputs the providers (like breaking bad is available on Netflix).
The main aim of this application was to practice using APIs and ...
- 41
0
votes
2
answers
101
views
Python script for Web scraping
I wrote a Python script for Web scraping of a Website. Please review my code and suggest me any changes or make me aware of my blunders/mistakes?.
I wrote the almost same script for other websites ...
0
votes
1
answer
63
views
Config Variables Initialization | Python
I found interesting the way Python's requests library does the status_code data structure initialization (see code here). I am reusing it in a hobby project, but ...
- 301
3
votes
1
answer
51
views
Refactor Web Scraper
I wrote a simple Zoopla real estate scraper for just practicing what I learned so far in Python, Requests, BeautifulSoup and overall web scraping fundamentals. By looking at my code I feel like there ...
2
votes
2
answers
154
views
Scrape webelements
I have been working with requests where I as easy as it is. Scraping the webpage and add it into a dict and print the payload if we find a new value or not.
...
- 503
2
votes
1
answer
754
views
Run python File With get request flask API
Code is Working Review and Recommend What Are Best Practice
What I am trying to achieve.
API with Flask that runs any python file from the current directory.
Run the file and get output in JSON
...
2
votes
2
answers
87
views
Web scraping and design pattern lifting
I would like to get some feed back on my code; the goal is to get all agencies address of banks. I wrote a pretty simple brute force algorithm.
I was wondering if you would have any advice to improve ...
- 273
2
votes
1
answer
58
views
Scraping webpage elements using Python
I have been trying to improve my knowledge with Python and I think the code is pretty forward. However I do dislike abit the coding style I have done where I use too much try except in a content there ...
- 503
3
votes
0
answers
90
views
Python REST API Data Extraction with requests library
Is there a better way to write this piece of code to extract data from a REST API with the requests library? It currently takes about 10 minutes to finish extracting the data from the REST API. I ...
- 31
2
votes
1
answer
74
views
Beautifulsoup scrape posts that has the word python in it with no python tag
This is my first time web scraping, and here is the code I whipped up:
...
- 964
2
votes
1
answer
159
views
Do I need to use inheritance and classes for my OOP webscraper? [closed]
I am currently writing python code that scrapes information from the web.
I have to scrape several sites, but there a two types of procedures:
Directly scrape from the website
Download pdf and scrape ...
- 41
5
votes
2
answers
282
views
Python Web scraping
I am 12 days old into Python and web scraping and managed to write my first ever automation script. Please review my code and point out blunders If any.
What do I want to achieve?
I want to scrape ...
1
vote
0
answers
26
views
Trouble returning web scraping output as python dictionary [closed]
So I am attempting to scrape a website of its staff roster (https://www.milb.com/greenville/ballpark/frontoffice) and I want the end product to be a dictionary in the format of {staff: position}. I am ...
- 11
3
votes
0
answers
498
views
Python Async Request using Tor and User-Agent Rotation
The code below will scrape data from three APIs. The APIs are for property listings. So, for each listing, I will have one request. There will be around 20000 requests. That's why I use ayncio to make ...
- 131
5
votes
0
answers
70
views
Scraping a hiring website using python's requests and BeautifulSoup
I'm designing a scraping application using python, requests and BeautifulSoup4.
I decided to divide the logic into two classes:
Spider : gets the base url ...
- 151
1
vote
1
answer
261
views
Python program that gets my hackerrank code and scores using requests [closed]
I'm trying to extend my coding skills to other languages by doing projects. (learned Java in school) and since python is very popular right now thats what I ended up choosing for this project, though ...
- 11
8
votes
1
answer
134
views
Finding unimplemented tasks on Rosetta Code using Requests
This is a Find unimplemented tasks task from Rosetta Code:
Given the name of a language on Rosetta Code, find all tasks which are
not implemented in that language.
There is already a solution ...
- 1,913
6
votes
1
answer
2k
views
Web scraping Google Trends in Python
Question
Recently I have tried to create a web scraping program to get data from Google Trends. It uses an RSS feed to do so. My question is as follows:
How can I improve my code such that it is more ...
1
vote
1
answer
3k
views
Python requests downloading big files in slower speed than wget
This is a simple toy downloader using python's requests library.
I’ve monitored the download process is slower on an ethernet connected box. The realtime speed is ...
- 11
4
votes
1
answer
113
views
Python program that gets basic information on a Github user
I've created a basic Python CLI script, which takes a Github username, and returns some basic information associated with said username. I use the requests for HTTP requests (I'm new to this area) and ...
- 1,088
5
votes
1
answer
5k
views
Python scraping tables
I started practicing in web-scraping few days ago. I made this code to extract data from a wikipedia page. There are several tables that classify mountains based on their height. However there is a ...
4
votes
1
answer
366
views
Scraping web data with Python 3
Below is a portion of the code i have written to scrape the bikesales.com.au website for details of bikes for sales (The full code is here). This finds all the 'href' attributes on each search page ...
- 245
7
votes
2
answers
1k
views
Repeatable HTTP requests in Python
I have a simple python script that's repeating a HTTP response/request. I would like to know how I could have constructed this better, particularly if there's a way I could have incorporated the slice ...
- 1,631
3
votes
1
answer
948
views
PyCryptoScraper to scrape currency rates
I created a PyCryptoScraper script to scrape crypto-currencies from Coingecko. Now I want to know if you could give me some hints or improvements for my script, which works fine.
The whole code also ...
- 491
3
votes
1
answer
3k
views
Downloading files using Python-requests
I wrote a Python script to download files using multiple (source) IP addresses -- kindly suggest any improvements.
...
- 155
2
votes
1
answer
1k
views
Object-oriented URL loading and HTTP client with requests in Python
I have an API which I want to consume in Python 3.x using the requests module.
Right now, my solution is to create Requests objects for each URL like the ...
- 121
8
votes
1
answer
4k
views
Web Scraping with Python + asyncio
I've been working at speeding up my web scraping with the asyncio library. I have a working solution, but am unsure as to how pythonic it is or if I am properly ...
- 83
7
votes
1
answer
10k
views
Python auth using requests
I am trying to implement a custom ApiClient in Python, using requests.
The way it does authentication is by:
login(username, password) -> get back a token if valid, http error code if not
set the ...
- 215
5
votes
0
answers
349
views
Choose the Transport Layer Security protocol - requests python
I am using using the following code to choose the transport layer security protocol while issuing a request to server. Could someone please review it.
This is in reference to the following question:
...
- 151
5
votes
1
answer
4k
views
Python API client
I'm creating a Python API client, after thinking and checking some open source Python clients I created this class, and I'm looking for any feedback (design, best practices...), this is an example URL ...
- 153
8
votes
1
answer
203
views
Timed requests and parsing JSON in Python
I wrote this tiny script to pull the JSON feed from the CitiBike website:
...
- 337