9

Check for failure metals = requests.get('https://www.lme.com/').text should be response = requests.get('https://www.lme.com/') response.raise_for_status() metals = response.text String literals don't need parens date = datetime.datetime.now() time = (f'{date.day}/{date.month}/{date.year} | {date.hour}:{date.minute}') would only need parens if it spans ...


7

Some style issues first: Package names should be all lowercase ASCII letters. No camelCase, PascalCase, snake_case or kebab-case. So tennisproject and scanner. Local variables should never be uppercase SNAKE_CASE, but camelCase. So atpUrlPrefix instead of ATP_URL_PREFIX and so on. You probably want those to be class constants anyways, which use uppercase ...


3

Type hints def create_signature(username, password): can be def create_signature(username: str, password: str) -> bytes: Quote style consistency Pick single or double: 'Authorization' : "Basic " Connection string security Please (please) do not store a password as a hard-coded source global: SERVER = 'localhost' USER = 'root' PASSWORD = '' ...


3

Requirements Move these: # pip install requests # pip install beautifulsoup4 into a pip-compatible requirements.txt: requests beautifulsoup4 Type hints They will help. I don't know if id_ is an int or str here, but: def get_url(id_: int) -> str: Path parsing This: l = url.split("/") return l[4] if len(l) > 4 else "" is a little ...


3

I think it would be better to handle the API stuff to convert prices using a Python package like CurrencyConverter. The site that you are using currently might be down some day or may blacklist your IP because of the many requests that your code is making to convert the prices. Also, your code seems to be better converted to a Scrapy project if you are ...


2

You can tidy up the code a bit by directly returning results from functions and not using temporary variables you use only once. I would also do the conversion to floats directly in the functions. BeautifulSoup can directly work on the binary response.content, no need to decode it into a string yourself by using response.text. def get_metal_values(): ...


2

I see you used your custom ScraperException exception in two different modes: the first is below: private static List<String> noEmptyElseThrow(final List<String> weeks) throws ScraperException{ if (weeks.isEmpty()) { throw new ScraperException("Please provide a historical time range! Cannot rank otherwise!"); } else { ...


2

Global nomenclature urlHalf should be URL_HALF. That said, a better name would be DOMAIN, since that's what that string actually is. Further, after declaring this you don't use it correctly; these: base_url = "https://codingbat.com" login_url = "https://codingbat.com/login" java_url = "https://codingbat.com/java/" could be ...


1

As a suggestion it is good to always maintain configuration outside the project and import it in the code and use it. And separating the similar functionality into individual files is also a good practice for a cleaner project structure.


Only top voted, non community-wiki answers of a minimum length are eligible