3
\$\begingroup\$
from bs4 import BeautifulSoup
import requests
import re
import datetime

metal_translation = {"Aluminium": "Aluminio", "Copper": "Cobre", "Zinc": "Zinc", "Nickel": "Níquel", "Lead": "Plomo", "Tin": "Estaño",
                     "Aluminium Alloy": "Aleación de Aluminio", "Cobalt": "Cobalto", "Gold*": "Oro*", "Silver*": "Plata*",
                     "Steel Scrap**": "Chatarra de Acero", "NASAAC": "NASAAC", "Steel Rebar**": "Varilla de Acero"}

def get_metal_values():
    names = []
    prices = []
    response = requests.get('https://www.lme.com/')
    response.raise_for_status()
    metals = response.text
    soup = BeautifulSoup(metals, 'lxml')
    metal_table = soup.find("table", attrs={"class": "ring-times"})
    metal_table_names, metal_table_prices = metal_table.tbody.find_all("th"), metal_table.tbody.find_all("td")
    for name in metal_table_names:
        names.append(name.text.replace("LME ", ""))
    for price in metal_table_prices:
        prices.append(price.text.strip())
    return names, prices

def get_peso_conversion():
    response = requests.get('https://themoneyconverter.com/USD/MXN')
    response.raise_for_status()
    peso = response.text
    soup1 = BeautifulSoup(peso, 'lxml')
    conversion = soup1.find("div", class_="cc-result").text
    rate = re.search("\d{2}\.\d{4}", conversion).group()
    return rate

def get_time():
    d = datetime.datetime.now()
    date_time = d.strftime("%d/%m/%Y | %H:%M:%S")
    return date_time

def convert_values():
    names, prices = get_metal_values()
    rate = get_peso_conversion()
    for k, v in zip(names, prices):
        v = (float(v.replace(",", "")) * float(rate))
        print(f'{metal_translation[k]}: {v:.2f} $')

def program_run():
    print("Metal Prices by ETHAN HETRICK")
    print("================================================")
    print(f'{get_time()} | 1 USD = {get_peso_conversion()} MXN\n')
    print("Precios de metales de London Metal Exchange: (Por tonelada métrica, *Por onza Troy)\n")
    convert_values()
    print("================================================")

program_run()

Example Output:

input("\nEscribe 'x' para terminar.\n")

Metal Prices by ETHAN HETRICK
================================================
26/06/2020 | 21:59:00 | 1 USD = 23.0622 MXN

Precios de metales de London Metal Exchange: (Por tonelada métrica, *Por onza Troy)

Aluminio: 36484.40 $
Cobre: 138038.80 $
Zinc: 47438.95 $
Níquel: 293097.50 $
Plomo: 41004.59 $
Estaño: 391826.78 $
Aleación de Aluminio: 27997.51 $
NASAAC: 27444.02 $
Cobalto: 657272.70 $
Oro*: 40711.70 $
Plata*: 410.74 $
Chatarra de Acero: 6065.36 $
Varilla de Acero: 9720.72 $
================================================

Escribe 'x' para terminar.

This is an updated version of my code from a previous post of mine thanks to @Reinderien

My fiances father owns a metal recycling business in Mexico City, Mexico which requires him to do a lot of calculations when negotiating with clients. Since he was doing this by hand, I decided to automate this for him. I used London Metal Exchange, which is his preferred site to get the current prices, for the metal prices and also got the exchange rate for USD to MXN and applied that as well. I also needed to translate everything to spanish so I used a dictionary. This is my first time utilizing webscraping and the datetime module so any advice to make this program run more efficiently or make the code more concise is much appreciated!

\$\endgroup\$
2
\$\begingroup\$

You can tidy up the code a bit by directly returning results from functions and not using temporary variables you use only once.

I would also do the conversion to floats directly in the functions.

BeautifulSoup can directly work on the binary response.content, no need to decode it into a string yourself by using response.text.

def get_metal_values():
    response = requests.get('https://www.lme.com/')
    response.raise_for_status()
    soup = BeautifulSoup(response.content, 'lxml')
    metal_table = soup.find("table", attrs={"class": "ring-times"})
    names = [name.text.replace("LME ", "").rstrip("*")
             for name in metal_table.tbody.find_all("th")]
    prices = [float(price.text.strip().replace(",", ""))
              for price in metal_table.tbody.find_all("td")]
    return zip(names, prices)

def get_peso_conversion():
    response = requests.get('https://themoneyconverter.com/USD/MXN')
    response.raise_for_status()
    soup = BeautifulSoup(response.content, 'lxml')
    conversion = soup.find("div", class_="cc-result").text
    return float(re.search("\d{2}\.\d{4}", conversion).group())

def get_time():
    return datetime.datetime.now().strftime("%d/%m/%Y | %H:%M:%S")

def convert_values():
    exchange_rate = get_peso_conversion()
    for name, price in get_metal_values():
        print(f'{metal_translation[name]}: {price*exchange_rate:.2f} $')

Note that I also stripped the trailing "*" in the names as I didn't see a reason to keep them.

| improve this answer | |
\$\endgroup\$
  • \$\begingroup\$ I actually used the * to signify a change in units in the end result so I kept that. I like how you got rid of a lot of unnecessary variables I was storing! I usually don't put a function in return but I will start doing that thanks. \$\endgroup\$ – Ethan Hetrick Jun 27 at 16:49
  • 1
    \$\begingroup\$ @EthanHetrick Yeah, I also like function calls in return. But it is always a balancing act, you don't want too many unnecessary temporary variables around, but also not make it one unreadable line. I think here it works fine like this. \$\endgroup\$ – Graipher Jun 27 at 16:52
  • \$\begingroup\$ would you know how to get the numbers to show in a more readable format? For example, something that costs 60 thousand dollars and 99 cents in the Spanish-speaking world might be written $60.000,99. Is there a way I can accomplish this? \$\endgroup\$ – Ethan Hetrick Jun 27 at 17:05
  • \$\begingroup\$ @EthanHetrick "$ {:,.2f}" should do the trick. Note the , before the .. \$\endgroup\$ – Graipher Jun 27 at 17:09
  • 1
    \$\begingroup\$ @VisheshMangla Whilst I'm not a fan of 'aligned with opening delimiter' style the style of the comprehensions are fine. The comprehension fits nicely on the two lines and is really easy to read. (Even for someone that despises the indent style). Your suggested edit however made the code longer, harder to read and generally goes completely against the the founding idea of comprehensions because of these things. \$\endgroup\$ – Peilonrayz Jun 28 at 8:47
3
\$\begingroup\$

I think it would be better to handle the API stuff to convert prices using a Python package like CurrencyConverter.

The site that you are using currently might be down some day or may blacklist your IP because of the many requests that your code is making to convert the prices.

Also, your code seems to be better converted to a Scrapy project if you are interested in web scraping. Scrapy Tutorial by ScrappingHub

| improve this answer | |
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.