Real time metal prices and conversion fro USD to MXN using BeautifulSoup updated version

Question

from bs4 import BeautifulSoup
import requests
import re
import datetime

metal_translation = {"Aluminium": "Aluminio", "Copper": "Cobre", "Zinc": "Zinc", "Nickel": "Níquel", "Lead": "Plomo", "Tin": "Estaño",
                     "Aluminium Alloy": "Aleación de Aluminio", "Cobalt": "Cobalto", "Gold*": "Oro*", "Silver*": "Plata*",
                     "Steel Scrap**": "Chatarra de Acero", "NASAAC": "NASAAC", "Steel Rebar**": "Varilla de Acero"}

def get_metal_values():
    names = []
    prices = []
    response = requests.get('https://www.lme.com/')
    response.raise_for_status()
    metals = response.text
    soup = BeautifulSoup(metals, 'lxml')
    metal_table = soup.find("table", attrs={"class": "ring-times"})
    metal_table_names, metal_table_prices = metal_table.tbody.find_all("th"), metal_table.tbody.find_all("td")
    for name in metal_table_names:
        names.append(name.text.replace("LME ", ""))
    for price in metal_table_prices:
        prices.append(price.text.strip())
    return names, prices

def get_peso_conversion():
    response = requests.get('https://themoneyconverter.com/USD/MXN')
    response.raise_for_status()
    peso = response.text
    soup1 = BeautifulSoup(peso, 'lxml')
    conversion = soup1.find("div", class_="cc-result").text
    rate = re.search("\d{2}\.\d{4}", conversion).group()
    return rate

def get_time():
    d = datetime.datetime.now()
    date_time = d.strftime("%d/%m/%Y | %H:%M:%S")
    return date_time

def convert_values():
    names, prices = get_metal_values()
    rate = get_peso_conversion()
    for k, v in zip(names, prices):
        v = (float(v.replace(",", "")) * float(rate))
        print(f'{metal_translation[k]}: {v:.2f} $')

def program_run():
    print("Metal Prices by ETHAN HETRICK")
    print("================================================")
    print(f'{get_time()} | 1 USD = {get_peso_conversion()} MXN\n')
    print("Precios de metales de London Metal Exchange: (Por tonelada métrica, *Por onza Troy)\n")
    convert_values()
    print("================================================")

program_run()

Example Output:

input("\nEscribe 'x' para terminar.\n")

Metal Prices by ETHAN HETRICK
================================================
26/06/2020 | 21:59:00 | 1 USD = 23.0622 MXN

Precios de metales de London Metal Exchange: (Por tonelada métrica, *Por onza Troy)

Aluminio: 36484.40 $
Cobre: 138038.80 $
Zinc: 47438.95 $
Níquel: 293097.50 $
Plomo: 41004.59 $
Estaño: 391826.78 $
Aleación de Aluminio: 27997.51 $
NASAAC: 27444.02 $
Cobalto: 657272.70 $
Oro*: 40711.70 $
Plata*: 410.74 $
Chatarra de Acero: 6065.36 $
Varilla de Acero: 9720.72 $
================================================

Escribe 'x' para terminar.

This is an updated version of my code from a previous post of mine thanks to @Reinderien

My fiances father owns a metal recycling business in Mexico City, Mexico which requires him to do a lot of calculations when negotiating with clients. Since he was doing this by hand, I decided to automate this for him. I used London Metal Exchange, which is his preferred site to get the current prices, for the metal prices and also got the exchange rate for USD to MXN and applied that as well. I also needed to translate everything to spanish so I used a dictionary. This is my first time utilizing webscraping and the datetime module so any advice to make this program run more efficiently or make the code more concise is much appreciated!

Answer 1

You can tidy up the code a bit by directly returning results from functions and not using temporary variables you use only once.

I would also do the conversion to floats directly in the functions.

BeautifulSoup can directly work on the binary response.content, no need to decode it into a string yourself by using response.text.

def get_metal_values():
    response = requests.get('https://www.lme.com/')
    response.raise_for_status()
    soup = BeautifulSoup(response.content, 'lxml')
    metal_table = soup.find("table", attrs={"class": "ring-times"})
    names = [name.text.replace("LME ", "").rstrip("*")
             for name in metal_table.tbody.find_all("th")]
    prices = [float(price.text.strip().replace(",", ""))
              for price in metal_table.tbody.find_all("td")]
    return zip(names, prices)

def get_peso_conversion():
    response = requests.get('https://themoneyconverter.com/USD/MXN')
    response.raise_for_status()
    soup = BeautifulSoup(response.content, 'lxml')
    conversion = soup.find("div", class_="cc-result").text
    return float(re.search("\d{2}\.\d{4}", conversion).group())

def get_time():
    return datetime.datetime.now().strftime("%d/%m/%Y | %H:%M:%S")

def convert_values():
    exchange_rate = get_peso_conversion()
    for name, price in get_metal_values():
        print(f'{metal_translation[name]}: {price*exchange_rate:.2f} $')

Note that I also stripped the trailing "*" in the names as I didn't see a reason to keep them.

Answer 2

I think it would be better to handle the API stuff to convert prices using a Python package like CurrencyConverter.

The site that you are using currently might be down some day or may blacklist your IP because of the many requests that your code is making to convert the prices.

Also, your code seems to be better converted to a Scrapy project if you are interested in web scraping. Scrapy Tutorial by ScrappingHub