Scraping current day Counter-Strike match results from a website

Question

As a fan of competitive Counter-Strike, I like to keep up with who is currently winning and who is losing. There is a website that provides me with just that. I thought it would be cool if I could display the results in a way different than visiting the website.

Here is an example result from the source of the website.

<div class="result-con" data-zonedgrouping-entry-unix="1499867767000"><a href="/matches/2312610/penta-vs-kinguin-esea-premier-season-25-europe" class="a-reset">
                        <div class="result">
                          <table>
                            <tr>
                              <td class="team-cell">
                                <div class="line-align team1">
                                  <div class="team ">PENTA</div>
<img alt="PENTA" src="https://static.hltv.org/images/team/logo/5395" class="team-logo" title="PENTA"></div>
                              </td>
                              <td class="result-score"><span class="score-lost">5</span> - <span class="score-won">16</span></td>
                              <td class="team-cell">
                                <div class="line-align team2"><img alt="Kinguin" src="https://static.hltv.org/images/team/logo/6134" class="team-logo" title="Kinguin">
                                  <div class="team bold">Kinguin</div>
                                </div>
                              </td>
                              <td class="event"><img alt="ESEA Premier Season 25 Europe" src="https://static.hltv.org/images/eventLogos/2879.png" class="event-logo smartphone-only" title="ESEA Premier Season 25 Europe"><span class="event-name">ESEA Premier Season 25 Europe</span></td>
                              <td class="star-cell">
                                <div class="map-and-stars">
                                  <div class="stars"><i class="fa fa-star star"></i></div>
                                  <div class="map map-text">trn</div>
                                </div>
                              </td>
                            </tr>
                          </table>
                        </div>
                      </a></div>

I haven't done anything relating to web scraping before, so this is my first take at it. The code does all that I want it to, however looking at it gives me headaches. It's not pretty and most likely not very efficient either. I'd appreciate constructive feedback on it.

I am aware that the code is not reusable and I should probably refactor it, but I want to learn more efficient scraping methods first.

Each match result contains a timestamp, which refers to the date the game was played. In the for loop, I compare it to today's date and if they match, the match information is assigned to appropriate variables.

#!/usr/bin/python3
from time import localtime, strftime

import requests
from bs4 import BeautifulSoup

RESULTS_PAGE_URL = 'https://www.hltv.org/results?stars=1'
current_date = strftime('%B %d %Y')

html_source = requests.get(RESULTS_PAGE_URL).text
soup = BeautifulSoup(html_source, 'lxml')

print(soup.title.text)
print('Results for {}\n'.format(current_date))

for result in soup.find_all('div', class_='result-con'):
    timestamp = int(result['data-zonedgrouping-entry-unix']) / 1000
    match_date = strftime('%B %d %Y', localtime(timestamp))
    if match_date == current_date:
        losing_team = result.find('div', class_='team ').text
        losing_team_score = result.find('span', class_='score-lost').text

        winning_team = result.find('div', class_='team bold').text
        winning_team_score = result.find('span', class_='score-won').text

        event = result.find('span', class_='event-name').text

        print('{} {} - {} {}'.format(winning_team, winning_team_score, losing_team_score, losing_team))
        print('{}\n'.format(event))

Answer 1

The code is pretty clean, great job.

Some of the things I would point out:

• there is a way to find only the current date games directly - without filtering them out in the loop. We can incapsulate the filtering logic in a searching function:

  def current_games(tag):
      is_result_tag = tag.name == 'div' and 'result-con' in tag.get('class', [])
      if not is_result_tag:
          return False
  
      timestamp = int(tag['data-zonedgrouping-entry-unix']) / 1000
      return strftime('%B %d %Y', localtime(timestamp)) == current_date
  
  
  for result in soup(current_games):
  

  Note that soup(current_games) is a shorthand for soup.find_all(current_games).

• I personally prefer CSS selectors to find* methods

• .get_text() is generally more preferable than .text
• you can use f-strings if on Python 3.6

Modified code:

#!/usr/bin/python3
from time import localtime, strftime

import requests
from bs4 import BeautifulSoup


RESULTS_PAGE_URL = 'https://www.hltv.org/results?stars=1'
current_date = strftime('%B %d %Y')

html_source = requests.get(RESULTS_PAGE_URL).text
soup = BeautifulSoup(html_source, 'lxml')

print(soup.title.get_text())
print(f'Results for {current_date}\n')


def current_games(tag):
    is_result_tag = tag.name == 'div' and 'result-con' in tag.get('class', [])
    if not is_result_tag:
        return False

    timestamp = int(tag['data-zonedgrouping-entry-unix']) / 1000
    return strftime('%B %d %Y', localtime(timestamp)) == current_date


for result in soup(current_games):
    losing_team = result.select_one('.team').get_text()
    losing_team_score = result.select_one('.score-lost').get_text()

    winning_team = result.select_one('.team.bold').get_text()
    winning_team_score = result.select_one('.score-won').get_text()

    event = result.select_one('.event-name').get_text()

    print(f'{winning_team} {winning_team_score} - {losing_team_score} {losing_team}')
    print(f'{event}\n')

# fnatic will always be the best

A more generic solution that also handles ties might be:

for result in soup(current_games):
    teams = [team.get_text() for team in result.select(".team")]
    score = [item.get_text() for item in result.select("[class^=score]")]

    event = result.select_one('.event-name').get_text()

    print(f'{teams[0]} {score[0]} - {teams[1]} {score[1]}')
    print(f'{event}\n')