I have collected a .csv file with some statistics about football games in the following format. Here is a sample .csv file.


My code does the following:

  • Calculate a summary of statistics for the given subset of games,
  • Calculate a summary of statistics for the each individual team,
  • Filter games by date or by range of some stat, and
  • Print a summary as html or csv.

I have some questions about my code.

  1. How should I write unit tests for checking the correctness of functions that calculate stats?
  2. How to make a function that prints output to work with an arbitrary list of fields, instead of a particular one? Since I print a lot of fields, passing them one by one is tedious. Maybe I could create some common templates and pass one of them?
  3. Can I simplify calculate_team_stats()? Maybe it can be improved by using Counter() or some third party library.

Any or all other feedback is welcomed!

import csv
import datetime
from collections import namedtuple, defaultdict
from statistics import mean

FILENAME = 'epl 18_19 games.csv'

Game = namedtuple('Game', ['Date', 'Home', 'Away', 'HomeShots', 'AwayShots',
                           'HomeBT', 'AwayBT', 'HomeCrosses', 'AwayCrosses',
                           'HomeCorners', 'AwayCorners', 'HomeGoals',
                           'AwayGoals', 'HomeXG', 'AwayXG'])

def csv_to_list_of_games(filename=FILENAME):
    Makes a list of Game from a csv file.

    games = []
    with open(FILENAME) as f:
        csv_reader = csv.reader(f)
        for game in csv_reader:
            date = game[0].split('.')
            year = int(date[2])
            month = int(date[1])
            day = int(date[0])
            date_object = datetime.date(year, month, day)
            games.append(Game(date_object, *game[1:]))

    return games

def get_teams_list(games):
    Makes a list of teams in the given list of games.

    return list(set([game.Home for game in games] + [game.Away for game in games]))

def get_games_by_team(teamname, games):
    Returns a list of Game featuring the given team.

    return [game for game in games if game.Home == teamname or game.Away == teamname]

def calculate_team_stats(teams, games):
    Calculates team stats for each team in the list.

    team_stats = dict()
    for team in teams:
        team_stats[team] = defaultdict(int)
        team_stats[team]['HomeShotsFor'] = sum(int(game.HomeShots) for game in games if game.Home == team)
        team_stats[team]['HomeShotsAgainst'] = sum(int(game.AwayShots) for game in games if game.Home == team)
        team_stats[team]['HomeBoxTouchesFor'] = sum(int(game.HomeBT) for game in games if game.Home == team)
        team_stats[team]['HomeBoxTouchesAgainst'] = sum(int(game.AwayBT) for game in games if game.Home == team)
        team_stats[team]['HomeCrossesFor'] = sum(int(game.HomeCrosses) for game in games if game.Home == team)
        team_stats[team]['HomeCrossesAgainst'] = sum(int(game.AwayCrosses) for game in games if game.Home == team)
        team_stats[team]['HomeCornersFor'] = sum(int(game.HomeCorners) for game in games if game.Home == team)
        team_stats[team]['HomeCornersAgainst'] = sum(int(game.AwayCorners) for game in games if game.Home == team)
        team_stats[team]['HomeGoalsFor'] = sum(int(game.HomeGoals) for game in games if game.Home == team)
        team_stats[team]['HomeGoalsAgainst'] = sum(int(game.AwayGoals) for game in games if game.Home == team)
        team_stats[team]['HomeXGoalsFor'] = sum(float(game.HomeXG) for game in games if game.Home == team)
        team_stats[team]['HomeXGoalsAgainst'] = sum(float(game.AwayXG) for game in games if game.Home == team)
        team_stats[team]['HomeGames'] = sum(1 for game in games if game.Home == team)

        team_stats[team]['AwayShotsFor'] = sum(int(game.AwayShots) for game in games if game.Away == team)
        team_stats[team]['AwayShotsAgainst'] = sum(int(game.HomeShots) for game in games if game.Away == team)
        team_stats[team]['AwayBoxTouchesFor'] = sum(int(game.AwayBT) for game in games if game.Away == team)
        team_stats[team]['AwayBoxTouchesAgainst'] = sum(int(game.HomeBT) for game in games if game.Away == team)
        team_stats[team]['AwayCrossesFor'] = sum(int(game.AwayCrosses) for game in games if game.Away == team)
        team_stats[team]['AwayCrossesAgainst'] = sum(int(game.HomeCrosses) for game in games if game.Away == team)
        team_stats[team]['AwayCornersFor'] = sum(int(game.AwayCorners) for game in games if game.Away == team)
        team_stats[team]['AwayCornersAgainst'] = sum(int(game.HomeCorners) for game in games if game.Away == team)
        team_stats[team]['AwayGoalsFor'] = sum(int(game.AwayGoals) for game in games if game.Away == team)
        team_stats[team]['AwayGoalsAgainst'] = sum(int(game.HomeGoals) for game in games if game.Away == team)
        team_stats[team]['AwayXGoalsFor'] = sum(float(game.AwayXG) for game in games if game.Away == team)
        team_stats[team]['AwayXGoalsAgainst'] = sum(float(game.HomeXG) for game in games if game.Away == team)
        team_stats[team]['AwayGames'] = sum(1 for game in games if game.Away == team)

        team_stats[team]['ShotsFor'] += team_stats[team]['HomeShotsFor'] + team_stats[team]['AwayShotsFor']
        team_stats[team]['ShotsAgainst'] += team_stats[team]['HomeShotsAgainst'] + team_stats[team]['AwayShotsAgainst']
        team_stats[team]['CrossesFor'] += team_stats[team]['HomeCrossesFor'] + team_stats[team]['AwayCrossesFor']
        team_stats[team]['CrossesAgainst'] += team_stats[team]['HomeCrossesAgainst'] + team_stats[team]['AwayCrossesAgainst']
        team_stats[team]['BoxTouchesFor'] += team_stats[team]['HomeBoxTouchesFor'] + team_stats[team]['AwayBoxTouchesFor']
        team_stats[team]['BoxTouchesAgainst'] += team_stats[team]['HomeBoxTouchesAgainst'] + team_stats[team]['AwayBoxTouchesAgainst']
        team_stats[team]['CornersFor'] += team_stats[team]['HomeCornersFor'] + team_stats[team]['AwayCornersFor']
        team_stats[team]['CornersAgainst'] += team_stats[team]['HomeCornersAgainst'] + team_stats[team]['AwayCornersAgainst']

        team_stats[team]['GoalsFor'] += team_stats[team]['HomeGoalsFor'] + team_stats[team]['AwayGoalsFor']
        team_stats[team]['GoalsAgainst'] += team_stats[team]['HomeGoalsAgainst'] + team_stats[team]['AwayGoalsAgainst']
        team_stats[team]['XGoalsFor'] += team_stats[team]['HomeXGoalsFor'] + team_stats[team]['AwayXGoalsFor']
        team_stats[team]['XGoalsAgainst'] += team_stats[team]['HomeXGoalsAgainst'] + team_stats[team]['AwayXGoalsAgainst']
        team_stats[team]['Games'] += team_stats[team]['HomeGames'] + team_stats[team]['AwayGames']

        team_stats[team]['HomeShotsRatio'] = team_stats[team]['HomeShotsFor'] / (team_stats[team]['HomeShotsFor'] + team_stats[team]['HomeShotsAgainst'])
        team_stats[team]['AwayShotsRatio'] = team_stats[team]['AwayShotsFor'] / (team_stats[team]['AwayShotsFor'] + team_stats[team]['AwayShotsAgainst'])
        team_stats[team]['ShotsRatio'] = team_stats[team]['ShotsFor'] / (team_stats[team]['ShotsFor'] + team_stats[team]['ShotsAgainst'])

        team_stats[team]['HomeCrossesRatio'] = team_stats[team]['HomeCrossesFor'] / (team_stats[team]['HomeCrossesFor'] + team_stats[team]['HomeCrossesAgainst'])
        team_stats[team]['AwayCrossesRatio'] = team_stats[team]['AwayCrossesFor'] / (team_stats[team]['AwayCrossesFor'] + team_stats[team]['AwayCrossesAgainst'])
        team_stats[team]['CrossesRatio'] = team_stats[team]['CrossesFor'] / (team_stats[team]['CrossesFor'] + team_stats[team]['CrossesAgainst'])

        team_stats[team]['HomeBoxTouchesRatio'] = team_stats[team]['HomeBoxTouchesFor'] / (team_stats[team]['HomeBoxTouchesFor'] + team_stats[team]['HomeBoxTouchesAgainst'])
        team_stats[team]['AwayBoxTouchesRatio'] = team_stats[team]['AwayBoxTouchesFor'] / (team_stats[team]['AwayBoxTouchesFor'] + team_stats[team]['AwayBoxTouchesAgainst'])
        team_stats[team]['BoxTouchesRatio'] = team_stats[team]['BoxTouchesFor'] / (team_stats[team]['BoxTouchesFor'] + team_stats[team]['BoxTouchesAgainst'])

        team_stats[team]['HomeCornersRatio'] = team_stats[team]['HomeCornersFor'] / (team_stats[team]['HomeCornersFor'] + team_stats[team]['HomeCornersAgainst'])
        team_stats[team]['AwayCornersRatio'] = team_stats[team]['AwayCornersFor'] / (team_stats[team]['AwayCornersFor'] + team_stats[team]['AwayCornersAgainst'])
        team_stats[team]['CornersRatio'] = team_stats[team]['CornersFor'] / (team_stats[team]['CornersFor'] + team_stats[team]['CornersAgainst'])

        team_stats[team]['HomeGoalsRatio'] = team_stats[team]['HomeGoalsFor'] / (team_stats[team]['HomeGoalsFor'] + team_stats[team]['HomeGoalsAgainst'])
        team_stats[team]['AwayGoalsRatio'] = team_stats[team]['AwayGoalsFor'] / (team_stats[team]['AwayGoalsFor'] + team_stats[team]['AwayGoalsAgainst'])
        team_stats[team]['GoalsRatio'] = team_stats[team]['GoalsFor'] / (team_stats[team]['GoalsFor'] + team_stats[team]['GoalsAgainst'])

        team_stats[team]['HomeXGoalsRatio'] = team_stats[team]['HomeXGoalsFor'] / (team_stats[team]['HomeXGoalsFor'] + team_stats[team]['HomeXGoalsAgainst'])
        team_stats[team]['AwayXGoalsRatio'] = team_stats[team]['AwayXGoalsFor'] / (team_stats[team]['AwayXGoalsFor'] + team_stats[team]['AwayXGoalsAgainst'])
        team_stats[team]['XGoalsRatio'] = team_stats[team]['XGoalsFor'] / (team_stats[team]['XGoalsFor'] + team_stats[team]['XGoalsAgainst'])

        team_stats[team]['CornersTotalPg'] = (team_stats[team]['CornersFor'] + team_stats[team]['CornersAgainst']) / team_stats[team]['Games']

        team_stats[team]['HomeBoxTouchesTotal'] = (team_stats[team]['HomeBoxTouchesFor'] + team_stats[team]['HomeBoxTouchesAgainst'])
        team_stats[team]['AwayBoxTouchesTotal'] = (team_stats[team]['AwayBoxTouchesFor'] + team_stats[team]['AwayBoxTouchesAgainst'])

        team_stats[team]['HomeBoxTouchesTotalPg'] = team_stats[team]['HomeBoxTouchesTotal'] / team_stats[team]['HomeGames']
        team_stats[team]['AwayBoxTouchesTotalPg'] = team_stats[team]['AwayBoxTouchesTotal'] / team_stats[team]['AwayGames']

        team_stats[team]['BoxTouchesTotalPg'] = (team_stats[team]['HomeBoxTouchesTotal'] + team_stats[team]['AwayBoxTouchesTotal']) / team_stats[team]['Games']

    return team_stats

def print_team_stats_html(team_stats):
    Prints a subset of team stats in HTML format.

    headers = ['Team', 'HomeBoxTouchesRatio', 'AwayBoxTouchesRatio',
               'HomeBoxTouchesTotalPg', 'AwayBoxTouchesTotalPg',
               'HomeCornersRatio', 'AwayCornersRatio']

    print('<table border=1>')
    print('<tr>', end='')

    for header in headers:
        print('<th>{}</th>'.format(header), end='')

    for key, value in sorted(team_stats.items()):

def find_games_by_teams_stats(home_stat, away_stat, home_value, away_value, teams_stats, games, home_epsilon=0.05, away_epsilon=0.05):
    Finds teams with home and away stat <= EPSILON <= and returns a list of games between those teams.

    relevant_home_teams = []
    relevant_away_teams = []

    for team in teams_stats:
        if abs(teams_stats[team][home_stat] - home_value) <= home_epsilon:
        if abs(teams_stats[team][away_stat] - away_value) <= away_epsilon:

    return [game for game in games if game.Home in relevant_home_teams and game.Away in relevant_away_teams]

def calculate_sample_stats(games):
    Calculates summary statistics for the given list of Game.

    avg_home_corners = mean(int(game.HomeCorners) for game in games)
    avg_away_corners = mean(int(game.AwayCorners) for game in games)
    avg_home_bt = mean(int(game.HomeBT) for game in games)
    avg_away_bt = mean(int(game.AwayBT) for game in games)
    avg_home_goals = mean(int(game.HomeGoals) for game in games)
    avg_away_goals = mean(int(game.AwayGoals) for game in games)
    avg_home_xgoals = mean(float(game.HomeXG) for game in games)
    avg_away_xgoals = mean(float(game.AwayXG) for game in games)
    avg_home_bt_ratio = avg_home_bt / (avg_home_bt + avg_away_bt)
    avg_away_bt_ratio = avg_away_bt / (avg_home_bt + avg_away_bt)

    stats = {
        'games_count': len(games),
        'avg_home_corners': avg_home_corners,
        'avg_away_corners': avg_away_corners,
        'avg_home_bt': avg_home_bt,
        'avg_away_bt': avg_away_bt,
        'avg_home_goals': avg_home_goals,
        'avg_away_goals': avg_away_goals,
        'avg_home_xgoals': avg_home_xgoals,
        'avg_away_xgoals': avg_away_xgoals,
        'avg_home_bt_ratio': avg_home_bt_ratio,
        'avg_away_bt_ratio': avg_away_bt_ratio,

    return stats

def print_sample_stats(stats):
    Prints the statistical summary of the list of Game.

    print(f'{stats["games_count"]} games have been found')
    print(f'Average home corners: {stats["avg_home_corners"]:.2f}')
    print(f'Average away corners: {stats["avg_away_corners"]:.2f}')
    print(f'Average home BoxTouches: {stats["avg_home_bt"]:.2f}')
    print(f'Average away BoxTouches: {stats["avg_away_bt"]:.2f}')
    print(f'Average home Goals: {stats["avg_home_goals"]:.2f}')
    print(f'Average away Goals: {stats["avg_away_goals"]:.2f}')
    print(f'Average home Xgoals: {stats["avg_home_xgoals"]:.2f}')
    print(f'Average away Xgoals: {stats["avg_away_xgoals"]:.2f}')
    print(f'Average home BoxTouches ratio: {stats["avg_home_bt_ratio"]:.3f}')
    print(f'Average away BoxTouches ratio: {stats["avg_away_bt_ratio"]:.3f}')

if __name__ == '__main__':
    games = csv_to_list_of_games(FILENAME)
    teams = get_teams_list(games)
    team_stats = calculate_team_stats(teams, games)
    relevant_games = find_games_by_teams_stats('HomeBoxTouchesRatio', 'AwayBoxTouchesRatio', 0.55, 0.45, team_stats, games, 0.03, 0.03)
    relevant_stats = calculate_sample_stats(relevant_games)
    print(set(game.Home for game in relevant_games))
    print(set(game.Away for game in relevant_games))
  • \$\begingroup\$ Hey Peilonrayz, I will be happy to get any feedback! Feel free to answer only a particular question(s). \$\endgroup\$ Jun 6, 2020 at 22:54
  • \$\begingroup\$ Ah perfect :) When going over your question I got a little confused by your second question. Could you help me figure it out? Could you possibly rephrase it. It will likely help me understand if you relate it to the code, is there a specific function or group of functions that this problem is in? Sorry to be a bother :( \$\endgroup\$
    – Peilonrayz
    Jun 6, 2020 at 23:04
  • \$\begingroup\$ Sure. Please take a look at print_team_stats_html(), even though team_stats has tens of fields, the function prints only 7 of them and there is no way to specify which ones to print. I would like to parameterize it, however if to list all the fields ony be one, it would be extremely tedious. So my idea is to create some subsets of fields to print at the top-level and pass one of them as a parameter to specify which fields I would like to print. \$\endgroup\$ Jun 6, 2020 at 23:08
  • \$\begingroup\$ For example, one template prints only home stats, whereas another one prints only stats related to scored goals, etc. \$\endgroup\$ Jun 6, 2020 at 23:10
  • 1
    \$\begingroup\$ You may take a look at pandas. If you haven't heard of it that this may be a perfect oppurtunity to learn the library. It has very great tools for handling csv,xlsx,etc types of data and it works better with jupyter notebook. \$\endgroup\$ Jun 7, 2020 at 2:31

2 Answers 2


You're already doing a lot of things well; it's nice to see a question from someone who already knows the language and is looking for ways to get better.

  1. How should I write unit tests for checking the correctness of functions that calculate stats?
    Generally you'll want one (or a few) "happy path" unit tests: hard-code some sample data as part of the unit test, and assert that the result of the calculations is whatever you've confirmed it ought to be.
    You'll also want a couple failure tests, that check that your program fails when it ought to fail, for example if given malformed data.
  2. How to make a function that prints output to work with an arbitrary list of fields, instead of a particular one?
    You're thinking of fields as strings. Sometimes you need that, but you'd also be well served by thinking of fields as functions from a defined data structure to a contained datum or sub-structure. A dict would be appropriate for converting from fields-as-names to fields-as-getters. Then you can just loop or use a comprehension or whatever.
  3. Can I simplify calculate_team_stats()?
    Yes; the reason it's so unweildly now is because you're using a flat data structure, and you're relying too much on dicts. Dicts aren't great for structured data because they have very little structure. When you know the structure in advance, a tree of NamedTuples is often better.

Other stuff:

  • You're ready for typing. Type-hinted code is easier to reason about, both for you and for your IDE. I also recommend using mypy in parallel with your linter and unit tests to ensure your types are correct.
  • A function with type-hints will need fewer comments (often none) to be readable.
  • Break up your functions even smaller. For example making a Game from a csv row should be its own function, and then csv_to_list_of_games is quite short.
  • More generally, nest stuff more. This includes classes.
  • Rely even more on comprehensions.
  • Use a DictReader to parse the csv file, that way you're not relying on the order of the fields.
  • When a function takes a lot of args, try to avoid letting the order matter by passing them as keyword args.
  • Rely more on the libraries you're using, for example let datetime handle parsing for you.
  • We use lists a lot because they're flexible, but if a more constrained structure will do then use that. For example if you're going to get a set of teams, why turn it back into a list?
  • Do your data conversions when you parse the data, not later when you use it.

I mocked out the parse-and-calculate half, and checked it with mypy. I didn't actually test it or attempt the filter-and-print half:

import csv
import datetime
import itertools
from statistics import mean
from typing import Iterable, Mapping, NamedTuple, Set, Tuple

FILENAME = 'epl 18_19 games.csv'

class IntegerStats(NamedTuple):
    shots: int
    box_touches: int
    crosses: int
    corners: int
    goals: int
    x_goals: float

def sum_integer_stats(*stats: IntegerStats) -> IntegerStats:
    return IntegerStats(  # This could be one line, but let's keep it verbose. 
        shots=sum(s.shots for s in stats),
        box_touches=sum(s.box_touches for s in stats),
        crosses=sum(s.crosses for s in stats),
        corners=sum(s.corners for s in stats),
        goals=sum(s.goals for s in stats),
        x_goals=sum(s.x_goals for s in stats)

class RatioStats(NamedTuple):
    shots: float
    box_touches: float
    crosses: float
    corners: float
    goals: float
    x_goals: float

class Game(NamedTuple):
    date: datetime.date
    home_team: str
    home_stats: IntegerStats
    away_team: str
    away_stats: IntegerStats

    def teams(self) -> Tuple[str, str]:
        return (self.home_team, self.away_team)

def row_to_game(row: Mapping[str, str]) -> Game:
    return Game(
        date=datetime.datetime.strptime(row['Date'], '%d.%m.%Y').date(),

def csv_to_list_of_games(filename: str) -> Iterable[Game]:
    with open(FILENAME) as f:
        csv_reader = csv.DictReader(f)
        return [row_to_game(row) for row in csv_reader]

def get_teams_set(games: Iterable[Game]) -> Set[str]:
    return set(itertools.chain.from_iterable(game.teams() for game in games))

def get_games_by_team(teamname: str, games: Iterable[Game]) -> Iterable[Game]:
    return [game for game in games if teamname in game.teams()]

class TeamGameSetStats(NamedTuple):
    made: IntegerStats  # call it `made` because `for` is a python keyword.
    against: IntegerStats
    totals: IntegerStats
    ratios: RatioStats
    totals_per_game: RatioStats
    games: int

def team_gameset_stats(own_stats: Iterable[IntegerStats],
                       opposing_stats: Iterable[IntegerStats]
                       ) -> TeamGameSetStats:
    made = sum_integer_stats(*own_stats)
    against = sum_integer_stats(*opposing_stats)
    totals = sum_integer_stats(made, against)
    game_count = len(list(itertools.chain(own_stats, opposing_stats)))
    return TeamGameSetStats(
            shots=made.shots / (made.shots + against.shots),
            box_touches=made.box_touches / (made.box_touches + against.box_touches),
            crosses=made.crosses / (made.crosses + against.crosses),
            corners=made.corners / (made.corners + against.corners),
            goals=made.goals / (made.goals + against.goals),
            x_goals=made.x_goals / (made.x_goals + against.x_goals)
            shots=totals.shots / game_count,
            box_touches=totals.box_touches / game_count,
            crosses=totals.crosses / game_count,
            corners=totals.corners / game_count,
            goals=totals.goals / game_count,
            x_goals=made.x_goals / game_count

class TeamStats(NamedTuple):
    home: TeamGameSetStats
    away: TeamGameSetStats
    agregate: TeamGameSetStats

def team_stats(teamname: str, games: Iterable[Game]) -> TeamStats:
    home_games = [g for g in games if g.home_team == teamname]
    own_home_stats = [g.home_stats for g in home_games]
    opposing_home_stats = [g.away_stats for g in home_games]
    away_games = [g for g in games if g.away_team == teamname]
    own_away_stats = [g.away_stats for g in away_games]
    opposing_away_stats = [g.home_stats for g in away_games]
    return TeamStats(
        home=team_gameset_stats(own_stats=own_home_stats, opposing_stats=opposing_home_stats),
        away=team_gameset_stats(own_stats=own_away_stats, opposing_stats=opposing_away_stats),
            own_stats=own_home_stats + own_away_stats,
            opposing_stats=opposing_home_stats + opposing_away_stats

def calculate_team_stats(teams: Set[str], games: Iterable[Game]) -> Mapping[str, TeamStats]:
    return {
        team: team_stats(team, games)
        for team in teams
  • \$\begingroup\$ Thanks for the great review, I have learned a lot! Could you elaborate on "fields as functions from a defined data structure to a contained datum or sub-structure"? \$\endgroup\$ Jun 15, 2020 at 19:31

Fields as functions from a defined data structure to a contained datum or sub-structure

Elaborating a bit on this point, which I was too abstract about.

Hopefully it's intuitive that, technical details of any particular language/context aside, "fields", "attributes", "properties", etc are all kinda synonymous. We usually visualize objects like this as either a list of ordered pairs \$(\text{name}, \text{value})\$, or as a table where each row is an object and the column-headers are the field-names. That's fine and totally appropriate.

But there's another way of thinking about what a field is, which is reflected in the particular "property" implementation in some languages including Python. (It's also how everything works in Haskell.)
A property of an object is a function from objects of that type to some value which we think of as contained with those objects.

This is relevant to your task because all the "properties" you had of your team_stats items are still conceptually valid in a nested structure like I implemented. But now instead of

PROPERTY("HomeBoxTouchesTotalPg")} := lambda team_stats: team_stats["HomeBoxTouchesTotalPg"]

you'll have

PROPERTY("HomeBoxTouchesTotalPg") := lambda team_stats: team_stats.home.totals_per_game.box_touches

Writing them all out will be a bit of a chore. Sorry.

But then you can do

def print_sample_stats(stats: TeamStats, *fields: str) -> None:
    for field in fields:
        if field not in TeamStats.fields:
            raise NotImplementedError(field)
        print(f"{field}: {TeamStats.fields[field](stats)}")

Of course you'll have plenty of opportunities to make it more complicated than that if you like.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.