2
\$\begingroup\$

Problem: Given two csv files science_courses.csv and other_courses.csv, which has students name and their respective course points. Calculate the CGPA and sort by the same (highest to lowest CGPA).

sample file content:

science_courses.csv:

Name,Physics,Chemistry
Susan,6.0,5.5
James,4.7,8.5
Linda,4.5,6.3
Mathew,7.7,5.4
-----------------------

other_courses.csv

Name,History,Math,Arts
Linda,8.3,7.4,6.9
James,6.3,7.5,5.9
Mathew,5.8,8.9,4.9
Susan,8.8,9.0,5.9

Approach:

  1. Open both files in and make a dictionary. ex:{studentA:[mark-1,mark-2..mark-n],studentB:[mark-1...mark-n]}
  2. Iterate through the dictionary of the list and calculate CGPA and store in a dictionary. ex:{studentA:cgpa, studentB:cgpa}
  3. Sort the results by cgpa (highest to lowest) and print

solution:

def get_all_courses(a,b):
    all_courses = {}
    with open(a, 'r') as sci, open(b, 'r') as othr:
        next(sci)
        for i in sci:
            sc = i.strip('\n').split(',')
            all_courses[sc[0]] = sc[1:]

        for j in othr:
            ot = j.strip('\n').split(',')
            name = ot[0]
            if name in all_courses:
                all_courses[name] = all_courses.get(name) + ot[1:]
                
    return all_courses

def get_results():
    avg_marks = {}
    result = get_all_courses('science_courses.csv','other_courses.csv')
    for k,v in result.items():
        total = sum(map(float, v))
        avg = round(total/len(v),2)
        avg_marks[k] = avg
        
    for i in sorted(avg_marks.items(), key=lambda x: x[1], reverse=True): print i

get_results()

Please suggest better alternatives to:

  1. simplify the book keeping of the marks extracted from files
  2. simplify/optimise the calculation and sorting based on cgpa.
\$\endgroup\$
1
\$\begingroup\$

First of all: don't use Python2, it's end of life. The telltale sign that you're using it is the lack of parentheses to the print function.

Since Python has a built-in library for reading CSV files, it would make sense to use it. The parsing would be more elegant and straightforward.

Your function get_all_courses expects exactly two arguments. It would be more flexible to use only one argument, but call the function as many times as necessary. Or use a list of files as parameter.

As for the aggregation, I think your logic is fairly to the point.

The next step is to write a function that reads from a single file (given as parameter) and returns a dict-like structure. Which you did. But I'd rather return the file data with as little transformation as possible and instead delegate the consolidation to another function to better separate concerns.

My implementation follows below. I wouldn't call it superior, because there are actually more steps involved. It is more for demonstration purposes of the features available in Python.

For more flexibility I have added a delimiter parameter in the csv reader routine, in case you have to parse files that are not comma-delimited.

My example returns a list of dictionaries from each file, like:

[{'Name': 'Susan', 'Physics': '6.0', 'Chemistry': '5.5'}, {'Name': 'James', 'Physics': '4.7', 'Chemistry': '8.5'}, {'Name': 'Linda', 'Physics': '4.5', 'Chemistry': '6.3'}, {'Name': 'Mathew', 'Physics': '7.7', 'Chemistry': '5.4'}]

The idea is to merge those lists into one single list, hence the extend method.

Finally, we need to apply some form of "group by" to the results and calculate averages. We want to group by name. For this we use the itertools.groupby function. There is one caveat though: the dataset must be sorted by whatever column(s) shall be used as the grouping key. Therefore we sort our list of dict by name beforehand.

We end up with something like this:

James : [{'Name': 'James', 'Physics': '4.7', 'Chemistry': '8.5'}, {'Name': 'James', 'History': '6.3', 'Math': '7.5', 'Arts': '5.9'}]
Linda : [{'Name': 'Linda', 'Physics': '4.5', 'Chemistry': '6.3'}, {'Name': 'Linda', 'History': '8.3', 'Math': '7.4', 'Arts': '6.9'}]
Mathew : [{'Name': 'Mathew', 'Physics': '7.7', 'Chemistry': '5.4'}, {'Name': 'Mathew', 'History': '5.8', 'Math': '8.9', 'Arts': '4.9'}]
Susan : [{'Name': 'Susan', 'Physics': '6.0', 'Chemistry': '5.5'}, {'Name': 'Susan', 'History': '8.8', 'Math': '9.0', 'Arts': '5.9'}]

At this point the rest is easy, we simply read the dict values for each student and compute averages while ignoring the 'Name' key. (Actually I am not completely satisfied with this attempt, and there are countless approaches possible).


import csv
from itertools import groupby
from operator import itemgetter
from functools import reduce
from statistics import mean


def get_data_from_file(filepath: str, delimiter: str = ",") -> list:
    """Read a CSV file and return a list of dict eg:
    Return a list of dict eg:
    [
        {'Name': 'Susan', 'Physics': '6.0', 'Chemistry': '5.5'},
        ...
    ]

    Default field delimiter is a comma
    """
    with open(filepath) as csv_file:
        csv_reader = csv.DictReader(csv_file, delimiter=delimiter)
        return list(csv_reader)


all_courses = []
all_courses.extend(get_data_from_file(filepath='science_courses.csv'))
all_courses.extend(get_data_from_file(filepath='other_courses.csv'))

# the dataset must be sorted before we apply groupby
all_courses = sorted(all_courses, key=itemgetter('Name'))

# group by Name
for key, group in groupby(all_courses, lambda x: x["Name"]):

    # get all points for this student
    student_points = list(group)

    # merge the dicts - source: https://stackoverflow.com/a/16048368
    student_points = reduce(lambda a, b: dict(a, **b), student_points)

    # drop the "Name" key
    del student_points["Name"]

    # compute average of points for student
    points_average = mean([float(value) for value in student_points.values()])
    print(f"Student: {key}, average: {points_average}")

The bottom line is that parsing the CSV file can be streamlined with the built-in library. Python provides a number of functions/libs for aggregating results, such as groupby, or statistics.

PS: for more complex usage Pandas is very popular among python programmers. In fact the same task could have been achieved with Pandas instead, and more likely in a more efficient way.


\$\endgroup\$
1
  • \$\begingroup\$ +1 for import csv \$\endgroup\$ Aug 28 at 9:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.