5
\$\begingroup\$

I have a CSV like the following.

Category,Position,Name,Time,Team,avg_power,20minWKG,Male?,20minpower
A,1,Tom Smith ,00:41:58.95,7605,295,4.4,1,299.2
A,2,James Johnson,00:41:58.99,2740,281,4.5,1,283.95
A,3,Tom Lamb,00:41:59.25,1634,311,4.2,1,315
B,1,Elliot Farmer,00:45:06.23,7562,306,3.9,1,312
B,2,Matt Jones,00:45:10.10,4400,292,4.0,1,300
B,3,Patrick James,00:45:39.83,6508,299,4.1,1,311.6

I get the time of the rider in first place of category 'A'. I then give any rider that is 15% slower than this 0 points in a new CSV. For example if the first rider take 1 minute 40 seconds, then anyone slower than 1 minute 55 seconds will get 0 points.

def convert(seconds):  # function to convert amount of seconds to a time format
    seconds = seconds % (24 * 3600)
    hour = seconds // 3600
    seconds %= 3600
    minutes = seconds // 60
    seconds %= 60
    return "%d:%02d:%02d" % (hour, minutes, seconds)


with open("results.csv", 'rt', encoding='UTF-8', errors='ignore') as file:  # opening the full results file
    reader = csv.reader(file, skipinitialspace=True, escapechar='\\')  # skipping headers
    MaleCategoryList = []  # setting category as blank so a change is recognised
    for row in reader:
             if row[0] not in MaleCategoryList:
                    if row[0] == "A":
                        firstPlaceTime = datetime.strptime(row[3], "%H:%M:%S.%f")
                        timeInSecs = firstPlaceTime.second + firstPlaceTime.minute * 60 + firstPlaceTime.hour * 3600
                        timeDifference = timeInSecs * 1.15
                        MaxTime = datetime.strptime(convert(timeDifference), "%H:%M:%S")
# some code here which is not relevant i.e calculate points
             if cat == "A" and datetime.strptime(row[3], "%H:%M:%S.%f") > MaxTime:
                    points = int(0)
                    position_for_file = "DQ Time-Cut"
                    cat = "Time Cut"
             data = {'Position': position_for_file, 'Category': cat, 'Name': name, 'Club': club,
                        'Points': points, 'Time': time}  # dictionary of data to write to CSV

I feel that my code is very messy and inefficient. This is as there are lots of if in my loop and I believe a lot of the calculations seem unnecessary. Do you have any ideas of how I could improve my code?

I would prefer to not use Pandas if possible. This is as I have never used. However if there is a fast and easily maintainable solution to this with Pandas then I would be interested.

\$\endgroup\$
6
\$\begingroup\$

Time calculation

timeInSecs = firstPlaceTime.second + firstPlaceTime.minute * 60 + firstPlaceTime.hour * 3600

The first thing to do is attempt to get this as a timespan and avoid your own time math. In other words,

from datetime import datetime, time, timedelta

MIDNIGHT = time()

# ...

first_place_time = datetime.strptime(row[3], "%H:%M:%S.%f")
time_span = first_place_time - datetime.combine(first_place_time, MIDNIGHT)
time_difference = time_span.total_seconds() * 1.15

Unpacking

You can unpack your row and avoid row[0], etc. fixed indexing, like:

category, position, name, race_time = row[:4]

Call reuse

This is written twice:

datetime.strptime(row[3], "%H:%M:%S.%f")

so store it to a temporary variable.

PEP8

MaleCategoryList should be male_category_list, and MaxTime should be max_time.

Convert

Given the above code, you can get rid of most of convert. You should not put the time through another round-trip using a string.

| improve this answer | |
\$\endgroup\$
  • \$\begingroup\$ Thanks, I'll have a go with your fixes soon. I'm fairly new to Python, I think unpacking is a good idea, I will implement that, could you explain the PEP8 section please from a quick look it seems like that's the general convention but why do you convert MaleCategoryList but not firstPlaceTime for example? I've also updated the code with a full CSV \$\endgroup\$ – PythonIsBae Jun 27 at 14:41
  • \$\begingroup\$ I just offered examples; first_place_time would also be affected. \$\endgroup\$ – Reinderien Jun 27 at 14:42
5
\$\begingroup\$

Since you asked for pandas, here is one way to do it:

import pandas as pd
import numpy as np

df = pd.read_csv("results.csv", encoding='UTF-8',
                 skipinitialspace=True, escapechar='\\')
# convert the times to timedeltas right away
df["Time"] = pd.to_timedelta(df["Time"])
# if it is not already sorted, do so
df = df.sort_values("Time")
# we are going to need this mask to distinguish between category A and the others
category_A = df["Category"] == "A"
# calculate the cutoff time, after which people get 0 points
cutoff = df.loc[category_A, "Time"].min().total_seconds() * 1.15
# calculate the points for all runners (however you do that)
df["Points"] = calculate_points(df)
# set the points to 0 for all runners in category A which are above the cutoff time
df.loc[category_A & (df["Time"].dt.total_seconds() > cutoff), "Points"] = 0
# save to new file
df.to_csv("output.csv")
print(df)

#   Category  Position           Name            Time  Team  avg_power  20minWKG  Male?  20minpower  Points
# 0        A         1     Tom Smith  00:41:58.950000  7605        295       4.4      1      299.20   100.0
# 1        A         2  James Johnson 00:41:58.990000  2740        281       4.5      1      283.95   100.0
# 3        B         1  Elliot Farmer 00:45:06.230000  7562        306       3.9      1      312.00    42.0
# 4        B         2     Matt Jones 00:45:10.100000  4400        292       4.0      1      300.00    42.0
# 5        B         3  Patrick James 00:45:39.830000  6508        299       4.1      1      311.60    42.0
# 2        A         3       Tom Lamb 00:51:59.250000  1634        311       4.2      1      315.00     0.0

Note that I modified the time of "Tom Lamb" so that there is actually a runner which is affected by the time cut


This is not the nicest code. It could be improved if this cut was done in each category, compared to the fastest person in that category, or, potentially, if you showed the point calculation.

| improve this answer | |
\$\endgroup\$
  • 1
    \$\begingroup\$ I have added point calculation but I’m not sure it’s the most useful. \$\endgroup\$ – PythonIsBae Jun 27 at 15:06
  • \$\begingroup\$ Also, I just re-read this the cut isn't done to everyone in the category as the calculation is if you are more than 15% above first-place time in Cat A you get 0 points. I like the pandas answer but I did not notice much more of a speed/debuggability improvement so didn't use it as is would be harder for me to change myself. \$\endgroup\$ – PythonIsBae Jun 27 at 22:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.