3
\$\begingroup\$

I'm writing a small program to plot new COVID-19 infections. As of right now, I have it so the program reads the given data file, pulls out the daily cases and dates for each country, and adds together all the cases for a given date. However, because both of the lists generated have lengths of over 2000, it currently runs extremely slowly. Is there any change I can make to improve the speed of my program?

import pylab as pl

cases = pd.read_csv("daily-cases-covid-19.csv")
dc = cases.loc[:,'Daily confirmed cases (cases)']
dd = cases.loc[:,'Date']




worldCases = []

for i in range(0,len(dd)):
    count = 0
    for j in range(0,len(dd)):
        if dd[j]==dd[i]:
            count+=dc[i]
    worldCases.append(count)

Here is an example of the CSV I am reading through. The purpose of the nested loops is to add together all of the confirmed cases in each country on a given date.

Afghanistan,AFG,"Jan 1, 2020",0
Afghanistan,AFG,"Jan 2, 2020",0
Afghanistan,AFG,"Jan 3, 2020",0
Afghanistan,AFG,"Jan 4, 2020",0
Afghanistan,AFG,"Jan 5, 2020",0
Afghanistan,AFG,"Jan 6, 2020",0
Afghanistan,AFG,"Jan 7, 2020",0
Afghanistan,AFG,"Jan 8, 2020",0
Afghanistan,AFG,"Jan 9, 2020",0
Afghanistan,AFG,"Jan 10, 2020",0
Afghanistan,AFG,"Jan 11, 2020",0
Afghanistan,AFG,"Jan 12, 2020",0
Afghanistan,AFG,"Jan 13, 2020",0
Afghanistan,AFG,"Jan 14, 2020",0
Afghanistan,AFG,"Jan 15, 2020",0
Afghanistan,AFG,"Jan 16, 2020",0
Afghanistan,AFG,"Jan 17, 2020",0
Afghanistan,AFG,"Jan 18, 2020",0
Afghanistan,AFG,"Jan 19, 2020",0
\$\endgroup\$
1
  • 1
    \$\begingroup\$ Welcome to Code Review! The current question title, which states your concerns about the code, is too general to be useful here. Please edit to the site standard, which is for the title to simply state the task accomplished by the code. Please see How to get the best value out of Code Review: Asking Questions for guidance on writing good question titles. \$\endgroup\$ – Sᴀᴍ Onᴇᴌᴀ May 29 '20 at 22:08
3
\$\begingroup\$

So it seams like you have a Pandas dataframe you are working with. At the moment it looks like this section of code does the following

worldCases = []                   

for i in range(0,len(dd)):               #iterate through all dates
    count = 0                            #setup a counter
    for j in range(0,len(dd)):           #iterate through all dates
        if dd[j]==dd[i]:                 
            count+=dc[i]                 #add one to counter if inner date == outer date
    worldCases.append(count)             #track number of times a unique date occurs

You are effectively binning your data, getting a count of the number of times each unique date occurs. Pandas offers you more efficient and convenient tools for doing this. Specifically look into the groupby method.

A much faster way to get the same output for worldCases would be to do the following:

# group the daily cases data by the date and then compute 
# the sum of cases within each date group

dc = 'Daily confirmed cases (cases)'
worldCases = cases.loc[:, dc].groupby(cases['Date']).sum()

Also, if you find yourself writing lots of loops like the above, you may want to check out the user guide for "group by" operations in the pandas documentation. It is a very powerful way of handling situations like this, and more, but can take a bit of getting used to. It can feel like a whole different way of thinking about your data.

\$\endgroup\$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.