I've written some python code designed to take a csv of waypoints for a series of trips, and calculate the distance of each trip by the sum of the distance between the waypoints.
An example csv might be:
9e77d54918dd25c3f9d2e5354ec86666,0,2015-10-01T14:14:15.000Z,45.0988,7.5811,, 9e77d54918dd25c3f9d2e5354ec86666,1,2015-10-01T14:17:15.000Z,45.0967,7.5793,, 9e77d54918dd25c3f9d2e5354ec86666,2,2015-10-01T14:20:15.000Z,45.1012,7.6144,, 9e77d54918dd25c3f9d2e5354ec86666,3,2015-10-01T14:23:15.000Z,45.0883,7.6479,, 9e77d54918dd25c3f9d2e5354ec86666,4,2015-10-01T14:26:15.000Z,45.0774,7.6444,, ect...
I've got code working, using pandas and numpy, however I'm entirely self-taught and I want to know if there's any serious or obvious mistakes I'm using that might make my code inefficient. It currently takes quite a while to run, I'm guessing because of my for loop. The code I'm using is:
import pandas as pd import numpy as np from math import radians, cos, sqrt def dist(lat1, lon1, lat2, lon2): #short distances using Equirectangular approximation lat1, lon1, lat2, lon2 = map(radians, [lat1, lon1, lat2, lon2]) x = (lon2 - lon1) * cos( 0.5*(lat2+lat1) ) y = lat2 - lat1 D = 6371 * sqrt(x**2 + y**2) return D waypoint = pd.read_csv('TripRecordsReportWaypoints.csv',sep=',',header=None, usecols=[0,3,4], names=['TripID','Lat','Lon']) output = pd.DataFrame(columns = ['TripID','Distance','No. of Waypoints']) tripList = waypoint['TripID'].tolist() #creates list of tripids tripList = list(set(tripList)) #makes list unique for ID in tripList: temp = waypoint.loc[waypoint['TripID'] == ID] #creates a temporary dataframe with all waypoint for each trip temp['endLat'] = temp['Lat'].shift(periods=-1) #adds two columns with next waypoints lat and lon temp['endLon'] = temp['Lon'].shift(periods=-1) temp['Distance']=np.vectorize(dist)(temp['Lat'],temp['Lon'],temp['endLat'],temp['endLon']) #calculates distance, can change function 'dist' for more accuracy SumDist = temp['Distance'].sum() #calculates the total distance trpId = temp['TripID'].iloc #takes the tripid wpcount = temp.shape #length of dataframe temp2 = pd.DataFrame([[trpId,SumDist,wpcount]],columns=['TripID','Distance','No. of Waypoints']) #creates a single row dataframe with the total distance output = pd.concat([output,temp2]) #adds the row to the output output.to_csv('TripDistances.csv',sep=',')