# Function to calculate distances and nearest points between 2 GeoPandas dataframes

I have 2 geoPandas frames and want to calculate the distance and the nearest point (see functions below) from the geoSeries geometry from dataframe 1 (containing 156055 rows with unique POINT geometries) as to a geoSeries geometry in dataframe 2 (75 rows POINTS).

Question: How can the following code be optimized so as to make it quicker? As an example, I would love some code that uses the dask distributed dataframes map_partitions solution to apply the below functions on a partitioned part of a (sub)-dataframe so as to speed things up. Optimized vector-based cython code would also be a solution etc..

A peak in both dataframes:

df1:

df2:

In the ah frame (df2) there are 75 points, around which I place a buffer of 1000 meters. In each buffer I calculate the distance (using the points of df1 representing households) and nearest point.

My code so far works but, as said, is slow:

import pandas as pd
import geopandas as gpd
from shapely import wkt, wkb
from shapely.geometry import Polygon, Point, LinearRing
from shapely.ops import nearest_points
from tqdm import tqdm, tqdm_notebook

# crs
crs= {'init': 'epsg:28992'}

def calculate_distance(row, dest_geom, src_col='geometry',
target_col='distance'):
"""
Calculates distance between single Point geometry and GeoDF
with Point geometries.

Parameters
----------
dest_geom : Point geometry to which distances will be calculated to
src_col : column with Point objects from where the distances will be calculated from.
target_col : name of target column where the result will be stored.
"""
# calculate distances
dist = row[src_col].distance(dest_geom)
# to km
dist_km = dist/1000
# Assign the distance to the original data
row[target_col] = dist_km
return row

def find_nearest_point(row, geom_union, df1, df2, geom1_col='geometry',
geom2_col='geometry', src_column=None):
"""Find nearest point,return corresponding value from specified
column
Parameters
----------
geom_union = variable with unary union of points from second frame.
df1 = dataframe 1 containing geometry column (points)
df2 = dataframe 2 containing geometry column (points)
geom1_col = geometry column name from df1
geom2_col = geometry column name from df2
src_column =  columns from df2 to be retrieved based on nearest match
"""
# Find closest geometry
nearest = df2[geom2_col] == nearest_points(row[geom1_col], geom_union)[1]
# Get corresponding value from df2 (based on geometry match)
value = df2[nearest][src_column].get_values()[0]
return value


Here i apply the functions with a loop:

n=len(ah)

test = []
for i, row in enumerate(tqdm_notebook(list(ah['buffer'][:n]),
desc='distance_calc')):
sub_df = df.loc[(df.geometry.within(ah['buffer'][i])), :]
sub_df = (sub_df.apply(calculateDistance,
dest_geom= ah['geometry'][i],
target_col= 'distance', axis=1))
print ('shape sub_df {} = {}'.format(i, sub_df.shape))

indices = (sub_df.apply(nearest,
geom_union=ah.unary_union,
df1=sub_df,
df2=ah,
geom1_col='geometry',
src_column='zaak_id',
axis=1))

indices_frame = indices.to_frame()
sub_df = pd.concat([sub_df, indices_frame], axis=1)

test.append(sub_df)

test = pd.concat(test, axis=0)

• what if a review of your code points out that the performance could be increased without the use of dask distributed dataframes map_partitions ? would that still be acceptable? – Malachi Sep 4 '18 at 14:41
• yes. Made that clear now in the question – ad_s Sep 4 '18 at 14:51