JSON serialization object with huge IEnumerable

This code is looping over polygons geometries vertices in order to create a list of deltas (distance) between vertices coordinates to compact the geometry before sending it into the web.

Geometry output is expected to be on this format: [[x0, y0, (x0-x1), (y0-y1), (x1-x2), (y1-y2)]]

The output is a JSON object containing a list of features attributes and geometry deltas like this one:

{
features: [
{
attributes : {"CD_MUNCP" : 00000, "NOM_MUNCP": "Name1"},
geometry : [[5767767, -834778, -10, 199, 99, 332, 9, -9], [5787767, -837709, 123, 33, -31, 121, 0, 12330]]
}
]
}


I am using IEnumerable<> to create the geometry deltas because I thought it would be faster, but it appears this is not really compatible with JSON serialization (it takes very long to serialize) so I transform them using .ToList().

The most important thing with the code is the performance. It seems the bottleneck is the JSON serialization that takes more than 50% of the processing time.

Is there some optimizations or workarounds I can do to improve the performance?

using System;
using System.Collections.Generic;
using fastJSON;
using ESRI.ArcGIS.Geodatabase;
using ESRI.ArcGIS.Geometry;
using ESRI.ArcGIS.esriSystem;

namespace DesktopConsoleApplication1
{
class Program
{
static void Main(string[] args)
{
var watch = System.Diagnostics.Stopwatch.StartNew();
Result result = new Result
{
features = CreateResultFeatures().ToList()
};
string output = JSON.ToJSON(result);
watch.Stop();
Console.WriteLine(watch.ElapsedMilliseconds);
}

public class Result
{
public IEnumerable<ResultFeature> features { get; set; }
}

public class ResultFeature
{
public Dictionary<string, dynamic> attributes { get; set; }
public IEnumerable<IEnumerable<int>> geometry { get; set; }
}

public static IEnumerable<ResultFeature> CreateResultFeatures()
{
IWorkspace gdbWorkspace = FileGdbWorkspaceFromPath(@"\\vnageop1\geod\Maxime\test.gdb");
IFeatureWorkspace featureWorkspace = (IFeatureWorkspace)gdbWorkspace;
IFeatureClass featureClass = featureWorkspace.OpenFeatureClass(@"GEO09E04_MUNCP_GEN");
IQueryFilter queryFilter = new QueryFilterClass();
queryFilter.SubFields = "CD_MUNCP, NOM_MUNCP, SHAPE";

int cd_muncp_idx = featureClass.FindField("CD_MUNCP");
int nom_muncp_idx = featureClass.FindField("NOM_MUNCP");

using (ComReleaser comReleaser = new ComReleaser())
{
IFeatureCursor cursor = featureClass.Search(queryFilter, false);
IFeature feature = null;
while ((feature = cursor.NextFeature()) != null)
{
ResultFeature resultFeature = new ResultFeature
{
attributes = new Dictionary<string,dynamic>
{
{ "CD_MUNCP", Convert.ToString(feature.Value[cd_muncp_idx]) },
{ "NOM_MUNCP", Convert.ToString(feature.Value[nom_muncp_idx]) }
},
geometry = PolygonToDeltas(feature.Shape as IPolygon4).ToList()
};
yield return resultFeature;
}
}
}

public static IEnumerable<IEnumerable<int>> PolygonToDeltas(IPolygon4 polygon)
{
IGeometryBag exteriorRingGeometryBag = polygon.ExteriorRingBag;
IGeometryCollection exteriorRingGeometryCollection = exteriorRingGeometryBag as IGeometryCollection;
for (int i = 0; i < exteriorRingGeometryCollection.GeometryCount; i++)
{
IGeometry exteriorRingGeometry = exteriorRingGeometryCollection.get_Geometry(i);
IPointCollection exteriorRingPointCollection = exteriorRingGeometry as IPointCollection;
yield return CreateDeltas(exteriorRingPointCollection);

IGeometryBag interiorRingGeometryBag = polygon.get_InteriorRingBag(exteriorRingGeometry as IRing);
IGeometryCollection interiorRingGeometryCollection = interiorRingGeometryBag as IGeometryCollection;
for (int k = 0; k < interiorRingGeometryCollection.GeometryCount; k++)
{
IGeometry interiorRingGeometry = interiorRingGeometryCollection.get_Geometry(k);
IPointCollection interiorRingPointCollection = interiorRingGeometry as IPointCollection;
yield return CreateDeltas(exteriorRingPointCollection);
}
}
}

private static IEnumerable<int> CreateDeltas(IPointCollection pointCollection)
{
int previous_x = (int)pointCollection.get_Point(0).X;
int previous_y = (int)pointCollection.get_Point(0).Y;
yield return previous_x;
yield return previous_y;
for (int i = 1; i < pointCollection.PointCount; i++)
{
int current_x = (int)pointCollection.get_Point(i).X;
int current_y = (int)pointCollection.get_Point(i).Y;
yield return previous_x - current_x;
yield return previous_y - current_y;
previous_x = current_x;
previous_y = current_y;
}
}
}
}

• Without having run it, I'd say the bottleneck is the dynamic. It doesn't look like it's necessary. Try to replace it with a strong type and see how it performs then. – t3chb0t Oct 18 '18 at 7:45
• Please do not update the code in your question to incorporate feedback from answers, doing so goes against the Question + Answer style of Code Review. This is not a forum where you should keep the most updated version in your question. Please see what you may and may not do after receiving answers. – Mast Oct 18 '18 at 12:27

I haven't access to the ESRI assemblies, so it's impossible to test and comment about performance, but in this:

public static IEnumerable<IEnumerable<int>> PolygonToDeltas(IPolygon4 polygon)
{
IGeometryBag exteriorRingGeometryBag = polygon.ExteriorRingBag;
IGeometryCollection exteriorRingGeometryCollection = exteriorRingGeometryBag as IGeometryCollection;
for (int i = 0; i < exteriorRingGeometryCollection.GeometryCount; i++)
{
IGeometry exteriorRingGeometry = exteriorRingGeometryCollection.get_Geometry(i);
IPointCollection exteriorRingPointCollection = exteriorRingGeometry as IPointCollection;
yield return CreateDeltas(exteriorRingPointCollection);

IGeometryBag interiorRingGeometryBag = polygon.get_InteriorRingBag(exteriorRingGeometry as IRing);
IGeometryCollection interiorRingGeometryCollection = interiorRingGeometryBag as IGeometryCollection;
for (int k = 0; k < interiorRingGeometryCollection.GeometryCount; k++)
{
IGeometry interiorRingGeometry = interiorRingGeometryCollection.get_Geometry(k);
IPointCollection interiorRingPointCollection = interiorRingGeometry as IPointCollection;
// ISN'T THIS A BUG:
yield return CreateDeltas(exteriorRingPointCollection);
}
}
}


... it seems that you call CreateDeltas(exteriorRingPointCollection) both in the outer for-loop where it is expected to be called and in the inner for-loop where I would expect this call: CreateDeltas(interiorRingPointCollection). It seems to be a bug? If exteriorRingPointCollection is significantly larger than interiorRingPointCollection that could be a bottleneck for many interior rings per exterior ring?

By the way: have you tested if sending the absolute coordinates is a real problem?

From one of your comments below, I understand that the bottleneck must be at:

string output = JSON.ToJSON(result);


The result classes are defined as:

    public class Result
{
public IEnumerable<ResultFeature> features { get; set; }
}

public class ResultFeature
{
public Dictionary<string, dynamic> attributes { get; set; }
public IEnumerable<IEnumerable<int>> geometry { get; set; }
}


Where the ResultFeature.attributes is a Dictionary with value as dynamic. You could maybe try to change that to string or object or another defined type.

Alternatively:

You are using fastJSON which I have no experience with. Have you tried Newtonsoft.Json instead?

If you want to use c++ apis in C# it can be done using external references

• Sorry it's a typo. It should be yield return CreateDeltas(interiorRingPointCollection); in the inner loop. I have corrected the code in the question. – Below the Radar Oct 18 '18 at 11:50
• The problem with sending absolute coordinates is just a matter of size. By sending deltas, I can compact the json by more than 50% and recreating it on client side is an easy job. – Below the Radar Oct 18 '18 at 11:55
• @BelowtheRadar: Yes, but I just wonder if it's worth the trouble? Do you reach a limit of data size? – user73941 Oct 18 '18 at 12:14
• My guess is that it will take even more time to serialize the absolute coordinates to json and the download time will be longer. For instance some json with absolute coordinates could be 80Mo versus 8Mo for deltas. It takes only 1 second to create the deltas but 25seconds to serialize it to json... – Below the Radar Oct 18 '18 at 12:21