0
\$\begingroup\$

I'm working on a tool that converts data between two systems. System 1 presents me with arrays of latitudes and longitudes as doubles. System 2 requires an input of that array as a single Hex string.

I've been calling the following method to do the conversion, and then appending a string with the results, however I'm finding that it is extremely slow, as it's being called 12-13 million times in a typical conversion.

var string = "";
foreach(var value in list<geo>)
string += (DoubleToHex(value.lat) + DoubleToHex(value.lon))
public static string DoubleToHex(double input)
{
    byte[] byteArray = BitConverter.GetBytes(input);
    StringBuilder hex = new StringBuilder(byteArray.Length * 2);
    foreach (byte b in byteArray)
        hex.AppendFormat("{0:x2}", b);
    return hex.ToString().ToUpper();
}

What alternatives exist that would be quicker then using StringBuilder, (or what else am I missing)?

\$\endgroup\$
  • \$\begingroup\$ As a general rule, whenever you say "slow", you have to be more specific. How long does the code take, how big is the input data and how long do you expect the code to take, reasonably? \$\endgroup\$ – Roland Illig Dec 17 '19 at 13:03
3
\$\begingroup\$

Since the conversion only needs one line, I would suggest adding a method in the geo class that returns the latitude and longitude in a hex string:

public string ToHexString()
{
    return $"{BitConverter.ToString(BitConverter.GetBytes(lat))}
             {BitConverter.ToString(BitConverter.GetBytes(lon))}".Replace("-","");
}

Now converting a list of geo objects to one hex string is relatively simple:

List<geo> testList = new List<geo>();
string result = String.Join("",testList.Select(x => x.ToHexString()));

This way you get the same performance boost and more concise code.

\$\endgroup\$
3
\$\begingroup\$

string is immutable type, whenever you modify a string, it actually creates a new string.

if you convert it to StringBuilder such as :

var sb = new StringBuilder(string.Empty);

foreach (var value in list)
    sb.Append(DoubleToHex(value.lat)).Append(DoubleToHex(value.lon));

You'll gain more performance.

However, the DoubleToHex() also can simplified further to :

public static string DoubleToHex(double input)
{
    return BitConverter.ToString(BitConverter.GetBytes(input)).Replace("-", "");
}

this will boost the performance further.

\$\endgroup\$
  • \$\begingroup\$ And DoubleToHex(value.lat) + DoubleToHex(value.lon) does not create a new string? Why not append them both separately? \$\endgroup\$ – slepic Dec 17 '19 at 7:00
  • \$\begingroup\$ @slepic yes, it does, I have fixed it ;) \$\endgroup\$ – iSR5 Dec 17 '19 at 7:08
2
\$\begingroup\$

For such specific problems as you are facing here I almost always turn to BenchmarkDotNet, because it provide you with an easy what to try out different implementation strategies and various .NET features that might tweak the performance to your satisfaction.

Parallel conversion

Parallel.ForEach(...)

and allocate a fair chunk of the array(s) to each of the threads to maximise the CPU utilization. Also try to find the optimal number of threads can be beneficial. The optimal number is often somewhere between the number of cores on your CPU to 2 x CPU cores - but play around with in in BenchmarkDotNet

Reading and Writing

How are data being read and written between System 1 and System 2. Depending if it's database, REST, CSV, File, ... there might be numerous things that also can ben improved in that area. Maybe you can also partition your conversion and bulk read and write data in this part of the flow and not doing it directly on the conversion algorithm.

Maybe using the

async / await

patterns to be less dependent on I/0 bottlenecks.

Algorithm Tweaks

Other StackOverflow articles describe optimised algorithms to do this type of conversion - like Here and maybe in combo with the above you can get your application to perform in the way you need.

Hope you can use some of this for inspiration.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.