5
\$\begingroup\$

I'm currently using the following code to manage calls to WCF services that are unreliable, and or suffer performance load issues due to contention.

Right now this code is limited to solving issue with types that return void. It exponentially backs off from a subsequent call from the server, preventing unhelpful "hammering" of the server.

Question

  • Is there any possible way I can abstract this into something more generic for varying return types?

  • Have I caught all the appropriate exceptions worthy of a retry?

  • What is the most appropriate way I can communicate to the calling process what is going on with this code, perhaps offering an opportunity to cancel early?

  • Any other enhancements you may think of...

Usage Example:

Service<IOrderService>.Use(orderService=>
{
  orderService.PlaceOrder(request);
}

Source code

public delegate void UseServiceDelegate<T>(T proxy);

public static class Service<T>
{
    public static ChannelFactory<T> _channelFactory = new ChannelFactory<T>("");

    public static void Use(UseServiceDelegate<T> codeBlock)
    {
        IClientChannel proxy = (IClientChannel)_channelFactory.CreateChannel();
        bool success = false;


       Exception mostRecentEx = null;
       int millsecondsToSleep = 1000;

       for(int i=0; i<5; i++)  // Attempt a maximum of 5 times
       {
           try
           {
               codeBlock((T)proxy);
               proxy.Close();
               success = true;
               break;
           }

           // The following is typically thrown on the client when a channel is terminated due to the server closing the connection.
           catch (ChannelTerminatedException cte)
           {
              mostRecentEx = cte;
               proxy.Abort();
               //  delay (backoff) and retry
               Thread.Sleep(millsecondsToSleep  * (i + 1));
           }

           // The following is thrown when a remote endpoint could not be found or reached.  The endpoint may not be found or
           // reachable because the remote endpoint is down, the remote endpoint is unreachable, or because the remote network is unreachable.
           catch (EndpointNotFoundException enfe)
           {
              mostRecentEx = enfe;
               proxy.Abort();
               //  delay (backoff) and retry
               Thread.Sleep(millsecondsToSleep * (i + 1));
           }

           // The following exception that is thrown when a server is too busy to accept a message.
           catch (ServerTooBusyException stbe)
           {
              mostRecentEx = stbe;
               proxy.Abort();

               //  delay (backoff) and retry
               Thread.Sleep(millsecondsToSleep * (i + 1));
           }
           catch (TimeoutException timeoutEx)
           {
               mostRecentEx = timeoutEx;
               proxy.Abort();

               //  delay (backoff) and retry
               Thread.Sleep(millsecondsToSleep * (i + 1));
           }
           catch (CommunicationException comException)
           {
               mostRecentEx = comException;
               proxy.Abort();

               //  delay (backoff) and retry
               Thread.Sleep(millsecondsToSleep * (i + 1));
           }
           catch(Exception )
           {
                // rethrow any other exception not defined here
                // You may want to define a custom Exception class to pass information such as failure count, and failure type
                proxy.Abort();
                throw ; 
           }
       }
       if (success == false && mostRecentEx != null)
       {
           proxy.Abort();
           throw new Exception("WCF call failed after 5 retries.", mostRecentEx );
       }

    }
}
\$\endgroup\$
1
  • \$\begingroup\$ May be Polly could help: > Polly is a .NET resilience and transient-fault-handling library that allows developers to express policies such as Retry, Circuit Breaker, Timeout, Bulkhead Isolation, and Fallback in a fluent and thread-safe manner. Polly targets .NET 4.0, .NET 4.5 and .NET Standard 1.0. thepollyproject.org \$\endgroup\$
    – Sevenate
    Jan 26 '17 at 21:51
3
\$\begingroup\$

One clean up is to consolidate the common code in the catch blocks, by putting it into a function. Other ideas:

  • 5 tries is not many for WCF in general. I would increase it to 50 or so. If you do this, you need a ceiling to multiple applied to to the minimum sleep time.
  • The milliseconds to sleep should be initialized to a semi-random value: a minimum value, plus a random fraction of an additional amount. Usually I make the additional amount equal to the minimum value. This will keep all your clients from hammering the server with the same rhythm, in the event all clients start at the same time.

With regards to questions:

  1. Make the method generic, with a type parameter defining the return value.
  2. I usually have only two catch blocks, one for FaultException to communicate exceptions explicitly from server to client, and one for Exception, to catch everything else. The Exception block is where you initiate a retry. Read about FaultContractAttribute.
  3. The only good way to allow cancellation is to run all of this in a background thread, possibly using BackgroundWorker.
\$\endgroup\$
2
\$\begingroup\$

WCF has a built-in feature to deal with unreliable networks and such: reliable messaging. See the link for an explanation...

\$\endgroup\$
1
  • 2
    \$\begingroup\$ link-only answers are discouraged. please try to sum up the main points of the link you provided in case the page becomes unavailable (even if highly unlikely ;)) \$\endgroup\$
    – Vogel612
    Feb 27 '14 at 17:17
2
\$\begingroup\$

It's not enough to just catch the exception. Just catching it doesn't prevent your users to wait for the whole timeout duration. You need to build your application so that it will "fail fast" in an event of timeout. Take a look at this post. It shows how you can achieve this using the Polly open source. Here is part of code. Full explanation is in the post:

   public NavigationHistoryController(INavigationHistoryProvider historyProvider)
    {
        _historyProvider = historyProvider;
        _policy = Policy.Handle<SqlException>(ex => ex.Number == TimeoutExceptionCode).CircuitBreaker(10, TimeSpan.FromMinutes(10));
    }

    public List<History> GetHistory()
    {
        try
        {
            return _policy.Execute(() => _historyProvider.Get());
        }
        catch (SqlException exception)
        {
            return null;
        }
        catch (BrokenCircuitException)
        {
            return new List<History>();
        }
    }
\$\endgroup\$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.