IResourceLoader: Balancing Semaphores

Recently I need to get balance getting resources from a restricted number of sources. So, for example...

I am getting resource R, and I have factories A, B and C creating those Rs. Each of those factories has a very limited capacity for creating those resources, and can only create two Rs at a time. It is easy to put the factories behind a semaphore and limit how many threads can be requesting resources from each factory at a time.

The challenge is evenly balancing the workload between all three factories. Also, please note that you can't just round robin the semaphores because there is no way to ensure that each operation will complete in the same amount of time.

To do this I created a generic IResourceLoader interface, and made two implementations: one to wrap a semaphore, and the other to wrap and balance a collection of IResourceLoaders. Below is the implementation, complete with unit tests; let's take a look!

Interface

public interface IResourceLoader<T>

{

    int Available { get; }

    int Count { get; }

    int MaxConcurrency { get; }

    Task<T> GetAsync(CancellationToken cancelToken = default(CancellationToken));

    bool TryGet(out Task<T> resource, CancellationToken cancelToken = default(CancellationToken));

}

ConcurrentDictionary.GetOrAdd and Thread Safety

.NET 4.0 added the awesome System.Collections.Concurrent namespace, which includes the very useful ConcurrentDictionary class. While the ConcurrentDictionary is thread safe, it can experience problems with adding values during high concurrency...

ConcurrentDictionary.GetOrAdd may invoke the valueFactory multiple times per key.

This behavior will only happens under high load, and even if the valueFactory does get invoked multiple times the dictionary entry will only ever be set once. Normally this is not much of a problem. However, if you are using this Dictionary to store large or expensive objects (such as unmanaged resources or database connections), then the accidental instantiation of multiple of these could be a real problem for your application.

Don't worry, there is a very simple solution to avoid this problem: just create Lazy wrappers for your expensive objects. That way it will not matter how many times the valueFactory is called, because only one instance of the resource itself will ever actually be accessed and instantiated.