c# - Wrapping rate limiting API call

Question

I have access an API call that accepts a maximum rate of calls per second. If the rate is exceeded, an exception is thrown.

I would like to wrap this call into an abstraction that does the necessary to keep the call rate under the limit. It would act like a network router: handling multiple calls and returning the results to the correct caller caring about the call rate. The goal is to make the calling code as unaware as possible about that limitation. Otherwise, every part in the code having this call would have to be wrapped into a try-catch!

For example: Imagine that you can call a method from an extern API that can add 2 numbers. This API can be called 5 times per second. Anything higher than this will result in an exception.

To illustrate the problem, the external service that limits the call rate is like the one in the answer to this question

How to build a rate-limiting API with Observables?

ADDITIONAL INFO:

Since you don't want the worry about that limit every time you call this method from any part of your code, you think about designing a wrapper method that you could call without worrying about the rate limit. On the inside you care about the limit, but on the outside you expose a simple async method.

It's similar to a web server. How does it return the correct pack of results to the correct customer?

Multiple callers will call this method, and they will get the results as they come. This abstraction should act like a proxy.

How could I do it?

I'm sure the firm of the wrapper method should be like

public async Task<Results> MyMethod()

And inside the method it will perform the logic, maybe using Reactive Extensions (Buffer). I don't know.

But how? I mean, multiple calls to this method should return the results to the correct caller. Is this even possible?

Thank you a lot!

Answer 1

There are rate limiting libraries available (see Esendex's TokenBucket Github or Nuget).

Usage is very simple, this example would limit polling to 1 a second

// Create a token bucket with a capacity of 1 token that refills at a fixed interval of 1 token/sec.
ITokenBucket bucket = TokenBuckets.Construct()
  .WithCapacity(1)
  .WithFixedIntervalRefillStrategy(1, TimeSpan.FromSeconds(1))
  .Build();

// ...

while (true)
{
  // Consume a token from the token bucket.  If a token is not available this method will block until
  // the refill strategy adds one to the bucket.
  bucket.Consume(1);

  Poll();
}

I have also needed to make it async for a project of mine, I simply made an extension method:

public static class TokenBucketExtensions
{
    public static Task ConsumeAsync(this ITokenBucket tokenBucket)
    {
        return Task.Factory.StartNew(tokenBucket.Consume);
    }
}

Using this you wouldn't need to throw/catch exceptions and writing a wrapper becomes fairly trivial