NetCore Ocelot 之 Load Balancer

Ocelot can load balance across available downstream services for each Route. This means you can scale your downstream services and Ocelot can use them effectively.

The Type of load balancer availble are:

   LeastConnection  - tracks which service are dealing with requests and sends new requests to services with least existing requests.The algorithm state is not distributed across a cluster of Ocelot's.

   RoundRobin - loops through available services and sends requests. The algorithm state is not distributed across a cluster of Ocelot's.

    NoLoadBalancer  -takes the first available service from config or service discovery.

    CookieStickySesssions - Uses a cookie to stick all requests to a specific server.

Please note that if you give more than one DownstreamHostAndPort or you are using a Service Discovery provider such as Consul and this returns more than one service then CookieStickSession uses round robin to select the next server. This is hard code at the moment but could be changed.

 "LoadBalancerOptions": {
        //"Type": "CustomRandomLoadBalancer"
        "Type": "CookieStickySessions",
        "Key": "ASP.NET_SessionId",
        "Expiry": 1800000
      }

In order to create and use a custom load balancer you can do the following. Below we setup a basic load balancing config and the Type is CustomLoadBalancer this is the name of a class we will setup to do load balancing.

{
      "DownstreamPathTemplate": "/api/service1",
      "DownstreamScheme": "https",
      "DownstreamHttpMethod": "Get",
      "UpstreamHttpMethod": [ "Options", "Get", "Post", "Put", "Delete" ],
      "UpstreamPathTemplate": "/Ocelot/service1",
      //"UpstreamHost": "localhost",//404 Not found
      "UseServiceDiscovery": true,
      "ServiceName": "serviceA",
      /*
      LeastConnection
      RoundRobin
      NoLoadBalance
      */
      "LoadBalancerOptions": {
        "Type": "CustomRandomLoadBalancer"
      },
      "RateLimitOptions": {
        "ClientWhiteList": [ "NoLimitedAPI" ],
        "EnableRateLimiting": true,
        "Period": "10s", //1s, 5m, 1h, 1d
        "PeriodTimespan": 10, //retry after n second/seconds
        "Limit": 3
      },
      "authenticationoptions": {
        "authenticationproviderkey": "authenticationkey",
        "allowedscopes": [ "apiscope1" ]
      },
      "Key": "service1",
      "Priority": 11
    }

Then you need to create a class that implements the ILoadBalancer interface. 

public class CustomRandomLoadBalancer : ILoadBalancer
{
        private readonly Func<Task<List<Service>>> _services;
        private readonly Object _lock = new object();
        private int _index;

        public CustomRandomLoadBalancer(Func<Task<List<Service>>> services)
        {
            _services = services;
        }

        public async Task<Response<ServiceHostAndPort>> Lease(HttpContext httpContext)
        {
            var services = await _services();
            if (services == null)
                return new ErrorResponse<ServiceHostAndPort>(new ErrorInvokingLoadBalancerCreator(new Exception("Load balance algorithm error.")));
            lock (_lock)
            {
                if (services.Count == 1)
                    return new OkResponse<ServiceHostAndPort>(services[0].HostAndPort);
                _index = new Random().Next(services.Count);
                return new OkResponse<ServiceHostAndPort>(services[_index].HostAndPort);
            }
        }

        public void Release(ServiceHostAndPort hostAndPort)
        {
        }
    }

Finally you need to register this class with Ocelot. I have used the most complex example below to show all of the data/types that can be passed into the factory that create load balancers.

Func<IServiceProvider, DownstreamRoute, IServiceDiscoveryProvider, CustomLoadBalancer> loadBalancerFactoryFunc = (serviceProvider, Route, serviceDiscoveryProvider) => new CustomLoadBalancer(serviceDiscoveryProvider.Get);

s.AddOcelot()
    .AddCustomLoadBalancer(loadBalancerFactoryFunc);

However there is a much simple example that will work the same.

builder.Services.AddOcelot()
    //.AddCacheManager(c => c.WithDictionaryHandle())// cache manager
    .AddSingletonDefinedAggregator<CustomAdvancedAggregator>()
    .AddCustomLoadBalancer((serviceProvider, route, serviceDiscoveryProvider) => new CustomRandomLoadBalancer(serviceDiscoveryProvider.Get))
    .AddConsul()
    .AddPolly();

When you enable custom load balancers Ocelot looks up your load balancer by its class name when it decides if it should do load balancing. If it finds a match it will use your laod balancer to laod balance. If Ocelot cannot match the load balancer type in your configuration with name of registered load balancer class then you will recieve a HTTP 500 internal error. If your load balancer factory throw an exception when Ocelot calls it you will receiver a Http 500 internal server error.

Remember if you specify no load balancer in your config Ocelot will not try and load balance.

The random custom load balancer e.g.