Enhancing K8s Gateway API with Easegress Without Changing a Single Line of Code

In the article “Revolutionize Your Kubernetes Experience with Easegress: Kubernetes Gateway API”, we explored the powerful capabilities of the Kubernetes Gateway API. Today, we will present how to use the flexibility of Kubernetes Gateway to enhance its functionalities by using existing filters and resilience policies in Easegress without changing a single line of code.

Through this article, you will learn how to equip the Kubernetes Gateway API with resilient fault-tolerance capabilities without modifying any code.

Why Enhance the K8s Gateway API?

We already know that Easegress possesses robust resilient fault-tolerance features, including circuit breaking, rate limiting, and retries. With these features, Easegress can effectively protect backend services. However, in the current Kubernetes Gateway API standards, the protection mechanisms for backend services are not clearly defined. The standards are more about traffic forwarding, load balancing, redirection, and so on. So, how can we implement protection for backend services in Kubernetes Gateway? How can we equip the Kubernetes Gateway API with capabilities like circuit breaking, rate limiting, and retries? This is the key question we need to explore today.

Kubernetes Gateway ExtensionRef: The Glue Between Kubernetes and Easegress

First, let’s understand how the Kubernetes Gateway API, through the ingenious configuration of ExtensionRef [1], provides a way to implement custom functionalities. Below is an example of an HTTPRoute, demonstrating how to reference resources within a cluster:"

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: httproute-extension
spec:
  rules:
  - matches:
    - path:
        value: /test
    filters: 
    - type: ExtensionRef
      # Referencing the FilterSpec resource through ExtensionRef.
      extensionRef:
        group: "easegress.megaease.com"
        kind: "FilterSpec"
        name: "rate-limiter"
    backendRefs:
    - name: service-a
      port: 8080

This ExtensionRef references a ‘FilterSpec’ resource named ‘rate-limiter’ in the ’easegress.megaease.com’ group. This configuration will be recognized by the Easegress Gateway Controller [2] and transformed into the corresponding Easegress settings. This expands the functionality of the Kubernetes Gateway API, enabling the HTTPRoute to have rate limiting capabilities.

Custom Resource Definitions: Balancing Security and Flexibility

To seamlessly integrate the advanced functionalities of Easegress, we chose Custom Resource Definition (CRD) as our solution. Compared to directly using ConfigMap, it has a smaller impact and offers better flexibility. Below is the corresponding CRD configuration:

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: filterspecs.easegress.megaease.com
spec:
  group: easegress.megaease.com
  versions:
    - name: v1
      served: true
      storage: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                name:
                  type: string
                kind:
                  type: string
                spec:
                  type: string
  scope: Namespaced
  names:
    plural: filterspecs
    singular: filterspec
    kind: FilterSpec

In this CustomResourceDefinition, we defined the ’easegress.megaease.com’ group and the ‘FilterSpec’ kind. Our definition is designed with compatibility in mind, retaining only the three most essential attributes: name, kind, and spec. Where name and kind are common to all Easegress Filters, and spec is the specific configuration of the Filter, where the corresponding yaml configuration can be placed for use.

Practical Exercise

Next, we will take RateLimiter [3] and ResponseAdaptor [4] as examples, which are two of the many Filters provided by Easegress.

First, let’s create the corresponding Kubernetes resources:

apiVersion: easegress.megaease.com/v1
kind: FilterSpec
metadata:
  name: rate-limiter
spec:
  name: rate-limiter
  kind: RateLimiter
  spec: |
    policies:
    - name: policy
      limitRefreshPeriod: 5000ms
      limitForPeriod: 1
    defaultPolicyRef: policy
    urls:
    - url:
        prefix: /
      policyRef: policy    

---

apiVersion: easegress.megaease.com/v1
kind: FilterSpec
metadata:
  name: response-adaptor
spec:
  name: response-adaptor
  kind: ResponseAdaptor
  spec: |
    header:
      add: 
        X-Eg-Response-Adaptor: "true"    

This RateLimiter allows only one request to pass in a 5-second period. The ResponseAdaptor adds an X-Eg-Response-Adaptor header to the HTTP response.

To use these extensions in HTTPRoute, you simply need to reference these Filters when creating the HTTPRoute. A specific example is as follows:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: example-route-2
spec:
  parentRefs:
  - kind: Gateway
    name: example-gateway
    sectionName: example-listener
  rules:
  - matches:
    - path:
        value: /test
    filters: 
    - type: ExtensionRef
      extensionRef:
        # use rate-limiter
        group: "easegress.megaease.com"
        kind: "FilterSpec"
        name: "rate-limiter"
    - type: ExtensionRef
      extensionRef:
        # use response-adaptor
        group: "easegress.megaease.com"
        kind: "FilterSpec"
        name: "response-adaptor"
    backendRefs:
    - name: hello-service
      port: 60002

Thus, after creating this HTTPRoute, our Easegress Gateway Controller will incorporate the specified rate limiter and response adaptor by reference. This endows the HTTPRoute with the capabilities of rate limiting and response modification.

Next, we perform some simple tests. The environment we use is minikube, and we map the port of the Gateway to nodePort 30081. Then we login for testing using minikube ssh. More details on the configuration can be found in our official documentation [2].

docker@minikube:~$ curl http://127.0.0.1:30081/test -v 
...
< Date: Thu, 23 Nov 2023 02:57:59 GMT
< X-Eg-Response-Adaptor: true  # ResponseAdaptor works
< Connection: close
< 
Hello, world!
Version: 2.0.0
Hostname: hello-deployment-688d8666c-xl9sb
* Closing connection 0



docker@minikube:~$ curl http://127.0.0.1:30081/test -v 
...
< HTTP/1.1 429 Too Many Requests
< X-Eg-Rate-Limiter: too-many-requests  # RateLimiter works
< Date: Thu, 23 Nov 2023 02:58:00 GMT
...

Our test results show that the first request is successful and includes the X-Eg-Response-Adaptor header, while the second request is rejected due to the effect of the rate limiter.

Circuit Breaker and Retry Strategies

Furthermore, we have also provided definitions for circuit breakers and retry strategies [5], further enhancing the resilience and reliability of the network.

apiVersion: easegress.megaease.com/v1
kind: FilterSpec
metadata:
  name: circuit-breaker
spec:
  name: circuit-breaker
  kind: CircuitBreaker
  spec: |
    slidingWindowType: TIME_BASED
    failureRateThreshold: 60
    slidingWindowSize: 200    

--- 

apiVersion: easegress.megaease.com/v1
kind: FilterSpec
metadata:
  name: retry
spec:
  name: retry
  kind: Retry
  spec: |
    maxAttempts: 3
    waitDuration: 500ms    

Through this method, we can easily acquire various advanced functionalities of Easegress in Kubernetes Gateway.

[1] Kubernetes Gateway ExtensionRef https://gateway-api.sigs.k8s.io/reference/spec/#gateway.networking.k8s.io/v1.LocalObjectReference
[2] Easegress Gateway Controller https://github.com/megaease/easegress/blob/main/docs/04.Cloud-Native/4.2.Gateway-API.md
[3] Easegress RateLimiter Filter https://github.com/megaease/easegress/blob/main/docs/07.Reference/7.02.Filters.md#ratelimiter
[4] Easegress ResponseAdaptor Filter https://github.com/megaease/easegress/blob/main/docs/07.Reference/7.02.Filters.md#responseadaptor
[5] Easegress Resilience https://github.com/megaease/easegress/blob/main/docs/02.Tutorials/2.4.Resilience.md