Kubernetes Gateway API
Gateway API 是 Kubernetes 1.19 版本引入的一种新的 API 规范,会成为 Ingress 的下一代替代方案。主要原因是 Ingress 资源对象不能很好的满足网络需求,很多场景下 Ingress 控制器都需要通过定义 annotations 或者 crd 来进行功能扩展,这对于使用标准和支持是非常不利的,新推出的 Gateway API 旨在通过可扩展的面向角色的接口来增强服务网络。
2022 年 5 月份Kubernetes Gateway API才发布了 Beta 版本,当前大多数组织应该还在使用稳定的 Ingress API。
官方地址:https://gateway-api.sigs.k8s.io/
官方仓库:https://github.com/kubernetes-sigs/gateway-api
安装 Gateway API CRDS
k8s 版本:v1.30.1
Gateway API 版本:v1.2.1
$ kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.2.1/standard-install.yaml
customresourcedefinition.apiextensions.k8s.io/gatewayclasses.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/gateways.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/grpcroutes.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/httproutes.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/referencegrants.gateway.networking.k8s.io created
$ kubectl api-resources | grep gateway
gatewayclasses gc gateway.networking.k8s.io/v1 false GatewayClass
gateways gtw gateway.networking.k8s.io/v1 true Gateway
grpcroutes gateway.networking.k8s.io/v1 true GRPCRoute
httproutes gateway.networking.k8s.io/v1 true HTTPRoute
referencegrants refgrant gateway.networking.k8s.io/v1beta1 true ReferenceGrant
安装 Gateway API Controller
Gateway API 只是安装了一些 crd,实现了上层 api,真正的 Gateway API 下游实现(包含 GatewayClass 资源),即基础设备供应商,类似 ingress-controller,支持列表参考 Gateway Controller - Kubernetes Gateway API
这里安装 Envoy Gateway API Controller 为例。
安装metallb LoadBalancer
首先安装metallb LoadBalancer,参考地址:https://metallb.universe.tf/installation/ :
# 首先 kube-proxy 如果使用 ipvs 模式,则启用严格 ARP:
$ cat /etc/systemd/system/kube-proxy.service
[Unit]
Description=Kubernetes Kube-Proxy Server
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=network.target
[Service]
WorkingDirectory=/var/lib/kube-proxy
ExecStart=/opt/kube/bin/kube-proxy \
--config=/var/lib/kube-proxy/kube-proxy-config.yaml
Restart=always
RestartSec=5
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
# strictARP: False 改为 strictARP: True
vi /var/lib/kube-proxy/kube-proxy-config.yaml
kind: KubeProxyConfiguration
apiVersion: kubeproxy.config.k8s.io/v1alpha1
bindAddress: 0.0.0.0
clientConnection:
kubeconfig: "/etc/kubernetes/kube-proxy.kubeconfig"
# 根据clusterCIDR 判断集群内部和外部流量,配置clusterCIDR选项后,kube-proxy 会对访问 Service IP 的请求做 SNAT
clusterCIDR: "172.20.0.0/16"
conntrack:
maxPerCore: 32768
min: 131072
tcpCloseWaitTimeout: 1h0m0s
tcpEstablishedTimeout: 24h0m0s
healthzBindAddress: 0.0.0.0:10256
# hostnameOverride 值必须与 kubelet 的对应一致,否则 kube-proxy 启动后会找不到该 Node,从而不会创建任何 iptables 规则
hostnameOverride: "k8s-10-10-10-147"
metricsBindAddress: 0.0.0.0:10249
mode: "ipvs"
ipvs:
excludeCIDRs: null
minSyncPeriod: 0s
scheduler: ""
strictARP: True
syncPeriod: 30s
tcpFinTimeout: 0s
tcpTimeout: 0s
udpTimeout: 0s
$ systemctl daemon-reload
$ systemctl restart kube-proxy.service
$ kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.14.8/config/manifests/metallb-native.yaml
namespace/metallb-system created
customresourcedefinition.apiextensions.k8s.io/bfdprofiles.metallb.io created
customresourcedefinition.apiextensions.k8s.io/bgpadvertisements.metallb.io created
customresourcedefinition.apiextensions.k8s.io/bgppeers.metallb.io created
customresourcedefinition.apiextensions.k8s.io/communities.metallb.io created
customresourcedefinition.apiextensions.k8s.io/ipaddresspools.metallb.io created
customresourcedefinition.apiextensions.k8s.io/l2advertisements.metallb.io created
customresourcedefinition.apiextensions.k8s.io/servicel2statuses.metallb.io created
serviceaccount/controller created
serviceaccount/speaker created
role.rbac.authorization.k8s.io/controller created
role.rbac.authorization.k8s.io/pod-lister created
clusterrole.rbac.authorization.k8s.io/metallb-system:controller created
clusterrole.rbac.authorization.k8s.io/metallb-system:speaker created
rolebinding.rbac.authorization.k8s.io/controller created
rolebinding.rbac.authorization.k8s.io/pod-lister created
clusterrolebinding.rbac.authorization.k8s.io/metallb-system:controller created
clusterrolebinding.rbac.authorization.k8s.io/metallb-system:speaker created
configmap/metallb-excludel2 created
secret/metallb-webhook-cert created
service/metallb-webhook-service created
deployment.apps/controller created
daemonset.apps/speaker created
validatingwebhookconfiguration.admissionregistration.k8s.io/metallb-webhook-configuration created
$ kubectl -n metallb-system get pod
NAME READY STATUS RESTARTS AGE
controller-6dd967fdc7-9vzk7 1/1 Running 0 4m9s
speaker-tfn99 1/1 Running 0 4m9s
- Controler:Deployment,用于监听 Service 的变更,分配/回收 IP 地址。
- Speaker:DaemonSet,对外广播 Service 的 IP 地址。
首先创建一个地址池:
# vi ip-pool.yaml
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
name: ip-pool
namespace: metallb-system
spec:
addresses:
- 10.10.10.200-10.10.10.245
- 注意:提供的 IP 段必须跟 k8s 集群主机所在同一个网段中,且尚未使用的 IP 段
创建一个广播声明,关联上面的 IP 池对象:
# vi advertise.yaml
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
name: l2adver
namespace: metallb-system
spec:
ipAddressPools:
- ip-pool
- 如果不设置关联到 IPAdressPool,L2Advertisement 默认会关联所有可用的 IPAdressPool
应用:
kubectl apply -f ip-pool.yaml
kubectl apply -f advertise.yaml
安装 Envoy Gateway API Controller
官方地址:https://gateway.envoyproxy.io/docs/tasks/quickstart/
helm install eg --create-namespace oci://docker.io/envoyproxy/gateway-helm --version v0.0.0-latest -n envoy-gateway-system --skip-crds
-
--version v0.0.0-latest 最新版本,也可以指定版本 --version v1.2.4
-
--skip-crds 跳过 Gateway API CRDS ,前面已经安装
$ kubectl -n envoy-gateway-system get all
NAME READY STATUS RESTARTS AGE
pod/envoy-gateway-7774bd75b7-ksk4z 1/1 Running 0 2m27s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/envoy-gateway ClusterIP 10.68.158.175 <none> 18000/TCP,18001/TCP,18002/TCP,19001/TCP 2m27s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/envoy-gateway 1/1 1 1 2m27s
NAME DESIRED CURRENT READY AGE
replicaset.apps/envoy-gateway-7774bd75b7 1 1 1 2m27s
创建 GatewayClass:
# vi GatewayClass.yaml
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
name: eg
spec:
controllerName: gateway.envoyproxy.io/gatewayclass-controller
创建 deployment 示例:
# vi backend.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: backend
---
apiVersion: v1
kind: Service
metadata:
name: backend
labels:
app: backend
service: backend
spec:
ports:
- name: http
port: 80
targetPort: 80
selector:
app: backend
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: backend
spec:
replicas: 1
selector:
matchLabels:
app: backend
version: v1
template:
metadata:
labels:
app: backend
version: v1
spec:
serviceAccountName: backend
containers:
- image: nginx:latest
imagePullPolicy: IfNotPresent
name: backend
ports:
- containerPort: 80
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
创建 Gateway 与 HTTPRoute
# vi frontend.yaml
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: eg
spec:
gatewayClassName: eg
listeners:
- name: http
protocol: HTTP
port: 80
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: backend
spec:
parentRefs:
- name: eg
hostnames:
- "www.example.com"
rules:
- backendRefs:
- group: ""
kind: Service
name: backend
port: 80
weight: 1
matches:
- path:
type: PathPrefix
value: /
应用:
$ kubectl apply -f GatewayClass.yaml
gatewayclass.gateway.networking.k8s.io/eg created
$ kubectl get Gatewayclass
NAME CONTROLLER ACCEPTED AGE
eg gateway.envoyproxy.io/gatewayclass-controller True 11s
$ kubectl apply -f backend.yaml
serviceaccount/backend created
service/backend created
deployment.apps/backend created
$ kubectl get all
NAME READY STATUS RESTARTS AGE
pod/backend-59c5fd8d48-qgd4k 1/1 Running 0 25s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/backend ClusterIP 10.68.184.237 <none> 80/TCP 25s
service/kubernetes ClusterIP 10.68.0.1 <none> 443/TCP 6h24m
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/backend 1/1 1 1 25s
NAME DESIRED CURRENT READY AGE
replicaset.apps/backend-59c5fd8d48 1 1 1 25s
$ kubectl apply -f frontend.yaml
gateway.gateway.networking.k8s.io/eg created
httproute.gateway.networking.k8s.io/backend created
# 可以看到配置完成后,由 GatewayClass 绑定的 Controller 为我们提供一个具体存在 Pod 作为流量入口,
# 需要注意的是,各家实现在此处还是略有不同,比如说 Envoy 当你创建 Gateway 资源后,Envoy Controller
# 会创建一个 Deployment 资源为你提供入口流量 Pod ,然而 Nginx 则是自己本身就是流量入口 Pod 不会创建新的。
$ kubectl -n envoy-gateway-system get pod
NAME READY STATUS RESTARTS AGE
envoy-default-eg-e41e7b31-5dbf9db8bf-bh5zd 2/2 Running 0 116s
envoy-gateway-7774bd75b7-ksk4z 1/1 Running 0 20m
$ kubectl -n envoy-gateway-system get deployment
NAME READY UP-TO-DATE AVAILABLE AGE
envoy-default-eg-e41e7b31 1/1 1 1 2m
envoy-gateway 1/1 1 1 20m
$ kubectl -n envoy-gateway-system get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
envoy-default-eg-e41e7b31 LoadBalancer 10.68.238.117 10.10.10.200 80:31762/TCP 2m4s
envoy-gateway ClusterIP 10.68.158.175 <none> 18000/TCP,18001/TCP,18002/TCP,19001/TCP 20m
$ kubectl get gateway
NAME CLASS ADDRESS PROGRAMMED AGE
eg eg 10.10.10.200 True 4m36s
$ kubectl get httproute
NAME HOSTNAMES AGE
backend ["www.example.com"] 4m38s
访问:
export GATEWAY_HOST=$(kubectl get gateway/eg -o jsonpath='{.status.addresses[0].value}')
curl --verbose --header "Host: www.example.com" http://$GATEWAY_HOST/
标签:metallb,Kubernetes,created,Gateway,API,io,k8s,gateway,backend
From: https://www.cnblogs.com/leffss/p/18623541