helm 安装ingress-nginx

Ingress-Nginx Controller 支持多种方式安装：

• 使用heml安装chart
• 使用kubectl apply，使用YAML文件；

详情可参考：https://kubernetes.github.io/ingress-nginx/deploy/

本文实践使用helm安装ingress-nginx

环境信息

# k8s版本
root@master1:~# kubectl get node
NAME      STATUS   ROLES           AGE    VERSION
master1   Ready    control-plane   169m   v1.28.2
node1     Ready    <none>          168m   v1.28.2
node2     Ready    <none>          168m   v1.28.2

# helm版本
root@master1:~# helm version
version.BuildInfo{Version:"v3.15.2", GitCommit:"1a500d5625419a524fdae4b33de351cc4f58ec35", GitTreeState:"clean", GoVersion:"go1.22.4"}

[root@k8s-master1 ~]# cat /etc/os-release

# 操作系统和内核版本
root@master1:~# cat /etc/issue
Ubuntu 24.04 LTS \n \l

root@master1:~# uname -r
6.8.0-31-generic

根据配套关系，我这里的k8s 1.28.2安装Ingress-nginx 1.10.1版本。

helm 安装 ingress-nginx

下载chart包

# 添加仓库
[root@master1 ~]# helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
"ingress-nginx" has been added to your repositories

# 更新
[root@master1 ~]# helm repo update

# 查询ingress-nginx的chart包，会查询到当前ingress-nginx最新的chart包
[root@master1 ~]# helm search repo ingress-nginx
NAME                            CHART VERSION   APP VERSION     DESCRIPTION                                       
ingress-nginx/ingress-nginx     4.10.1           1.10.1         Ingress controller for Kubernetes using NGINX a...

# 下载chart包
[root@master1 ~]# mkdir ingress-nginx
[root@master1 ~]# cd ingress-nginx/
[root@master1 ingress-nginx]# helm pull ingress-nginx/ingress-nginx
[root@master1 ingress-nginx]# ls
ingress-nginx-4.10.1.tgz

修改values.yaml文件

# 解压chart包
[root@k8s-master1 ingress-nginx]# tar xf ingress-nginx-4.10.1.tgz
# 查看chart目录结构
[root@k8s-master1 ingress-nginx]# ls ingress-nginx/
changelog  Chart.yaml  ci  OWNERS  README.md  README.md.gotmpl  templates  tests  values.yaml
# 修改values.yaml文件
[root@k8s-master1 ingress-nginx]# vi ingress-nginx/values.yaml

根据自己的部署的需求修改如下几个位置：

1. 修改镜像仓库地址。以 ingress-nginx-contorller 的为例，默认是registry.k8s.io，如果无法访问，可以改为内部或国内的仓库地址。

controller:
  name: controller
  enableAnnotationValidations: false
  image:
    ## Keep false as default for now!
    chroot: false
    registry: registry.k8s.io
    image: ingress-nginx/controller
    ## for backwards compatibility consider setting the full image url via the repository value below
    ## use *either* current default registry/image or repository format or installing chart by providing the values.yaml will fail
    ## repository:
    tag: "v1.10.1"
    digest: sha256:e24f39d3eed6bcc239a56f20098878845f62baa34b9f2be2fd2c38ce9fb0f29e

备注：

本文环境所需的镜像是通过有代理的机器拉取后，离线导入到k8s环境中。

2. 修改hostNetwork

  # -- Required for use with CNI based kubernetes installations (such as ones set up by kubeadm),
  # since CNI and hostport don't mix yet. Can be deprecated once https://github.com/kubernetes/kubernetes/issues/23920
  # is merged
  hostNetwork: true
  ## Use host ports 80 and 443
  ## Disabled by default

3. 修改dnsPolicy

  # -- Optionally change this to ClusterFirstWithHostNet in case you have 'hostNetwork: true'.
  # By default, while using host network, name resolution uses the host's DNS. If you wish nginx-controller
  # to keep resolving names inside the k8s network, use ClusterFirstWithHostNet.
  dnsPolicy: ClusterFirstWithHostNet

4. 添加label，用于部署调度。

  nodeSelector:
    kubernetes.io/os: linux
    ingress: "true"

如果只希望ingress-nginx部署到特定的几个节点，可以进行改设置。

5. 修改部署类型为

  # -- Use a `DaemonSet` or `Deployment`
  kind: DaemonSet
  # -- Annotations to be added to the controller Deployment or DaemonSet
  ##

本文部署的环境只修改了hostNetwork、dnsPolicy和kind，其他保持默认。

部署chart

# 创建ns
[root@k8s-master1 ingress-nginx]# kubectl create ns ingress-nginx
namespace/ingress-nginx created

# 在chart目录下执行安装
[root@k8s-master1 ingress-nginx]# ls
changelog  Chart.yaml  ci  OWNERS  README.md  README.md.gotmpl  templates  tests  values.yaml
[root@k8s-master1 ingress-nginx]# helm install ingress-nginx -n ingress-nginx .

NAME: ingress-nginx
LAST DEPLOYED: Mon Jul  1 18:20:48 2024
NAMESPACE: ingress-nginx
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
The ingress-nginx controller has been installed.
It may take a few minutes for the load balancer IP to be available.
You can watch the status by running 'kubectl get service --namespace ingress-nginx ingress-nginx-controller --output wide --watch'

An example Ingress that makes use of the controller:
  apiVersion: networking.k8s.io/v1
  kind: Ingress
  metadata:
    name: example
    namespace: foo
  spec:
    ingressClassName: nginx
    rules:
      - host: www.example.com
        http:
          paths:
            - pathType: Prefix
              backend:
                service:
                  name: exampleService
                  port:
                    number: 80
              path: /
    # This section is only required if TLS is to be enabled for the Ingress
    tls:
      - hosts:
        - www.example.com
        secretName: example-tls

If TLS is enabled for the Ingress, a Secret containing the certificate and key must also be provided:

  apiVersion: v1
  kind: Secret
  metadata:
    name: example-tls
    namespace: foo
  data:
    tls.crt: <base64 encoded cert>
    tls.key: <base64 encoded key>
  type: kubernetes.io/tls
  
# 查看创建的资源
root@master1:~/ingress-nginx# kubectl get all -n ingress-nginx
NAME                                 READY   STATUS    RESTARTS   AGE
pod/ingress-nginx-controller-dddq8   1/1     Running   0          2m49s
pod/ingress-nginx-controller-p7gd9   1/1     Running   0          2m49s

NAME          TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)              AGE
service/ingress-nginx-controller  LoadBalancer   10.102.230.242   <pending>     80:30974/TCP,443:30194/TCP   2m49s
service/ingress-nginx-controller-admission ClusterIP      10.110.51.58     <none>  443/TCP           2m49s

NAME              DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR     AGE
daemonset.apps/ingress-nginx-controller   2      2     2     2      2        kubernetes.io/os=linux   2m49s

# 本次部署不涉及外部LB，修改service/ingress-nginx-controller类型为NodePort
root@master1:~/ingress-nginx# kubectl edit -n ingress-nginx service/ingress-nginx-controller
...
  type: NodePort
...

创建实例测试 Ingress

my-nginx.yaml配置文件：

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-nginx
spec:
  selector:
    matchLabels:
      run: my-nginx
  replicas: 1
  template:
    metadata:
      labels:
        run: my-nginx
    spec:
      containers:
      - name: my-nginx
        image: nginx:latest
        resources:
          limits:
            memory: "128Mi"
            cpu: "500m"
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-service
spec:
  selector:
    run: my-nginx
  type: ClusterIP
  ports:
  - protocol: TCP
    port: 8080
    targetPort: 80
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: example-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
  ingressClassName: nginx
  rules:
    - host: test.ingress.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: nginx-service
                port:
                  number: 8080

创建 Deployment、service、ingress：

[root@master1 ingress-nginx]# kubectl create -f my-nginx.yaml
deployment.apps/my-nginx created
service/nginx-service created
ingress.networking.k8s.io/example-ingress created

nginx-ingress-controller所在的节点与域名做好hosts绑定，访问后端pod服务：

# 查看ingress-nginx部署的节点
root@master1:~/ingress-nginx# kubectl get pod -n ingress-nginx -o wide
NAME                             READY   STATUS    RESTARTS   AGE     IP             NODE    NOMINATED NODE   READINESS GATES
ingress-nginx-controller-dddq8   1/1     Running   0       9m29s   192.168.0.62   node1   <none>        <none>
ingress-nginx-controller-p7gd9   1/1     Running   0       9m29s   192.168.0.63   node2   <none>       <none>


# 在master节点上添加域名解析，域名指向nginx-ingress-controller所在节点的ip
[rootmaster ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.0.62 test.ingress.com

# curl测试
[root@master1 ~]# curl test.ingress.com
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

问题记录：

访问ingress域名，提示404：

root@master1:~/ingress-nginx# curl http://test.ingress.com
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx</center>
</body>
</html>

首先测试直接访问pod、service返回都正常，后来网上查到类似案例，修改ingress增加 ingressClassName: nginx：

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: example-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
  ingressClassName: nginx		# 增加的部分
  rules:
    - host: test.ingress.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: nginx-service
                port:
                  number: 8080

# 更新
kubectl apply -f my-nginx.yaml