首页 > 编程语言 >k8s service ipvs模式下nodePort实现

k8s service ipvs模式下nodePort实现

时间：2023-11-11 13:23:24浏览次数：44  
标签：11 00 service 0x00 29 192.168 0c ipvs nodePort

部署nodePort+StatefulSet

apiVersion: v1
kind: Service
metadata:
  name: nginx
spec:
  ports:
  - port: 80
  selector:
    app: nginx
  type: NodePort
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: nginx
spec:
  podManagementPolicy: Parallel
  serviceName: nginx
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.9.1

启用iptables trace调试

iptables -t raw -A PREROUTING -p tcp -s 192.168.0.105 -d 192.168.0.104 --dport 31531 -j TRACE
iptables -t raw -A OUTPUT -p tcp -s 192.168.0.105 -d 192.168.0.104 --dport 31531 -j TRACE
Nov 11 11:57:16 slave kernel: TRACE: raw:PREROUTING:policy:2 IN=ens33 OUT= MAC=00:0c:29:d0:ff:9b:00:0c:29:9e:20:65:08:00 SRC=192.168.0.105 DST=192.168.0.104 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=56832 DF PROTO=TCP SPT=51422 DPT=31531 SEQ=3403463061 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A000F67950000000001030307)
Nov 11 11:57:16 slave kernel: TRACE: mangle:PREROUTING:policy:1 IN=ens33 OUT= MAC=00:0c:29:d0:ff:9b:00:0c:29:9e:20:65:08:00 SRC=192.168.0.105 DST=192.168.0.104 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=56832 DF PROTO=TCP SPT=51422 DPT=31531 SEQ=3403463061 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A000F67950000000001030307)
Nov 11 11:57:16 slave kernel: TRACE: nat:PREROUTING:rule:1 IN=ens33 OUT= MAC=00:0c:29:d0:ff:9b:00:0c:29:9e:20:65:08:00 SRC=192.168.0.105 DST=192.168.0.104 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=56832 DF PROTO=TCP SPT=51422 DPT=31531 SEQ=3403463061 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A000F67950000000001030307)
Nov 11 11:57:16 slave kernel: TRACE: nat:KUBE-SERVICES:rule:2 IN=ens33 OUT= MAC=00:0c:29:d0:ff:9b:00:0c:29:9e:20:65:08:00 SRC=192.168.0.105 DST=192.168.0.104 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=56832 DF PROTO=TCP SPT=51422 DPT=31531 SEQ=3403463061 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A000F67950000000001030307)
Nov 11 11:57:16 slave kernel: TRACE: nat:KUBE-NODE-PORT:rule:1 IN=ens33 OUT= MAC=00:0c:29:d0:ff:9b:00:0c:29:9e:20:65:08:00 SRC=192.168.0.105 DST=192.168.0.104 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=56832 DF PROTO=TCP SPT=51422 DPT=31531 SEQ=3403463061 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A000F67950000000001030307)
Nov 11 11:57:16 slave kernel: TRACE: nat:KUBE-MARK-MASQ:rule:1 IN=ens33 OUT= MAC=00:0c:29:d0:ff:9b:00:0c:29:9e:20:65:08:00 SRC=192.168.0.105 DST=192.168.0.104 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=56832 DF PROTO=TCP SPT=51422 DPT=31531 SEQ=3403463061 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A000F67950000000001030307)
Nov 11 11:57:16 slave kernel: TRACE: nat:KUBE-MARK-MASQ:return:2 IN=ens33 OUT= MAC=00:0c:29:d0:ff:9b:00:0c:29:9e:20:65:08:00 SRC=192.168.0.105 DST=192.168.0.104 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=56832 DF PROTO=TCP SPT=51422 DPT=31531 SEQ=3403463061 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A000F67950000000001030307) MARK=0x4000
Nov 11 11:57:16 slave kernel: TRACE: nat:KUBE-NODE-PORT:return:2 IN=ens33 OUT= MAC=00:0c:29:d0:ff:9b:00:0c:29:9e:20:65:08:00 SRC=192.168.0.105 DST=192.168.0.104 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=56832 DF PROTO=TCP SPT=51422 DPT=31531 SEQ=3403463061 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A000F67950000000001030307) MARK=0x4000
Nov 11 11:57:16 slave kernel: TRACE: nat:KUBE-SERVICES:return:4 IN=ens33 OUT= MAC=00:0c:29:d0:ff:9b:00:0c:29:9e:20:65:08:00 SRC=192.168.0.105 DST=192.168.0.104 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=56832 DF PROTO=TCP SPT=51422 DPT=31531 SEQ=3403463061 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A000F67950000000001030307) MARK=0x4000
Nov 11 11:57:16 slave kernel: TRACE: nat:PREROUTING:rule:2 IN=ens33 OUT= MAC=00:0c:29:d0:ff:9b:00:0c:29:9e:20:65:08:00 SRC=192.168.0.105 DST=192.168.0.104 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=56832 DF PROTO=TCP SPT=51422 DPT=31531 SEQ=3403463061 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A000F67950000000001030307) MARK=0x4000
Nov 11 11:57:16 slave kernel: TRACE: nat:DOCKER:return:2 IN=ens33 OUT= MAC=00:0c:29:d0:ff:9b:00:0c:29:9e:20:65:08:00 SRC=192.168.0.105 DST=192.168.0.104 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=56832 DF PROTO=TCP SPT=51422 DPT=31531 SEQ=3403463061 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A000F67950000000001030307) MARK=0x4000
Nov 11 11:57:16 slave kernel: TRACE: nat:PREROUTING:policy:3 IN=ens33 OUT= MAC=00:0c:29:d0:ff:9b:00:0c:29:9e:20:65:08:00 SRC=192.168.0.105 DST=192.168.0.104 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=56832 DF PROTO=TCP SPT=51422 DPT=31531 SEQ=3403463061 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A000F67950000000001030307) MARK=0x4000
Nov 11 11:57:16 slave kernel: TRACE: mangle:INPUT:policy:1 IN=ens33 OUT= MAC=00:0c:29:d0:ff:9b:00:0c:29:9e:20:65:08:00 SRC=192.168.0.105 DST=192.168.0.104 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=56832 DF PROTO=TCP SPT=51422 DPT=31531 SEQ=3403463061 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A000F67950000000001030307) MARK=0x4000
Nov 11 11:57:16 slave kernel: TRACE: filter:INPUT:rule:1 IN=ens33 OUT= MAC=00:0c:29:d0:ff:9b:00:0c:29:9e:20:65:08:00 SRC=192.168.0.105 DST=192.168.0.104 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=56832 DF PROTO=TCP SPT=51422 DPT=31531 SEQ=3403463061 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A000F67950000000001030307) MARK=0x4000
Nov 11 11:57:16 slave kernel: TRACE: filter:KUBE-SERVICES:return:1 IN=ens33 OUT= MAC=00:0c:29:d0:ff:9b:00:0c:29:9e:20:65:08:00 SRC=192.168.0.105 DST=192.168.0.104 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=56832 DF PROTO=TCP SPT=51422 DPT=31531 SEQ=3403463061 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A000F67950000000001030307) MARK=0x4000
Nov 11 11:57:16 slave kernel: TRACE: filter:INPUT:rule:2 IN=ens33 OUT= MAC=00:0c:29:d0:ff:9b:00:0c:29:9e:20:65:08:00 SRC=192.168.0.105 DST=192.168.0.104 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=56832 DF PROTO=TCP SPT=51422 DPT=31531 SEQ=3403463061 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A000F67950000000001030307) MARK=0x4000
Nov 11 11:57:16 slave kernel: TRACE: filter:KUBE-EXTERNAL-SERVICES:return:1 IN=ens33 OUT= MAC=00:0c:29:d0:ff:9b:00:0c:29:9e:20:65:08:00 SRC=192.168.0.105 DST=192.168.0.104 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=56832 DF PROTO=TCP SPT=51422 DPT=31531 SEQ=3403463061 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A000F67950000000001030307) MARK=0x4000
Nov 11 11:57:16 slave kernel: TRACE: filter:INPUT:rule:3 IN=ens33 OUT= MAC=00:0c:29:d0:ff:9b:00:0c:29:9e:20:65:08:00 SRC=192.168.0.105 DST=192.168.0.104 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=56832 DF PROTO=TCP SPT=51422 DPT=31531 SEQ=3403463061 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A000F67950000000001030307) MARK=0x4000
Nov 11 11:57:16 slave kernel: TRACE: filter:KUBE-FIREWALL:return:3 IN=ens33 OUT= MAC=00:0c:29:d0:ff:9b:00:0c:29:9e:20:65:08:00 SRC=192.168.0.105 DST=192.168.0.104 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=56832 DF PROTO=TCP SPT=51422 DPT=31531 SEQ=3403463061 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A000F67950000000001030307) MARK=0x4000
Nov 11 11:57:16 slave kernel: TRACE: filter:INPUT:policy:8 IN=ens33 OUT= MAC=00:0c:29:d0:ff:9b:00:0c:29:9e:20:65:08:00 SRC=192.168.0.105 DST=192.168.0.104 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=56832 DF PROTO=TCP SPT=51422 DPT=31531 SEQ=3403463061 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A000F67950000000001030307) MARK=0x4000
Nov 11 11:57:16 slave kernel: TRACE: raw:OUTPUT:policy:2 IN= OUT=cni0 SRC=192.168.0.105 DST=10.16.1.12 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=56832 DF PROTO=TCP SPT=51422 DPT=80 SEQ=3403463061 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A000F67950000000001030307) MARK=0x4000
Nov 11 11:57:16 slave kernel: TRACE: mangle:OUTPUT:policy:1 IN= OUT=cni0 SRC=192.168.0.105 DST=10.16.1.12 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=56832 DF PROTO=TCP SPT=51422 DPT=80 SEQ=3403463061 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A000F67950000000001030307) MARK=0x4000
Nov 11 11:57:16 slave kernel: TRACE: filter:OUTPUT:rule:1 IN= OUT=cni0 SRC=192.168.0.105 DST=10.16.1.12 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=56832 DF PROTO=TCP SPT=51422 DPT=80 SEQ=3403463061 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A000F67950000000001030307) MARK=0x4000
Nov 11 11:57:16 slave kernel: TRACE: filter:KUBE-SERVICES:return:1 IN= OUT=cni0 SRC=192.168.0.105 DST=10.16.1.12 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=56832 DF PROTO=TCP SPT=51422 DPT=80 SEQ=3403463061 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A000F67950000000001030307) MARK=0x4000
Nov 11 11:57:16 slave kernel: TRACE: filter:OUTPUT:rule:2 IN= OUT=cni0 SRC=192.168.0.105 DST=10.16.1.12 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=56832 DF PROTO=TCP SPT=51422 DPT=80 SEQ=3403463061 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A000F67950000000001030307) MARK=0x4000
Nov 11 11:57:16 slave kernel: TRACE: filter:KUBE-FIREWALL:return:3 IN= OUT=cni0 SRC=192.168.0.105 DST=10.16.1.12 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=56832 DF PROTO=TCP SPT=51422 DPT=80 SEQ=3403463061 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A000F67950000000001030307) MARK=0x4000
Nov 11 11:57:16 slave kernel: TRACE: filter:OUTPUT:policy:4 IN= OUT=cni0 SRC=192.168.0.105 DST=10.16.1.12 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=56832 DF PROTO=TCP SPT=51422 DPT=80 SEQ=3403463061 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A000F67950000000001030307) MARK=0x4000
Nov 11 11:57:16 slave kernel: TRACE: mangle:POSTROUTING:policy:2 IN= OUT=cni0 SRC=192.168.0.105 DST=10.16.1.12 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=56832 DF PROTO=TCP SPT=51422 DPT=80 SEQ=3403463061 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A000F67950000000001030307) MARK=0x4000
Nov 11 11:57:16 slave kernel: TRACE: nat:POSTROUTING:rule:1 IN= OUT=cni0 SRC=192.168.0.105 DST=10.16.1.12 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=56832 DF PROTO=TCP SPT=51422 DPT=80 SEQ=3403463061 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A000F67950000000001030307) MARK=0x4000
Nov 11 11:57:16 slave kernel: TRACE: nat:KUBE-POSTROUTING:rule:3 IN= OUT=cni0 SRC=192.168.0.105 DST=10.16.1.12 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=56832 DF PROTO=TCP SPT=51422 DPT=80 SEQ=3403463061 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A000F67950000000001030307) MARK=0x4000
Nov 11 11:57:16 slave kernel: TRACE: nat:KUBE-POSTROUTING:rule:4 IN= OUT=cni0 SRC=192.168.0.105 DST=10.16.1.12 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=56832 DF PROTO=TCP SPT=51422 DPT=80 SEQ=3403463061 ACK=0 WINDOW=29200 RES=0x00 SYN URGP=0 OPT (020405B40402080A000F67950000000001030307)

转发路径分析

prerouting链
nat表KUBE-NODE-PORT-TCP链ipset有NodePort 31531

ipvs工作在input链，目的IP+目的端口匹配后经过ipvs完成负载均衡转发。

标签：11,00,service,0x00,29,192.168,0c,ipvs,nodePort
From： https://www.cnblogs.com/WJQ2017/p/17825814.html

相关文章

  • vue-cli-service vue.config.js配置 productionSourceMap与webpack中的devtool 关联详
    https://webpack.js.org/configuration/devtool/https://cli.vuejs.org/zh/config/#productionsourcemap https://github.com/vuejs/vue-cli/blob/f0f254e4bc81ed322eeb9f7de346e987e845068e/packages/%40vue/cli-service/lib/config/prod.js#L7 可以在源码中看到if(pro......
  • Android入门教程 | 四大组件之Service（前台服务，后台服务）
    Service是一种可在后台执行长时间运行操作而不提供界面的应用组件。服务可由其他应用组件启动，而且即使用户切换到其他应用，服务仍将在后台继续运行。此外，组件可通过绑定到服务与之进行交互，甚至是执行进程间通信(IPC)。例如，服务可在后台处理网络事务、播放音乐，执行文件I/O或与内......
  • CXF学习笔记(1)-HelloWorld!-发布webservice
    1.apache网站下载CXF http://cxf.apache.org/download.html 最新版本2.4.12.创建一个java工程，将以下jar包复制到工程的classpath下 所有的jar包都可以在${CXF_HOME}\lib目录中找到3.定义服务接口HelloWorldService因为这个接口将会被我们暴露为webservice,所以给该......
  • pojo层、dao层、service层、controller层的作用
    分层解耦介绍1.pojo层（model）实体层数据库在项目中的类model是模型的意思，与entity、domain、pojo类似，是存放实体的类。类中定义了多个类属性，并与数据库表的字段保持一致，一张表对应一个model类。主要用于定义与数据库对象应的属性，提供get/set方法,tostring方法,有参无参构造函......
  • Adding empty space into SQL Server Reporting Services
    REF:http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=1400080&SiteID=1(1)Ratherthaninsertingspaces,canyouadjustthepaddingpropertyonthetextbox?Ifyougoalistohavethefieldindented,thatshouldwork.(2)Anyreasonyoucouldn'......
  • How to grant permissions to a custom assembly that is referenced in a report in
    Ref:http://support.microsoft.com/kb/842419/en-us HowtograntpermissionstoacustomassemblythatisreferencedinareportinReportingServicesViewproductsthatthisarticleappliesto.ArticleID:842419LastReview:April3,2007Revision:2.1OnThisPa......
  • .net core webapi Startup services.AddHttpClient
    staticpublicclassCreditScoreServiceExtension{staticpublicvoidAddCreditScoreQueryServiceHttpClient(thisIServiceCollectionservices,IConfigurationconfig){services.AddSingleton<ICreditScoreQueryService,CreditS......
  • ServiceAccount ClusterRole ClusterRoleBinding
    RoleBinding的作用是把ServiceAccount绑定到Role上，Role规定了可以对资源做的操作，把ServiceAccount绑定到Role上就表示拿到这个ServiceAccount的程序就有了权限对资源做这些操作。当然，有ClusterRole和ClusterRoleBinding，ClusterRole可以在包括所有NameSpce和集群级别的资源或非资......
  • 【Azure Function App】解决Function App For Container 遇见ServiceUnavailable的异
    问题描述在使用Terraform创建FunctionApp后，部署函数时候遇见 ServiceUnavailable（BadRequest-- Encounteredanerror(ServiceUnavailable)fromhostruntime.）问题。查看FunctionApp的高级工具(Kudu)站点和默认站点，均出现ApplicationError页面。 问题解答查看Function......
  • 【Azure Function App】解决Function App For Container 遇见ServiceUnavailable的异
    问题描述在使用Terraform创建FunctionApp后，部署函数时候遇见 ServiceUnavailable（BadRequest-- Encounteredanerror(ServiceUnavailable)fromhostruntime.）问题。查看FunctionApp的高级工具(Kudu)站点和默认站点，均出现ApplicationError页面。 问题解答查看F......

赞助商

阅读排行

网站主页关于我们联系我们网站地图

本网站内容转载自其他媒体，侵权联系[admin##ips99.com]。

Copyright © 2020-2023 IPS99 版权所有 IPS99