如果你觉得这篇文章对你有帮助，请不要吝惜你的“关注”、“点赞”、“评价”，我们可以进一步讨论实现方案和细节。你的支持永远是我前进的动力~~~
你是不是遇到过，公司有自己的IDC机房，但运维能力较弱，部署的研发服务非常不稳定，因此大家都不愿意使用，转而在云上部署。
但是在云上部署，又引入了其他问题，最主要是本地和测试环境不能双向互联，联调时不能混合使用本地服务+测试环境进行方便的测试。
当遇到k8s时，问题就更加突出了，不管是在IDC还是云上环境，都不能轻易地做到双向互联。
现在有了新的解决方案：KtConnect
KtConnect提供了本地和测试环境集群的双向互联能力
项目地址：https://alibaba.github.io/kt-connect/#/
废话少说，直接上手用起来吧！！！
准备工作
安装，以windows为例
https://github.com/alibaba/kt-connect/releases/tag/v0.3.7
下载，解压到指定目录，设置环境变量
如解压到D盘，目录 D:\ktctl_0.3.7_Windows_x86_64
配置环境变量
ktctl 命令大全
打通本地和远程网络
以管理员身份运行终端，执行命令：ktctl connect，则本地和k8s集群会建立一个虚拟网络，k8s中的service、pod的ip，以及通过service的名字都是可以通过本地直接访问到的。
注意：因为我们的IDC网段与k8s service cicd网络有重合，开启以后devops部署平台和gitlab地址可能访问不了，可以添加参数排除掉：这样通过ip访问不了service，其他不影响
ktctl connect --excludeIps '192.168.0.0/16'
本地服务暴露并注册
mesh原理
ktctl mesh [service name] --expose [local:remote] --versionMark [versionMark]
例如: ktctl mesh tool-manager-2024-jar-changan-dev --expose 8080 --versionMark abc
ktctl mesh 命令参数：
远程流量路由至本地
安装Modheader浏览器插件，配置如下：流量即会路由到本地
远程debug
不需要暴露本地服务，直接连接到指定的pod ip即可
jvm启动参数上添加参数：
-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005
以我们真实的项目为例：
查pod ip
windows上安装docker desktop即可自动安装kubectl命令
本地配置好k8s的config文件
执行kubectl get po -owide即可查到查看到IP
查到的ip为172.28.16.123
或者在devops平台上查看
配置远程debug
端口为5055，忽略图中8000
到此可以在idea中设置断点，愉快的debug远程服务了
官方文档
https://rdc-incubator.github.io/kt-docs/#/zh-cn/quickstart
Quick Start Guide
In this chapter, we will deploy a demo app in Kubernetes cluster, use KtConnect to access it from local environment, and redirect request in cluster to a local service.
Create A Demo App In Cluster
Let's create a simple tomcat deployment, expose it as a service, and put a default html file as index page.
$ kubectl create deployment tomcat --image=tomcat:9 --port=8080
deployment.apps/tomcat created
$ kubectl expose deployment tomcat --port=8080 --target-port=8080
service/tomcat exposed
$ kubectl exec deployment/tomcat -c tomcat -- /bin/bash -c 'mkdir webapps/ROOT; echo "kt-connect demo v1" > webapps/ROOT/index.html'Fetch information of pod and service created above.
$ kubectl get pod -o wide --selector app=tomcat
NAME READY STATUS RESTARTS AGE IP ...
tomcat 1/1 Running 0 34s 10.51.0.162 ...
$ kubectl get svc tomcat
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
tomcat ClusterIP 172.16.255.111 <none> 8080/TCP 34sRemember the Pod IP(10.51.0.162) and Cluster IP(172.16.255.111) of tomcat service, they'll be used in following steps.
Access App From Local
KtConnect use command connectto let all cluster resources accessible from local (Root/Administrator permission is required).
** MacOS/Linux **
Execute ktctl connectcommand on Mac/Linux with sudo:
$ sudo ktctl connect
00:00AM INF KtConnect start at <PID>
... ...
00:00AM INF ---------------------------------------------------------------
00:00AM INF All looks good, now you can access to resources in the kubernetes cluster
00:00AM INF ---------------------------------------------------------------** Windows **
On Windows, execute ktctl connectcommand in terminal (If you are not login as Administrator, you should right-click on CMD and PowerShell icon, choose "Run as Administrator" to open a terminal):
> ktctl connect
00:00AM INF KtConnect start at <PID>
... ...
00:00AM INF ---------------------------------------------------------------
00:00AM INF All looks good, now you can access to resources in the kubernetes cluster
00:00AM INF ---------------------------------------------------------------Now any resource in the cluster can be directly accessed from local, you could use curlor web browser to verify.
Note：In Windows PowerShell, the curltool is conflict with a build-in command, please type curl.exeinstead of curl.
$ curl http://10.51.0.162:8080 # access Pod IP from local
kt-connect demo v1
$ curl http://172.21.6.39:8080 # access Cluster IP from local
kt-connect demo v1
$ curl http://tomcat:8080 # access Service via its name as domain name
kt-connect demo v1
$ curl http://tomcat.default:8080 # access Service via its name and namespace as domain name
kt-connect demo v1
$ curl http://tomcat.default.svc.cluster.local:8080 # access Service via fully qualified domain name
kt-connect demo v1Forward cluster traffic to local
In order to verify the scenarios where the service in cluster access services at local, we also start a Tomcat service locally and create an index page with different content.
$ docker run -d --name tomcat -p 8080:8080 tomcat:9
$ docker exec tomcat /bin/bash -c 'mkdir webapps/ROOT; echo "kt-connect local v2" > webapps/ROOT/index.html'KtConnect offers 2 commands for accessing local service from cluster in different cases.
-
Exchange：redirect all request to specified service in cluster to local
-
Mesh：redirect partial request to specified service in cluster (base on mesh rule) to local
** Exchange Command **
Intercept and forward all requests to the specified service in the cluster to specified local port, which is usually used to debug service in the test environment at the middle of an invocation-chain.
┌──────────┐ ┌─ ── ── ── ┌──────────┐
│ ServiceA ├─┬─►x│ ServiceB │ ┌─►│ ServiceC │
└──────────┘ │ ── ── ── ─┘ │ └──────────┘
exchange │
│ ┌──────────┐ │
└──►│ ServiceB'├─┘ (Local Instance)
└──────────┘Below command will transfer all the traffic of the tomcatservice previously deployed in the cluster to 8080port of developer's laptop.
$ ktctl exchange tomcat --expose 8080
00:00AM INF KtConnect start at <PID>
... ...
---------------------------------------------------------------
Now all request to service 'tomcat' will be redirected to local
---------------------------------------------------------------
... ...Visit the tomcatservice deployed to the cluster, and view the output result.
Note: you could also execute below command from any pod in the cluster to verify result, if you didn't run ktctl connectcommand locally.
$ curl http://tomcat:8080
kt-connect local v2The request to tomcatservice in the cluster is now routed to the local Tomcat instance, thus you can directly debug this service locally.
** Mesh Command **
Intercept and forward part of requests to the specified service in the cluster to specified local port. Usual used in team cooperation, one developer need to debug a service while do not want to disturb other developers.
┌──────────┐ ┌──────────┐ ┌──────────┐
│ ServiceA ├─┬──►│ ServiceB │─┬─►│ ServiceC │
└──────────┘ │ └──────────┘ │ └──────────┘
mesh │
│ ┌──────────┐ │
└──►│ ServiceB'├─┘ (Local Instance)
└──────────┘Mesh command has 2 execution modes. The default automode will automatically create corresponding routing rules for you withoutextra service mesh component.
In order to verify the results, firstly let's reset the content of the index page of the Tomcat service in the cluster, then use ktctl meshcommand to create a traffic rule:
$ kubectl exec deployment/tomcat -c tomcat -- /bin/bash -c 'mkdir webapps/ROOT; echo "kt-connect demo v1" > webapps/ROOT/index.html'
$ ktctl mesh tomcat --expose 8080
00:00AM INF KtConnect start at <PID>
... ...
--------------------------------------------------------------
Now you can access your service by header 'VERSION: feo3x'
--------------------------------------------------------------At the end of command log, a special HTTP Header is printed. Now if you access the tomcatservice as normal, the traffic would come into the original pods.
$ curl http://tomcat:8080
kt-connect demo v1If a request contains the HTTP Header shown by mesh command, the traffic will be redirected to local.
$ curl -H 'VERSION: feo3x' http://tomcat:8080
kt-connect local v2In actual use, it can be combined with the ModHeader plugin, so that only the requests made by developers from their browsers will access their local service processes.
The manualmode of mesh command provides possibility of more flexible route rule. In this mode, KtConnect will not automatically create the corresponding routing rules for you, the traffic accessing the service will randomly access the service in cluster and service at local. You can use any Service Mesh tool (such as Istio) to create routing rules based on the versionlabel of shadow pod to forward specific traffic to the local. Read Mesh Best Practicedoc for more detail
The most significant difference between ktctl meshand ktctl exchangecommands is that the latter will completely replace the original application instance, while the former will still retain the original service pod after the shadow pod is created, and the router pod will dynamically generate a versionheader (or label), so only specified traffic will be redirected to local, while ensuring that the normal traffic in the test environment is not effected.
Provide local service to others
In addition to the services that have been deployed to the cluster, during the development process, KtConnect can also be used to quickly "put" a local service to the cluster and turn it into a temporary service for other developers or other services in the cluster to use.
-
Preview: Register a local service as a service in the cluster
-
Forward: Redirect a local port to a cluster service. Can achieve to access services run on other developer's laptop via localhost while combining with previewcommand
** Preview Command **
Register a local service instance to cluster. Unlike the previous two commands, ktctl previewis mainly used to debug or preview a services under developing.
The following command will register the service running on the port 8080locally to the cluster as a service named tomcat-v2.
$ ktctl preview tomcat-v2 --expose 8080
00:00AM INF KtConnect start at <PID>
... ...
---------------------------------------------------------------
Now you can access your local service in cluster by name 'tomcat-v2'
---------------------------------------------------------------
... ...Now other services in the cluster can access the locally exposed service instance through the service name of tomcat-v2, and other developers can also preview the service directly through service name tomcat-v2after executing ktctl connecton their laptops.
$ curl http://tomcat-v2:8080
kt-connect local v2** Forward Command **
Redirect specified local port to any IP or service in the cluster. It is used to easily access a specific IP or service in the cluster using the localhostaddress during testing. The typical scenario is to access local services of other developers registered by previewcommand.
┌─────────────────────────────┐
forward | preview
┌────────┴───────┐ | ┌───────▼──────┐
│ localhost:8080 │ | │ local tomcat │
└────────────────┘ | └──────────────┘
Developer B | Developer AFor example, after a developer A runs the aforementioned previewcommand, another developer B can use the ktctl forwardcommand to map it to its own local 6060port.
$ ktctl forward tomcat-v2 6060:8080
00:00AM INF KtConnect start at <PID>
... ...
---------------------------------------------------------------
Now you can access port 8080 of service 'tomcat-v2' via 'localhost:6060'
---------------------------------------------------------------Now developer B can use the localhost:6060address to access the Tomcat service running locally by developer A.
When the forwarded traffic source is a service name in the cluster, the result is similar to the kubectl port-forward command, except the additional ability to automatically reconnect when the network is disconnected.
注意事项
kt-connect现在只对service生效，所以gateway的nacos配置不能使用lb，而要使用service
spring:
cloud:
gateway:
routes:
# 路由的ID,没有固定规则但要求唯一,建议配合服务名
# 配置为lb://xxxx ，则路由不生效
- id: tom
uri: http://tool-manager-2024-jar-changan-dev:8080
predicates:
- Path=/toolmanager/**
filters:
- StripPrefix=1
mesh方式对feign调用不生效，默认只对第一层生效
gateway -> tom -> iom
原因一：header请求头只会从gateway透传到tom， 从tom到iom时就丢失掉了
原因二：kt-connect现在只对service级别生效
因此要做链路上的改造才行， 有需要的可以联系我，可以提供全套的解决方案
路由不生效问题排查
ktctl birdseye 查看当前的service路由情况
ktctl clean 清除不可用的kt创建的资源， 可能就生效了
ktctl connect：长时间不用，本地与集群创建的链路可能会失效，需要重建连接
标签：实战,废话,8080,service,tomcat,ktctl,cluster,KtConnect,local From： https://blog.csdn.net/u013469646/article/details/142449945