kubeshark的介绍

在底层实现当中，Kubeshark主要使用到了Linux内核中的各种内置方法和API，隐藏了对流量数据的加解密实现，可以直接收集到K8s集群中的加密和未加密流量。对网络数据的收集主要使用了直接抓包法和基于拓展伯克利包过滤（eBPF）的数据包获取。直接抓包法涉及libpcap、AF_PACKET和PF_RING获取集群TCP流量，eBPF的方法主要使用到了加密流量 (TLS)，并挂钩到OpenSSL库和 Go 的crypto/tls包中某些函数的入口点和出口点。

kubeshark的部署

安装kubeshark

从https://github.com/kubeshark/kubeshark/releases/下载需要的版本。
本次测试过程，下载的版本是38.3

在k8s的master上执行

curl -Lo kubeshark https://github.com/kubeshark/kubeshark/releases/download/38.3/kubeshark_linux_amd64 && chmod 755 kubeshark
cp kubeshark /usr/bin/
kubeshark version

准备kubeshark的服务镜像

将自己本地的镜像导入到环境，并打tag推送到自己镜像仓库

docker tag harbor.voneyun.com/kubeshark/hub:latest harbor.shengydt.com/prod-meta/hub:latest
docker tag harbor.voneyun.com/kubeshark/worker:latest harbor.shengydt.com/prod-meta/worker:latest
docker tag harbor.voneyun.com/kubeshark/front:latest harbor.shengydt.com/prod-meta/front:latest
docker push harbor.shengydt.com/prod-meta/hub:latest
docker push harbor.shengydt.com/prod-meta/worker:latest
docker push harbor.shengydt.com/prod-meta/front:latest

制作配置文件

mkdir /root/.kubeshark
vi config.yaml 
tap:
    docker:
        registry: harbor.shengydt.com/prod-meta
        tag: latest
        imagepullpolicy: IfNotPresent
        imagepullsecrets: []
    proxy:
        worker:
            port: 8897
            srvport: 8897
        hub:
            port: 8898
            srvport: 80
        front:
            port: 8899
            srvport: 80
            nodeport: 8899
        host: 0.0.0.0
    regex: .*
    namespaces: []
    allnamespaces: false
    storagelimit: 200MB
    dryrun: false
    pcap: ""
    resources:
        worker:
            cpu-limit: 750m
            memory-limit: 1Gi
            cpu-requests: 50m
            memory-requests: 50Mi
        hub:
            cpu-limit: 750m
            memory-limit: 1Gi
            cpu-requests: 50m
            memory-requests: 50Mi
    servicemesh: true
    tls: true
    packetcapture: libpcap
    debug: false
logs:
    file: ""
kube:
    configpath: ""
    context: ""
selfnamespace: kubeshark
dumplogs: false
headless: false

启动kubeshark

kubeshark clean
kubeshark tap -n prod-meta 

kubeshark的使用

配置nginx代理到kubeshark

cat kubeshark.conf 
upstream  kubeshark8899 {
          server 172.16.80.50:8899;
              }
server {
    listen       80;
    listen       443 ssl;
    server_name  tmp.shengydt.com;

    ssl_certificate /usr/local/nginx/cert/shengydt.com.pem;
    ssl_certificate_key /usr/local/nginx/cert/shengydt.com.key;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
    ssl_ciphers TLS13-AES-256-GCM-SHA384:TLS13-CHACHA20-POLY1305-SHA256:TLS13-AES-128-GCM-SHA256:TLS13-AES-128-CCM-8-SHA256:TLS13-AES-128-CCM-SHA256:EECDH+CHACHA20:EECDH+AES128:RSA+AES128:EECDH+AES256:RSA+AES256:EECDH+3DES:RSA+3DES:!MD5;
    ssl_prefer_server_ciphers on;
    ssl_session_timeout 10m;
    ssl_session_cache builtin:1000 shared:SSL:10m;
    ssl_buffer_size 1400;
    add_header Strict-Transport-Security max-age=15768000;
    ssl_stapling on;
    ssl_stapling_verify on;
    if ($ssl_protocol = "") { return 301 https://$host$request_uri; }    

    # 日志中时间戳设置固定格式：2023-09-28 15:57:01
    if ($time_iso8601 ~ "^(\d{4})-(\d{2})-(\d{2})T(\d{2}):(\d{2}):(\d{2})") {
                set $year $1;
                set $month $2;
                set $day $3;
                set $hour $4;
                set $minutes $5;
                set $seconds $6;
    }


    location / {
                proxy_set_header   X-Real-IP        $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header Host $http_host;
                proxy_set_header X-Nginx-Proxy true;
                proxy_pass http://kubeshark8899;
                proxy_set_header X-Forwarded-Proto $scheme;
               access_log   /prod-meta-logs/nginx-logs/tmp.access.log json;
               error_log    /prod-meta-logs/nginx-logs/tmp.error.log ;
}


}

添加本地解析，使用域名访问：

流量监控