kunernetes中路由器反射模式下与外部路由器配置BGP,减少外部物理设备与K8S集群节点之间建立的BGP对等连接。如下为本次的拓扑图
1.配置K8S集群为路由器反射模式
1.1.关闭全互联模式
root@master:~/bgp# cat disable-node-mesh.yml
apiVersion: projectcalico.org/v3
kind: BGPConfiguration
metadata:
name: default
spec:
logSeverityScreen: Info
# 关闭calico默认的node-to-node mesh
nodeToNodeMeshEnabled: false
# 默认BGP集群的as号64512
asNumber: 64512
root@master:~/bgp# kubectl apply -f disable-node-mesh.yml
root@master:~/bgp#
1.2.配置反射路由器
root@master:~/bgp# kubectl label node master.sec.lab route-reflector=true
root@master:~/bgp# kubectl annotate node master.sec.lab projectcalico.org/RouteReflectorClusterID=244.0.0.1
root@master:~/bgp# cat peer-int-bgp.yml
apiVersion: projectcalico.org/v3
kind: BGPPeer
metadata:
name: peer-with-route-reflectors
spec:
nodeSelector: all()
peerSelector: route-reflector == 'true'
root@master:~/bgp# kubectl apply -f peer-int-bgp.yml
root@master:~/bgp#
1.3.查看反射路由器与各节点建立连接
# Master节点中
root@master:~/bgp# calicoctl node status
Calico process is running.
IPv4 BGP status
+--------------+---------------+-------+----------+-------------+
| PEER ADDRESS | PEER TYPE | STATE | SINCE | INFO |
+--------------+---------------+-------+----------+-------------+
| 10.22.4.12 | node specific | up | 05:20:48 | Established |
| 10.22.4.13 | node specific | up | 05:20:46 | Established |
+--------------+---------------+-------+----------+-------------+
IPv6 BGP status
No IPv6 peers found.
root@master:~/bgp#
# Node节点中
root@node01:~# calicoctl node status
Calico process is running.
IPv4 BGP status
+--------------+---------------+-------+----------+-------------+
| PEER ADDRESS | PEER TYPE | STATE | SINCE | INFO |
+--------------+---------------+-------+----------+-------------+
| 10.22.4.11 | node specific | up | 05:20:48 | Established |
+--------------+---------------+-------+----------+-------------+
IPv6 BGP status
No IPv6 peers found.
root@node01:~#
2.配置K8S反射路由器与外部路由建立BGP
2.1. 配置反射路由器
root@master:~/bgp# cat peer-ext-bgp.yml
apiVersion: projectcalico.org/v3
kind: BGPPeer
metadata:
name: bgppeer-to-external-device
spec:
nodeSelector: route-reflector == 'true'
peerIP: 10.22.4.1
keepOriginalNextHop: true
asNumber: 65002
root@master:~/bgp# kubectl apply -f peer-ext-bgp.yml
root@master:~/bgp#
2.2. 配置物理路由器
- 此处物理路由器为华为三层
<AR200>system-view
Enter system view, return user view with Ctrl+Z.
[AR200]
[AR200]bgp 65002
[AR200-bgp]display this
#
bgp 65002
router-id 10.22.1.1
peer 10.22.4.11 as-number 64512
peer 10.22.4.11 connect-interface Vlanif14
#
ipv4-family unicast
undo synchronization
default-route imported
import-route static
peer 10.22.4.11 enable
#
return
[AR200-bgp]
3.查看K8S BGP路由
3.1.查看邻居关系
- 在Master节点和Node节点
# 在Master节点上
root@master:~/bgp# calicoctl node status
Calico process is running.
IPv4 BGP status
+--------------+---------------+-------+----------+-------------+
| PEER ADDRESS | PEER TYPE | STATE | SINCE | INFO |
+--------------+---------------+-------+----------+-------------+
| 10.22.4.12 | node specific | up | 05:20:48 | Established |
| 10.22.4.13 | node specific | up | 05:20:46 | Established |
| 10.22.4.1 | node specific | up | 05:31:20 | Established |
+--------------+---------------+-------+----------+-------------+
IPv6 BGP status
No IPv6 peers found.
root@master:~/bgp#
# 在Node节点上
root@node01:~# calicoctl node status
Calico process is running.
IPv4 BGP status
+--------------+---------------+-------+----------+-------------+
| PEER ADDRESS | PEER TYPE | STATE | SINCE | INFO |
+--------------+---------------+-------+----------+-------------+
| 10.22.4.11 | node specific | up | 05:20:48 | Established |
+--------------+---------------+-------+----------+-------------+
IPv6 BGP status
No IPv6 peers found.
- 在华为路由器中查看
<AR200>display bgp peer
BGP local router ID : 10.22.1.1
Local AS number : 65002
Total number of peers : 1 Peers in established state : 1
Peer V AS MsgRcvd MsgSent OutQ Up/Down State PrefRcv
10.22.4.11 4 64512 27 25 0 00:19:05 Established 3
<AR200>
3.2.查看路由
- 在华为路由器中查看BGP路由
<AR200>display bgp routing-table
BGP Local router ID is 10.22.1.1
Status codes: * - valid, > - best, d - damped,
h - history, i - internal, s - suppressed, S - Stale
Origin : i - IGP, e - EGP, ? - incomplete
Total Number of Routes: 4
Network NextHop MED LocPrf PrefVal Path/Ogn
*> 0.0.0.0 0.0.0.0 0 0 ?
*> 10.244.25.0/24 10.22.4.13 0 64512i
*> 10.244.182.0/24 10.22.4.12 0 64512i
*> 10.244.187.0/24 10.22.4.11 0 64512i
<AR200>