模拟网络状况的有很多种,这里说的是Linux上的TC工具。
什么是TC
TC(traffic control)是Linux中的流量控制工具。它是通过控制netem来实现的网络场景模拟。该工具是直接对物理网卡生效的,如果是逻辑网卡,则该控制无效。如果是用的虚拟机,可视虚拟网卡为物理网卡。
什么是HTB
HTB是Hierarchy Token Bucket的缩写。通过在实践基础上的改进,它实现了一个丰富的连接共享类别体系。使用HTB可以很容易地保证每个类别的带宽,虽然它也允许特定的类可以突破带宽上限,占用别的类的带宽。HTB可以通过TBF(Token Bucket Filter)实现带宽限制,也能够划分类别的优先级。
TC使用步骤
要对网卡进行流量控制的配置,需要进行如下的步骤:
- 为网卡配置一个队列;
- 在该队列上建立分类;
- 根据需要建立子队列和子分类;
- 为每个分类建立过滤器;
- 建立与过滤器配合的路由表;
操作实例
正常情况下的网络表现
产生流量的语句: server端:
1. iperf -s
Client端:
1. iperf -c 172.17.211.143 -p 5001 -i 2 -P 5
产生的结果:
Server端:
1. [ 12] 0.0-10.1 sec 95.2 MBytes 79.3 Mbits/sec
2. [ 10] 0.0-10.1 sec 158 MBytes 131 Mbits/sec
3. [ 6] 0.0-10.1 sec 116 MBytes 95.7 Mbits/sec
4. [ 9] 0.0-10.2 sec 143 MBytes 118 Mbits/sec
5. [ 4] 0.0-10.3 sec 183 MBytes 150 Mbits/sec
6. [ 7] 0.0-10.3 sec 117 MBytes 96.0 Mbits/sec
7. [ 11] 0.0-10.3 sec 156 MBytes 127 Mbits/sec
8. [ 14] 0.0-10.3 sec 138 MBytes 113 Mbits/sec
9. [ 8] 0.0-10.3 sec 136 MBytes 111 Mbits/sec
10. [ 5] 0.0-10.3 sec 162 MBytes 132 Mbits/sec
11. [SUM] 0.0-10.3 sec 1.37 GBytes 1.14 Gbits/sec
Client端:
1. [ ID] Interval Transfer Bandwidth
2. [ 6] 0.0- 2.0 sec 88.9 MBytes 373 Mbits/sec
3. [ 4] 0.0- 2.0 sec 85.8 MBytes 360 Mbits/sec
4. [ 5] 0.0- 2.0 sec 68.2 MBytes 286 Mbits/sec
5. [ 7] 0.0- 2.0 sec 52.2 MBytes 219 Mbits/sec
6. [ 3] 0.0- 2.0 sec 92.5 MBytes 388 Mbits/sec
7. [SUM] 0.0- 2.0 sec 388 MBytes 1.63 Gbits/sec
8. [ 4] 2.0- 4.0 sec 62.4 MBytes 262 Mbits/sec
9. [ 6] 2.0- 4.0 sec 48.9 MBytes 205 Mbits/sec
10. [ 5] 2.0- 4.0 sec 27.4 MBytes 115 Mbits/sec
11. [ 3] 2.0- 4.0 sec 68.9 MBytes 289 Mbits/sec
12. [ 7] 2.0- 4.0 sec 44.6 MBytes 187 Mbits/sec
13. [SUM] 2.0- 4.0 sec 252 MBytes 1.06 Gbits/sec
14. [ 3] 4.0- 6.0 sec 45.5 MBytes 191 Mbits/sec
15. [ 5] 4.0- 6.0 sec 30.0 MBytes 126 Mbits/sec
16. [ 4] 4.0- 6.0 sec 54.8 MBytes 230 Mbits/sec
17. [ 6] 4.0- 6.0 sec 69.4 MBytes 291 Mbits/sec
18. [ 7] 4.0- 6.0 sec 53.1 MBytes 223 Mbits/sec
19. [SUM] 4.0- 6.0 sec 253 MBytes 1.06 Gbits/sec
20. [ 4] 6.0- 8.0 sec 40.4 MBytes 169 Mbits/sec
21. [ 6] 6.0- 8.0 sec 25.6 MBytes 107 Mbits/sec
22. [ 7] 6.0- 8.0 sec 76.1 MBytes 319 Mbits/sec
23. [ 3] 6.0- 8.0 sec 59.1 MBytes 248 Mbits/sec
24. [ 5] 6.0- 8.0 sec 38.2 MBytes 160 Mbits/sec
25. [SUM] 6.0- 8.0 sec 240 MBytes 1.00 Gbits/sec
26. [ 6] 8.0-10.0 sec 37.8 MBytes 158 Mbits/sec
27. [ 6] 0.0-10.0 sec 270 MBytes 227 Mbits/sec
28. [ 4] 8.0-10.0 sec 39.9 MBytes 167 Mbits/sec
29. [ 4] 0.0-10.1 sec 283 MBytes 234 Mbits/sec
30. [ 5] 8.0-10.0 sec 40.8 MBytes 171 Mbits/sec
31. [ 5] 0.0-10.1 sec 205 MBytes 169 Mbits/sec
32. [ 7] 8.0-10.0 sec 48.0 MBytes 201 Mbits/sec
33. [ 7] 0.0-10.1 sec 274 MBytes 227 Mbits/sec
34. [ 3] 8.0-10.0 sec 84.8 MBytes 355 Mbits/sec
35. [SUM] 8.0-10.0 sec 251 MBytes 1.05 Gbits/sec
36. [ 3] 0.0-10.2 sec 351 MBytes 289 Mbits/sec
37. [SUM] 0.0-10.2 sec 1.35 GBytes 1.14 Gbits/sec
做了多遍,都和上面的结果相似。5个线程加在一起每秒大概1Gbits左右。
模拟丢包
模拟命令
1. tc qdisc add dev eth0 root netem loss 10%
丢包率与带宽的关系
模拟丢包率 | 吞吐量 | 降幅 |
关闭模拟 | 1.14 Gbits/sec | |
1% | 1.14 Gbits/sec | 0 Mbits/sec |
2% | 602 Mbits/sec | 538 Mbits/sec |
3% | 335 Mbits/sec | 267 Mbits/sec |
4% | 215 Mbits/sec | 120 Mbits/sec |
5% | 130 Mbits/sec | 85 Mbits/sec |
6% | 79.9 Mbits/sec | 50.1 Mbits/sec |
7% | 58.9 Mbits/sec | 21 Mbits/sec |
8% | 39.7 Mbits/sec | 19.2 Mbits/sec |
9% | 24.7 Mbits/sec | 15 Mbits/sec |
10% | 19.3 Mbits/sec | 5.4 Mbits/sec |
模拟延迟
模拟命令
1. tc qdisc add dev eth0 root netem delay 100ms
延迟与带宽的关系
模拟延迟(按10%浮动) | 吞吐量 |
关闭模拟 | 1.14 Gbits/sec |
10 ms | 1.02 Gbits/sec |
20 ms | 483 Mbits/sec |
30 ms | 351 Mbits/sec |
40 ms | 254 Mbits/sec |
50ms | 201 Mbits/sec |
60 ms | 130 Mbits/sec |
70 ms | 116 Mbits/sec |
80 ms | 98.1 Mbits/sec |
90 ms | 92.0 Mbits/sec |
100 ms | 66.9 Mbits/sec |
HTB队列分层限制
分类配置
1. tc qdisc add dev eth0 root handle 1: htb default 2
2.
3. tc class add dev eth0 parent 1: classid 1:1 htb rate 100Mbps ceil 100Mbps
4. tc class add dev eth0 parent 1:1 classid 1:2 htb rate 20Mbps ceil 20Mbps
5. tc class add dev eth0 parent 1:1 classid 1:3 htb rate 50Mbps ceil 50Mbps
6. tc class add dev eth0 parent 1:1 classid 1:4 htb rate 20Mbps ceil 20Mbps
7.
8. tc filter add dev eth0 parent 1:0 protocol ip prio 100 route
9. tc filter add dev eth0 parent 1:0 protocol ip prio 100 route to 2 flowid 1:2
10. tc filter add dev eth0 parent 1:0 protocol ip prio 100 route to 3 flowid 1:3
11. tc filter add dev eth0 parent 1:0 protocol ip prio 100 route to 4 flowid 1:4
12.
13. ip route add 172.17.211.144 dev eth0 via 172.17.211.143 realm 2
14. [root@7dgroup ~]# tc -s class ls dev eth0
15. class htb 1:1 root rate 800000Kbit ceil 800000Kbit burst 1600b cburst 1600b
16. Sent 1350897 bytes 6146 pkt (dropped 0, overlimits 0 requeues 0)
17. rate 0bit 0pps backlog 0b 0p requeues 0
18. lended: 0 borrowed: 0 giants: 0
19. tokens: 234 ctokens: 234
20.
21. class htb 1:2 parent 1:1 prio 0 rate 160000Kbit ceil 160000Kbit burst 1600b cburst 1600b
22. Sent 1350897 bytes 6146 pkt (dropped 0, overlimits 0 requeues 0)
23. rate 0bit 0pps backlog 0b 0p requeues 0
24. lended: 5850 borrowed: 0 giants: 0
25. tokens: 1170 ctokens: 1170
26.
27. class htb 1:3 parent 1:1 prio 0 rate 400000Kbit ceil 400000Kbit burst 1600b cburst 1600b
28. Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
29. rate 0bit 0pps backlog 0b 0p requeues 0
30. lended: 0 borrowed: 0 giants: 0
31. tokens: 500 ctokens: 500
32.
33. class htb 1:4 parent 1:1 prio 0 rate 160000Kbit ceil 160000Kbit burst 1600b cburst 1600b
34. Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
35. rate 0bit 0pps backlog 0b 0p requeues 0
36. lended: 0 borrowed: 0 giants: 0
37. tokens: 1250 ctokens: 1250
限流效果
测试手段从A机往B机用iperf开5个线程发数据包。
- 不限制
1. [ 5] 0.0-10.0 sec 168 MBytes 140 Mbits/sec
2. [ 7] 0.0-10.1 sec 75.2 MBytes 62.4 Mbits/sec
3. [ 8] 0.0-10.2 sec 210 MBytes 172 Mbits/sec
4. [ 4] 0.0-10.3 sec 92.8 MBytes 75.7 Mbits/sec
5. [ 6] 0.0-10.3 sec 158 MBytes 129 Mbits/sec
6. [SUM] 0.0-10.3 sec 704 MBytes 574 Mbits/sec
- 限10M
7. [ 8] 0.0-10.2 sec 16.6 MBytes 13.7 Mbits/sec
8. [ 4] 0.0-10.2 sec 16.4 MBytes 13.5 Mbits/sec
9. [ 5] 0.0-10.2 sec 14.5 MBytes 11.9 Mbits/sec
10. [ 6] 0.0-10.2 sec 25.8 MBytes 21.2 Mbits/sec
11. [ 7] 0.0-10.2 sec 19.8 MBytes 16.2 Mbits/sec
12. [SUM] 0.0-10.2 sec 93.0 MBytes 76.4 Mbits/sec
- 限20M
13. [ 5] 0.0-10.1 sec 55.6 MBytes 46.0 Mbits/sec
14. [ 7] 0.0-10.2 sec 28.9 MBytes 23.8 Mbits/sec
15. [ 9] 0.0-10.2 sec 26.1 MBytes 21.6 Mbits/sec
16. [ 4] 0.0-10.2 sec 45.0 MBytes 37.1 Mbits/sec
17. [ 6] 0.0-10.2 sec 29.5 MBytes 24.3 Mbits/sec
18. [SUM] 0.0-10.2 sec 185 MBytes 153 Mbits/sec
- 限30M
19. [ 4] 0.0-10.2 sec 53.0 MBytes 43.7 Mbits/sec
20. [ 6] 0.0-10.2 sec 62.0 MBytes 51.1 Mbits/sec
21. [ 8] 0.0-10.2 sec 57.9 MBytes 47.7 Mbits/sec
22. [ 5] 0.0-10.2 sec 58.5 MBytes 48.2 Mbits/sec
23. [ 7] 0.0-10.2 sec 46.4 MBytes 38.2 Mbits/sec
24. [SUM] 0.0-10.2 sec 278 MBytes 229 Mbits/sec
- 限40M
25. [ 5] 0.0-10.1 sec 76.6 MBytes 63.5 Mbits/sec
26. [ 9] 0.0-10.1 sec 76.9 MBytes 63.6 Mbits/sec
27. [ 6] 0.0-10.1 sec 72.4 MBytes 59.9 Mbits/sec
28. [ 7] 0.0-10.1 sec 70.6 MBytes 58.5 Mbits/sec
29. [ 4] 0.0-10.1 sec 72.9 MBytes 60.3 Mbits/sec
30. [SUM] 0.0-10.1 sec 369 MBytes 305 Mbits/sec
- 限50M
31. [ 4] 0.0-10.1 sec 89.9 MBytes 74.5 Mbits/sec
32. [ 5] 0.0-10.1 sec 99.6 MBytes 82.5 Mbits/sec
33. [ 8] 0.0-10.1 sec 89.9 MBytes 74.3 Mbits/sec
34. [ 6] 0.0-10.1 sec 91.9 MBytes 76.0 Mbits/sec
35. [ 7] 0.0-10.2 sec 89.8 MBytes 74.1 Mbits/sec
36. [SUM] 0.0-10.2 sec 461 MBytes 381 Mbits/sec
- 限60M
37. [ 4] 0.0-10.1 sec 107 MBytes 89.1 Mbits/sec
38. [ 7] 0.0-10.1 sec 121 MBytes 101 Mbits/sec
39. [ 9] 0.0-10.1 sec 108 MBytes 89.3 Mbits/sec
40. [ 5] 0.0-10.1 sec 107 MBytes 89.1 Mbits/sec
41. [ 6] 0.0-10.1 sec 107 MBytes 89.2 Mbits/sec
42. [SUM] 0.0-10.1 sec 550 MBytes 457 Mbits/sec
- 限70M
43. [ 8] 0.0-10.1 sec 178 MBytes 148 Mbits/sec
44. [ 7] 0.0-10.1 sec 94.4 MBytes 78.5 Mbits/sec
45. [ 4] 0.0-10.1 sec 95.0 MBytes 78.9 Mbits/sec
46. [ 6] 0.0-10.1 sec 94.6 MBytes 78.6 Mbits/sec
47. [ 5] 0.0-10.1 sec 178 MBytes 148 Mbits/sec
48. [SUM] 0.0-10.1 sec 640 MBytes 531 Mbits/sec
- 限80M
49. [ 7] 0.0-10.0 sec 167 MBytes 140 Mbits/sec
50. [ 9] 0.0-10.1 sec 166 MBytes 137 Mbits/sec
51. [ 4] 0.0-10.2 sec 99.8 MBytes 82.4 Mbits/sec
52. [ 5] 0.0-10.2 sec 157 MBytes 129 Mbits/sec
53. [ 6] 0.0-10.2 sec 110 MBytes 90.2 Mbits/sec
54. [SUM] 0.0-10.2 sec 700 MBytes 574 Mbits/sec
- 限90M
55. [ 4] 0.0-10.0 sec 220 MBytes 184 Mbits/sec
56. [ 7] 0.0-10.2 sec 124 MBytes 102 Mbits/sec
57. [ 5] 0.0-10.2 sec 104 MBytes 85.2 Mbits/sec
58. [ 8] 0.0-10.2 sec 117 MBytes 96.2 Mbits/sec
59. [ 6] 0.0-10.2 sec 135 MBytes 111 Mbits/sec
60. [SUM] 0.0-10.2 sec 699 MBytes 573 Mbits/sec
- 限100M
61. [ 4] 0.0-10.1 sec 140 MBytes 116 Mbits/sec
62. [ 7] 0.0-10.1 sec 139 MBytes 116 Mbits/sec
63. [ 6] 0.0-10.1 sec 145 MBytes 121 Mbits/sec
64. [ 5] 0.0-10.1 sec 128 MBytes 106 Mbits/sec
65. [ 9] 0.0-10.1 sec 146 MBytes 121 Mbits/sec
66. [SUM] 0.0-10.1 sec 698 MBytes 579 Mbits/sec
有很多种手段可以模拟网络丢包、延迟、限流的情况,大家可以自行探索。