Nginx——Nginx生产环境压测与性能优化实战

标签：实战 http nginx 压测 Completed Nginx requests 80

摘要

nginx的主要作用有三个方面：1、作为 Web 服务器；2、负载均衡服务器；3、邮件代理服务器等三个方面。其特点是占有内存少，并发能力强，给使用者带来了很多的便利。nginx是一款轻量级的Web 服务器/反向代理服务器及电子邮件（IMAP/POP3）代理服务器，在BSD-like 协议下发行。但是在日常的工作中，很多同学接触不到nginx服务和设计，更加谈不上学习nginx在系统中的应用设计。在大公司中nginx服务基本有公司基础架构组的同学负责。小公司基本上对nginx的优化与性能提升不关心。本博文将详细的介绍真实企业中Nginx在生产环境压测与性能优化实战，给想要学习的同学提供一个参考。

一、生产环境机器配置

系统	内核	线程	内存	硬盘容量
Ubuntu 20.4	2个	4线程	16G	500G

二、Nginx的安装

2.1 Nginx的直接安装

ubuntun自动安装nginx

apt-get update

apt-get install nginx

2.2 Nginx的docker部署

docker pull nginx

docker images

建目录用于存放nginx配置文件、证书文件

mkdir /opt/docker/nginx/conf.d -p
touch /opt/docker/nginx/conf.d/nginx.conf
mkdir /opt/docker/nginx/cert -p

vim /opt/docker/nginx/conf.d/nginx.conf

1、不需要SSL的情况

server {
    listen 80;  # 监听80端口
    server_name example.com www.example.com;  # 自己的域名
    
    location / {
        proxy_set_header HOST $host;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

        proxy_pass http://127.0.0.1:8090;  # 需要代理的地址:端口
    }
}


2、需要SSL的情况

如果不需要访问http的时候强制重定向为https，可以用下面的配置
# 非强制重定向https
server {
    listen 80; #侦听80端口，如果强制所有的访问都必须是HTTPs的，这行需要注销掉
    listen 443 ssl; #侦听443端口，用于SSL
    server_name example.cn www.example.cn;  # 自己的域名
    # 注意文件位置，是从/etc/nginx/下开始算起的
    ssl_certificate 1_example_bundle.crt;
    ssl_certificate_key 2_example.key;
    ssl_session_timeout 5m;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:HIGH:!aNULL:!MD5:!RC4:!DHE;
    ssl_prefer_server_ciphers on;

    client_max_body_size 1024m;

    location / {
        proxy_set_header HOST $host;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
	    # 这里写的是我的腾讯云内网地址,不知道为啥,不能用127.0.0.1...
        proxy_pass http://xxx.xx.xx.xx:8090;
    }
}

如果需要访问http的时候强制重定向为https，可以用下面的配置

# 强制重定向
server {
    listen 443 ssl;
    server_name example.com www.example.com;  # 自己的域名
    # 注意文件位置，是从/etc/nginx/下开始算起的
    ssl_certificate 1_www.example.com_bundle.crt;
    ssl_certificate_key 2_www.example.com.key;
    ssl_session_timeout 5m;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:HIGH:!aNULL:!MD5:!RC4:!DHE;
    ssl_prefer_server_ciphers on;

    client_max_body_size 1024m;

    location / {
        proxy_set_header HOST $host;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        # 这里写的是我的腾讯云内网地址,不知道为啥,不能用127.0.0.1...
        proxy_pass http://172.17.0.8:9090;
    }
}
server {
     listen 80; # 监听80端口
     server_name example.com www.example.com;  # 绑定证书的域名
     #把http的域名请求转成https
     return 301 https://$host$request_uri; 
}

docker run -itd --name nginx -p 80:80 -p 443:443 -v /opt/docker/nginx/conf.d/nginx.conf:/etc/nginx/conf.d/nginx.conf -v /opt/docker/nginx/cert:/etc/nginx -m 100m nginx

动后，输入docker ps 查看是否启动成功！docker logs nginx 查看日志。

三、Nginx压测工具的安装

3.1 webbench安装（Linux）

webbench官网：http://home.tiscali.cz/~cz210552/webbench.html

yum install -y gcc ctags

mkdir -m 644 -p /usr/local/man/man1

wget http://www.ha97.com/code/webbench-1.5.tar.gz

tar zxvf webbench-1.5.tar.gz

cd webbench-1.5

make

make install

webbench使用

-c client数量

-t time测试时间

webbench -c 300 -t 30 http://xxx.xxx.xxx.xxx/BBS/login.html（目标地址）

3.2 Apache bench安装

Centos：yum -y install httpd-tools

Ubuntu： sudo apt-get install apache2-utils

# 查询相关版本

ab -V

root@ubuntu:/home/xjl# ab -V
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/


# 对目标ip 进访问 c表示单次访问量 n表示总的访问量

ab -c 100 -n 1000  https://targetip/

四、Nginx未优化下的生产环境压测试验

当单次访问超过100的时候压测的结果

root@ubuntu:/home/xjl# ab -c 100 -n 10000 http://192.168.25.144:80/index.html
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.25.144 (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Completed 10000 requests
Finished 10000 requests


Server Software:        nginx/1.14.0
Server Hostname:        192.168.25.144
Server Port:            80

Document Path:          /index.html
Document Length:        178 bytes

Concurrency Level:      100
Time taken for tests:   0.706 seconds
Complete requests:      10000
Failed requests:        0
Non-2xx responses:      10000
Total transferred:      3370000 bytes
HTML transferred:       1780000 bytes
Requests per second:    14154.44 [#/sec] (mean)
Time per request:       7.065 [ms] (mean)
Time per request:       0.071 [ms] (mean, across all concurrent requests)
Transfer rate:          4658.25 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    3   0.8      3       6
Processing:     1    4   0.9      4       8
Waiting:        0    3   1.0      3       7
Total:          1    7   0.7      7      10

Percentage of the requests served within a certain time (ms)
  50%      7
  66%      7
  75%      7
  80%      7
  90%      8
  95%      8
  98%      8
  99%      8
 100%     10 (longest request)

当单次访问超过1000的时候压测的结果

root@ubuntu:/home/xjl# ab -c 1000 -n 10000 http://192.168.25.144:80/index.html
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.25.144 (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Completed 10000 requests
Finished 10000 requests


Server Software:
Server Hostname:        192.168.25.144
Server Port:            80

Document Path:          /index.html
Document Length:        0 bytes

Concurrency Level:      1000
Time taken for tests:   0.796 seconds
Complete requests:      10000
Failed requests:        10000
   (Connect: 0, Receive: 0, Length: 9936, Exceptions: 64)
Non-2xx responses:      9936
Total transferred:      3348432 bytes
HTML transferred:       1768608 bytes
Requests per second:    12563.18 [#/sec] (mean)
Time per request:       79.598 [ms] (mean)
Time per request:       0.080 [ms] (mean, across all concurrent requests)
Transfer rate:          4108.10 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       13   35   6.9     35      57
Processing:    10   41  10.8     41      78
Waiting:        0   29  10.9     26      65
Total:         40   76   7.0     77      92

Percentage of the requests served within a certain time (ms)
  50%     77
  66%     79
  75%     80
  80%     81
  90%     83
  95%     84
  98%     84
  99%     85
 100%     92 (longest request)

当单次访问超过1024的时候压测的结果

root@ubuntu:/home/xjl# ab -c 2000 -n 10000 http://192.168.25.144:80/index.html
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.25.144 (be patient)
socket: Too many open files (24)

五、 Nginx优化下的生产环境压测试验

5.1 Nginx的服务出错

Nginx服务器访问量非常高，在Nginx的错误日志中不停的输出如下错误信息。

2020-07-23 02:53:49 [alert] 13576#0: accept() failed (24: Too many open files)
2020-07-23 02:53:49 [alert] 13576#0: accept() failed (24: Too many open files)
2020-07-23 02:53:49 [alert] 13576#0: accept() failed (24: Too many open files)
2020-07-23 02:53:49 [alert] 13576#0: accept() failed (24: Too many open files)
2020-07-23 02:53:49 [alert] 13576#0: accept() failed (24: Too many open files)
2020-07-23 02:53:49 [alert] 13576#0: accept() failed (24: Too many open files)
2020-07-23 02:53:49 [alert] 13576#0: accept() failed (24: Too many open files)
2020-07-23 02:53:49 [alert] 13576#0: accept() failed (24: Too many open files)
2020-07-23 02:53:49 [alert] 13576#0: accept() failed (24: Too many open files)
2020-07-23 02:53:49 [alert] 13576#0: accept() failed (24: Too many open files)

根据错误日志的输出信息，我们可以看出：是打开的文件句柄数太多了，导致Nginx报错了！既然我们能够从Nginx的错误日志中基本能够确定导致问题的原因，那这到底是不是Nginx本身的问题呢？答案为：是，也不全是！

原因很简单：Nginx无法打开那么多的文件句柄，一方面是因为我没有配置Nginx能够打开的最大文件数；另一方面是因为Ubuntu 20.4操作系统本身对打开的最大文件句柄数有限制，我同样没有配置操作系统的最大文件句柄数。所以说，不全是Nginx的锅！

我们可以在命令行输入如下命令来查看服务器默认配置的最大文件句柄数。

[root@binghe150 ~]# ulimit -n

1024

此时，当Nginx的连接数超过1024时，Nginx的错误日志中就会输出如下错误信息：

[alert] 13576#0: accept() failed (24: Too many open files)

5.2 Nginx临时增大连接数

使用如下命令可以把打开文件句柄数设置的足够大。

ulimit -n 655350

同时修改nginx.conf ，添加如下配置项。

worker_rlimit_nofile 655350;

注意：上述配置需要与error_log同级别。这样就可以解决Nginx连接过多的问题，Nginx就可以支持高并发（这里需要配置Nginx）。另外， ulimit -n还会影响到MYSQL的并发连接数。把它提高，也可以提高MySQL的并发。注意：用 ulimit -n 655350 修改只对当前的shell有效，退出后失效。

5.3 Nginx 永久增大连接数

若要令修改ulimits的数值永久生效，则必须修改配置文件，可以给ulimit修改命令放入/etc/profile里面，这个方法实在是不方便。还有一个方法是修改/etc/security/limits.conf配置文件，如下所示。

vim /etc/security/limits.conf

在文件最后添加如下配置项。

* soft nofile 655360
* hard nofile 655360

保存并退出vim编辑器。其中：星号代表全局， soft为软件，hard为硬件，nofile为这里指可打开的文件句柄数。

最后，需要注意的是：要使 limits.conf 文件配置生效，必须要确保 pam_limits.so 文件被加入到启动文件中。查看 /etc/pam.d/login 文件中是否存在如下配置。不存在，则需要添加上述配置项。

session required /lib64/security/pam_limits.so

5.4 Nginx优化总结

1、开启Linux系统epoll支持：epoll支持，能够大大提高系统网络IO的并发数。

2、扩大Linux文件句柄数限制：Nginx代理过程，将服务器业务请求数据缓存到本地文件，再将文件数据转发给请求客户端。高并发的客户端请求，必然要求服务器文件句柄的并发打开限制。

5.5 Nginx优化下的生产环境压测试验结果

root@ubuntu:/etc/nginx# ab -c 2000 -n 10000 http://192.168.25.144:80/index.html
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.25.144 (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Completed 10000 requests
Finished 10000 requests


Server Software:
Server Hostname:        192.168.25.144
Server Port:            80

Document Path:          /index.html
Document Length:        0 bytes

Concurrency Level:      2000
Time taken for tests:   0.783 seconds
Complete requests:      10000
Failed requests:        9952
   (Connect: 0, Receive: 0, Length: 7392, Exceptions: 2560)
Non-2xx responses:      7440
Total transferred:      2507280 bytes
HTML transferred:       1324320 bytes
Requests per second:    12766.24 [#/sec] (mean)
Time per request:       156.663 [ms] (mean)
Time per request:       0.078 [ms] (mean, across all concurrent requests)
Transfer rate:          3125.83 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0   66  14.9     67      95
Processing:    30   74  19.2     77     122
Waiting:        0   43  27.6     54     101
Total:         93  140  12.1    141     163

Percentage of the requests served within a certain time (ms)
  50%    141
  66%    147
  75%    149
  80%    150
  90%    153
  95%    156
  98%    161
  99%    162
 100%    163 (longest request)

root@ubuntu:/etc/nginx# ab -c 20000 -n 100000 http://192.168.25.144:80/index.html
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.25.144 (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:
Server Hostname:        192.168.25.144
Server Port:            80

Document Path:          /index.html
Document Length:        0 bytes

Concurrency Level:      20000
Time taken for tests:   56.537 seconds
Complete requests:      100000
Failed requests:        99839
   (Connect: 0, Receive: 0, Length: 12499, Exceptions: 87340)
Non-2xx responses:      12499
Total transferred:      4212163 bytes
HTML transferred:       2224822 bytes
Requests per second:    1768.75 [#/sec] (mean)
Time per request:       11307.399 [ms] (mean)
Time per request:       0.565 [ms] (mean, across all concurrent requests)
Transfer rate:          72.76 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:      273 7456 3218.9   8366   11155
Processing:    99 2761 3066.8    694   11195
Waiting:        0  379 1603.1      0   10953
Total:       6381 10217 2645.7  10197   21013

Percentage of the requests served within a certain time (ms)
  50%  10197
  66%  11118
  75%  11231
  80%  11422
  90%  11455
  95%  16615
  98%  20414
  99%  20810
 100%  21013 (longest request)

root@ubuntu:/etc/nginx# ab -c 30000 -n 100000 http://192.168.25.144:80/index.html
ab: Invalid Concurrency [Range 0..20000]
Usage: ab [options] [http[s]://]hostname[:port]/path

参考博文

Nginx并发访问优化 - 腾讯云开发者社区-腾讯云

【Nginx】并发量太高，Nginx扛不住？这次我错怪Nginx了！！ - 腾讯云开发者社区-腾讯云

webbench网站压力测试工具 | Mingor的在线笔记本s

标签：实战,http,nginx,压测,Completed,Nginx,requests,80
From： https://blog.51cto.com/u_13643065/6169262