首页 > 系统相关 >nginx日志按天分割

nginx日志按天分割

时间:2024-05-29 11:43:46浏览次数:16  
标签:hjdang web 05 request 按天 nginx 日志 com

原帖是这样的:

nginx 日志分割是比较常见的运维工作,关于这方面的文章也很多,通常无外乎两种做法:

cron定期执行shell脚本对日志文件进行归档。
使用专门日志归档logrotate。
以上方式与nginx其实没有特别的关系。 从nginx 0.7.6 版本开始,access_log 的路径配置可以包含变量,我们以此进行日志分割。

同时我们基于nginx的 timeiso8601 内嵌变量来获取时间。time_iso8601格式如下:

2018-09-21T16:01:02+02:00
然后使用正则表达式来获取所需时间的数据。

http {
  log_format default_format '$remote_addr - $remote_user [$time_iso8601] "$request" '
      '$status $body_bytes_sent "$http_referer" '
      '"$http_user_agent" "$http_x_forwarded_for" "$request_body"';
      
      
      server {
          listen 443 ssl;
          ... ...
          if ($time_iso8601 ~ '(\d{4}-\d{2}-\d{2})') {
            set $tttt $1;
          }
          
          access_log /log/blog_access_$tttt.log;
      }
}
主要关注两个地方:

要在外方法的log_format上添加 $time_iso8601 ,将原来的time_local修改为time_iso8601。
在单个server中,通过正则表达式截取 $time_iso8601 生成时间戳。
最后插一个话题,你知道 $time_iso8601 为什么叫这个名字吗?

国际标准ISO 8601,是国际标准化组织的日期和时间的表示方法,全称为《数据存储和交换形式·信息交换·日期和时间的表示方法》。目前是2004年12月1日发行的第三版“ISO8601:2004”以替代1998年的第一版“ISO8601:1998”与2000年的第二版“ISO8601:2000”。

  照着上面的做法试了下,发现了两个问题

2024/05/29 05:38:09 [warn] 2773055#2773055: *24990740 using uninitialized "tttt" variable while logging request, client: 185.15.178.156, server: hjdang.com, request: "GET /coupon HTTP/1.1", host: "hjdang.com"
2024/05/29 05:38:12 [warn] 2773056#2773056: *24990759 using uninitialized "tttt" variable while logging request, client: 178.171.65.26, server: hjdang.com, request: "GET /d/1782431195418632474 HTTP/1.1", host: "hjdang.com"
2024/05/29 05:38:19 [warn] 2773053#2773053: *24990776 using uninitialized "tttt" variable while logging request, client: 178.171.67.37, server: hjdang.com, request: "GET /d/1782431195418632265 HTTP/1.1", host: "hjdang.com"

1. 出现了上面的警告,原因是nginx访问 access_log这个参数是在定义了$ttt之前的,正确的配置是把access_log配置在location下面:

    location / {
        access_log  /data/log/nginx/www.hjdang_$tttt.log  main;
        proxy_pass   http://127.0.0.1:3000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        send_timeout 600;
        proxy_connect_timeout 300;
        proxy_send_timeout 300;
        proxy_read_timeout 300;
        proxy_buffer_size 128k;
        proxy_buffers 2 256k;
        proxy_busy_buffers_size 256k;
        proxy_temp_file_write_size 256k;
        client_max_body_size 10m;
        add_header Cache-Control "no-store,max-age=0";

    }

 

2. error.log发现下面错误:

2024/05/29 03:49:55 [error] 2732888#2732888: *24973771 testing "/etc/nginx/html" existence failed (2: No such file or directory) while logging request, client: 111.230.194.84, server: web.hjdang.com, request: "POST /goods/detail HTTP/1.1", upstream: "http://127.0.0.1:8238/goods/detail", host: "web.hjdang.com"
2024/05/29 03:50:00 [error] 2732888#2732888: *24973775 testing "/etc/nginx/html" existence failed (2: No such file or directory) while logging request, client: 122.96.46.53, server: web.hjdang.com, request: "POST /yunpan/resource/list HTTP/1.1", upstream: "http://127.0.0.1:8238/yunpan/resource/list", host: "web.hjdang.com", referrer: "https://www.hjdang.com/category/%E5%AD%A6%E4%B9%A0?q="
2024/05/29 03:50:00 [error] 2732888#2732888: *24973775 testing "/etc/nginx/html" existence failed (2: No such file or directory) while logging request, client: 122.96.46.53, server: web.hjdang.com, request: "POST /yunpan/top/article/list HTTP/1.1", upstream: "http://127.0.0.1:8238/yunpan/top/article/list", host: "web.hjdang.com", referrer: "https://www.hjdang.com/category/%E5%AD%A6%E4%B9%A0?q="
2024/05/29 03:50:01 [error] 2732888#2732888: *24973775 testing "/etc/nginx/html" existence failed (2: No such file or directory) while logging request, client: 122.96.46.53, server: web.hjdang.com, request: "POST /yunpan/resource/list HTTP/1.1", upstream: "http://127.0.0.1:8238/yunpan/resource/list", host: "web.hjdang.com", referrer: "https://www.hjdang.com/category/%E5%AD%A6%E4%B9%A0?q="
2024/05/29 03:50:01 [error] 2732888#2732888: *24973779 testing "/etc/nginx/html" existence failed (2: No such file or directory) while logging request, client: 122.96.46.53, server: web.hjdang.com, request: "POST /yunpan/resource/list HTTP/1.1", upstream: "http://127.0.0.1:8238/yunpan/resource/list", host: "web.hjdang.com", referrer: "https://www.hjdang.com/category/%E5%AD%A6%E4%B9%A0?q="
2024/05/29 03:50:18 [error] 2732888#2732888: *24973790 testing "/etc/nginx/html" existence failed (2: No such file or directory) while logging request, client: 122.96.46.53, server: web.hjdang.com, request: "POST /yunpan/resource/list HTTP/1.1", upstream: "http://127.0.0.1:8238/yunpan/resource/list", host: "web.hjdang.com", referrer: "https://www.hjdang.com/category/%E5%AD%A6%E4%B9%A0?q="
2024/05/29 03:50:18 [error] 2732888#2732888: *24973789 testing "/etc/nginx/html" existence failed (2: No such file or directory) while logging request, client: 122.96.46.53, server: web.hjdang.com, request: "POST /yunpan/resource/list HTTP/1.1", upstream: "http://127.0.0.1:8238/yunpan/resource/list", host: "web.hjdang.com", referrer: "https://www.hjdang.com/category/%E5%AD%A6%E4%B9%A0?q="
2024/05/29 03:50:21 [error] 2732891#2732891: *24973810 testing "/etc/nginx/html" existence failed (2: No such file or directory) while logging request, client: 180.233.86.12, server: web.hjdang.com, request: "POST /user/islogin HTTP/1.1", upstream: "http://127.0.0.1:8238/user/islogin", host: "web.hjdang.com", referrer: "https://www.hjdang.com/d/1638545005108868059"
2024/05/29 03:50:25 [error] 2732890#2732890: *24973818 testing "/etc/nginx/html" existence failed (2: No such file or directory) while logging request, client: 218.109.200.144, server: web.hjdang.com, request: "POST /user/islogin HTTP/1.1", upstream: "http://127.0.0.1:8238/user/islogin", host: "web.hjdang.com", referrer: "https://www.hjdang.com/d/1780795623239385089"
2024/05/29 03:50:27 [error] 2732888#2732888: *24973790 testing "/etc/nginx/html" existence failed (2: No such file or directory) while logging request, client: 122.96.46.53, server: web.hjdang.com, request: "POST /yunpan/resource/list HTTP/1.1", upstream: "http://127.0.0.1:8238/yunpan/resource/list", host: "web.hjdang.com", referrer: "https://www.hjdang.com/category/%E5%AD%A6%E4%B9%A0?q="
2024/05/29 03:50:29 [error] 2732890#2732890: *24973818 testing "/etc/nginx/html" existence failed (2: No such file or directory) while logging request, client: 218.109.200.144, server: web.hjdang.com, request: "POST /user/islogin HTTP/1.1", upstream: "http://127.0.0.1:8238/user/islogin", host: "web.hjdang.com", referrer: "https://www.hjdang.com/d/1780795623239385089"
2024/05/29 03:50:32 [error] 2732890#2732890: *24973834 testing "/etc/nginx/html" existence failed (2: No such file or directory) while logging request, client: 116.179.37.84, server: web.hjdang.com, request: "OPTIONS /user/islogin HTTP/1.1", upstream: "http://127.0.0.1:8238/user/islogin", host: "web.hjdang.com", referrer: "https://www.hjdang.com/d/1749969100689408029"
2024/05/29 03:50:32 [error] 2732890#2732890: *24973836 testing "/etc/nginx/html" existence failed (2: No such file or directory) while logging request, client: 116.179.37.196, server: web.hjdang.com, request: "POST /user/islogin HTTP/1.1", upstream: "http://127.0.0.1:8238/user/islogin", host: "web.hjdang.com", referrer: "https://www.hjdang.com/d/1749969100689408029"
2024/05/29 03:50:36 [error] 2732888#2732888: *24973790 testing "/etc/nginx/html" existence failed (2: No such file or directory) while logging request, client: 122.96.46.53, server: web.hjdang.com, request: "POST /yunpan/resource/list HTTP/1.1", upstream: "http://127.0.0.1:8238/yunpan/resource/list", host: "web.hjdang.com", referrer: "https://www.hjdang.com/category/%E4%B9%A6%E7%B1%8D?q="
2024/05/29 03:50:36 [error] 2732888#2732888: *24973790 testing "/etc/nginx/html" existence failed (2: No such file or directory) while logging request, client: 122.96.46.53, server: web.hjdang.com, request: "POST /yunpan/top/article/list HTTP/1.1", upstream: "http://127.0.0.1:8238/yunpan/top/article/list", host: "web.hjdang.com", referrer: "https://www.hjdang.com/category/%E4%B9%A6%E7%B1%8D?q="
2024/05/29 03:50:48 [error] 2732891#2732891: *24973810 testing "/etc/nginx/html" existence failed (2: No such file or directory) while logging request, client: 180.233.86.12, server: web.hjdang.com, request: "POST /yunpan/resource/list HTTP/1.1", upstream: "http://127.0.0.1:8238/yunpan/resource/list", host: "web.hjdang.com", referrer: "https://www.hjdang.com/list?q=%E5%9B%BD%E8%AF%AD%E9%9F%A9%E5%89%A7%EF%BC%8C%E5%8F%B0%E9%85%8D%E9%9F%A9%E5%89%A7%E5%90%88%E9%9B%86%EF%BC%9E500%E9%83%A8&x=0.44488566441600863"

原因是配置带变量的access_log必须指定root,加上   root   /var/www 就可以了; #一定要加root,否则access_og不能加变量

That's a common pitfall. Doc on access_log says:

The file path can contain variables (0.7.6+), but such logs have some constraints:

...
during each log write the existence of the request’s root directory is checked, and if it does not exist the log is not created.
You didn't set root directory so nginx falls back to default one that is /etc/nginx/html and it doesn't exists.

Just add root /var/www; (or some other existing path) or, as zezollo suggested, create /etc/nginx/html.

  

 

标签:hjdang,web,05,request,按天,nginx,日志,com
From: https://www.cnblogs.com/zjhgx/p/18219906

相关文章

  • 【Nginx】Referer配置
    server配置server{listen80;server_namelocalhost;add_headerX-Frame-OptionsSAMEORIGIN;add_headerReferer-Policyorigin;add_headerContent-Security-Policy"frame-ancestors'self'";add_headerX-Permi......
  • SpringBoot如何使用日志Logback,及日志等级详解
    SpringBoot默认已经集成了SLF4J(SimpleLoggingFacadeforJava)作为日志的接口,以及Logback作为日志的实现。这意味着在大多数情况下,你无需做额外的配置即可开始记录日志。下面是一个简要的指南,包括如何在SpringBoot应用中使用SLF4J和Logback,以及一些实际的代码示例。默......
  • nginx代理jenkins无法跳转问题
    我的jenkinsdockerrun-d--namejenkins-p7005:8080-eJENKINS_OPTS="--prefix=/jkns"-v/data/jenkins:/var/jenkins_home-v/data/app:/appjenkins/jenkins主要是需要加--prefix=/jknsnginx配置:location/jkns{proxy_set_headerReferer$http_refer......
  • Nginx R31 doc-15-Live Activity Monitoring 实时活动监控
    前言大家好,我是老马。很高兴遇到你。我们为java开发者实现了java版本的nginxhttps://github.com/houbb/nginx4j如果你想知道servlet如何处理的,可以参考我的另一个项目:手写从零实现简易版tomcatminicat手写nginx系列如果你对nginx原理感兴趣,可以读一下从......
  • WindowsCA证书服务(五)为nginx申请x509证书
    简介我们不止windows的IIS呀,还有nginx,apache等等服务器,这些服务器不止要证书,还要私钥呢。还有一些东东要p12格式的证书,包含证书链和私钥的单文件证书。比如说深信服的防火墙。openssl经过一番搜索,这次真得脱离windows了。即使我使用第三方软件,让你在windows下生成了密钥对,还......
  • 调用mybatisPlus的 Iservice中的save方法,后台日志打印出来Insert语句,但是没插入到数据
    问题:调用mybatisPlus的Iservice中的save方法,后台日志打印出来Insert语句,但是没插入到数据库表中;一共向两个表中插入数据;weather,向天气表中插入数据成功,但是在异步任务中向三方同步日志表中 third_request_log是只打印出了成功Insert语句和参数,但是实际未插入成功:代码如下:......
  • ingress 日志切割
    背景ingress本质上也是个nginx,多个项目日志全部往access.log中输入就显得杂乱无章可读性不高,不利于日志阅读关键信息获取,排错就异常痛苦。so闲话少叙开干!1、部署ingress我这里ingress是用helm部署的[root@master~]#helmlistNAMENAMESPACEREVISION......
  • nginx中$host、$http_host、$proxy_host的区别
    nginx中$host、$http_host、$proxy_host的区别变量 是否显示端口 值$host 不显示端口 浏览器请求的ip,不显示端口$http_host 端口存在则显示 浏览器请求的ip和端口号$proxy_host 默认80端口不显示,其它显示 被代理服务的ip和端口号配置nginx代理服务后不设置proxy_set_head......
  • 二进制日志(bin log)
    binlog主要应用场景:一是用于数据恢复二是用于数据复制5.1查看默认情况查看记录二进制日志是否开启:在MySQL8中默认情况下,二进制文件是开启的。mysql>showvariableslike'%log_bin%';+---------------------------------+----------------------------------+|Variabl......
  • Nginx企业级负载均衡:技术详解系列(12)—— 深入解析root、alias及location
    你好,我是赵兴晨,97年文科程序员。在生产服务器的Nginx配置中,我们总会遇到形形色色的配置方案。你是否曾注意到root和alias指令的巧妙应用?是否对那些五花八门的location匹配规则感到好奇?今天,咱们来聊聊Nginx配置中root和alias以及location的详细使用。root与aliasroot:指......