使用 ASP.NET Core HttpLoggingMiddleware 记录 http 请求/响应

        我们发布了一个应用程序，该应用程序运行在一个相当隐蔽的 WAF 后面。他们向我们保证，他们的产品不会以任何方式干扰我们的应用程序。这是错误的。他们删除了我们几乎所有的“自定义”标头。为了“证明”这一点，我构建了一个中间件，用于存储我们收到和发送的所有唯一标头。然后我创建了一个转储所有标头的端点。

我创建了一个自定义中间件，LogHeadersMiddleware。

public class LogHeadersMiddleware 
{
    private readonly RequestDelegate _next;
    private readonly ILogger<LogHeadersMiddleware> _logger;
    public static readonly List<string> RequestHeaders = new List<string>();
    public static readonly List<string> ResponseHeaders = new List<string>();

    public LogHeadersMiddleware(RequestDelegate next, ILogger<LogHeadersMiddleware> logger)
    {
        _next = next;
        _logger = logger;
    }

    public async Task Invoke(HttpContext context)
    {
        var uniqueRequestHeaders = context.Request.Headers
            .Where(x => RequestHeaders.All(r => r != x.Key))
            .Select(x => x.Key);
        RequestHeaders.AddRange(uniqueRequestHeaders);

        await _next.Invoke(context);
        var uniqueResponseHeaders = context.Response.Headers
            .Where(x => ResponseHeaders.All(r => r != x.Key))
            .Select(x => x.Key);
        ResponseHeaders.AddRange(uniqueResponseHeaders);
        
        // Log unique headers with ILogger
        ...
    }
}

然后我在中间件管道的早期就这样注册了它 

app.UseLogHeadersMiddleware();
app.Map("/show-headers", ShowHeaders);

因此，通过转到 /show-headers，您将看到我们的应用程序接收和发送的所有（唯一）标头的转储。 

新方法

微软现在已经在 ASP.NET Core 中添加了对 HTTP 日志记录的支持。
他们创建了自己的中间件，您可以像这样启用它：

public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
    // Important to have it as early as possible
    app.UseHttpLogging(); 
    ...
    ...
    ...
}

如果您不进行任何自定义，则默认记录以下内容（请求和响应）：

Request path
Status code
Querystring
Headers 

示例 - 默认实现

向 /user/login
主体发出 POST 请求

{
   "username": "josef",
   "password": "MyPassword"
}

Request 

info: Microsoft.AspNetCore.HttpLogging.HttpLoggingMiddleware[1]
      Request:
      Protocol: HTTP/1.1
      Method: POST
      Scheme: http
      PathBase:
      Path: /user/login
      QueryString:
      Accept: */*
      Connection: keep-alive
      Host: localhost:5000
      User-Agent: PostmanRuntime/7.28.1
      Accept-Encoding: gzip, deflate, br
      Content-Type: application/json
      Content-Length: 60 

Response

info: Microsoft.AspNetCore.HttpLogging.HttpLoggingMiddleware[2]
      Response:
      StatusCode: 200
      Content-Type: application/json; charset=utf-8
      Date: [Redacted]
      Server: [Redacted]
      Transfer-Encoding: chunked 

如您所见，使用默认配置，不会记录任何敏感信息。
让我们看看如何配置中间件来记录一些敏感数据。

示例 - 记录请求/响应主体

请求正文（危险）

这将记录请求标头和正文

services.AddHttpLogging(options =>
{
    options.LoggingFields = HttpLoggingFields.RequestPropertiesAndHeaders |
                            HttpLoggingFields.RequestBody;
});

Body体

{
    "username": "josef",
    "password": "MyPassword"
}

Request 

info: Microsoft.AspNetCore.HttpLogging.HttpLoggingMiddleware[1]
      Request:
      Protocol: HTTP/1.1
      Method: POST
      Scheme: http
      PathBase:
      Path: /user/login
      QueryString:
      Accept: */*
      Connection: keep-alive
      Host: localhost:5000
      User-Agent: PostmanRuntime/7.28.1
      Accept-Encoding: gzip, deflate, br
      Content-Type: application/json
      Content-Length: 60

info: Microsoft.AspNetCore.HttpLogging.HttpLoggingMiddleware[2]
      RequestBody: {
          "username": "josef",
          "password": "MyPassword"
      }

响应主体（危险）

这将记录响应标头和正文

services.AddHttpLogging(options =>
{
    options.LoggingFields = HttpLoggingFields.ResponsePropertiesAndHeaders |
                            HttpLoggingFields.ResponseBody;
});

Body

{
    "username": "josef",
    "password": "MyPassword"
}

Response

info: Microsoft.AspNetCore.HttpLogging.HttpLoggingMiddleware[1]
      Response:
      StatusCode: 200
      Content-Type: application/json; charset=utf-8

info: Microsoft.AspNetCore.HttpLogging.HttpLoggingMiddleware[2]
      ResponseBody: {"token":"very secret token"}

        如您所见，我们现在正在记录请求和响应主体。这在开发/调试期间可能非常有用，但您很可能永远不想在生产中这样做。通过盲目地记录请求/响应标头/主体，您可以（很可能）记录敏感数据。如果您觉得有必要记录标头和主体，请确保使用选择加入策略，如下所示：

services.AddHttpLogging(options =>
{
    options.LoggingFields = HttpLoggingFields.RequestPropertiesAndHeaders |
                            HttpLoggingFields.ResponsePropertiesAndHeaders;
    options.ResponseHeaders.Add("Non-Sensitive");
});

调用以下操作时...

[HttpGet("api/weather")]
public IActionResult Index()
{
    Response.Headers.Add("Sensitive", "034D4CD7-2FEB-4B19-86A3-CFCD5DB291AA");
    Response.Headers.Add("Non-Sensitive", "Hello");
    return new OkObjectResult(new
    {
        data = "Hello World"
    });
} 

...仅会记录非敏感标头值 

Request

info: Microsoft.AspNetCore.HttpLogging.HttpLoggingMiddleware[2]
Response:
StatusCode: 200
Content-Type: application/json; charset=utf-8
Date: [Redacted]
Server: [Redacted]
Transfer-Encoding: chunked
Sensitive: [Redacted]
Non-Sensitive: Hello

表现

考虑记录请求/响应主体时应用程序的性能会受到怎样的影响也很重要
。GET

没有 HttpLogging

❯ .\bombardier.exe -c 125 -n 100000 http://localhost:5000/api/weather
Bombarding http://localhost:5000/api/weather with 100000 request(s) using 125 connection(s)
 100000 / 100000 [==========================================] 100.00% 2287/s 43s
Done!
Statistics        Avg      Stdev        Max
  Reqs/sec      2292.70     326.38    7095.61
  Latency       54.52ms     2.29ms   109.00ms
  HTTP codes:
    1xx - 0, 2xx - 100000, 3xx - 0, 4xx - 0, 5xx - 0
    others - 0
  Throughput:   727.39KB/s

HttpLogging 的默认设置

❯ .\bombardier.exe -c 125 -n 100000 http://localhost:5000/api/weather
Bombarding http://localhost:5000/api/weather with 100000 request(s) using 125 connection(s)
 100000 / 100000 [==========================================] 100.00% 1492/s 1m6s
Done!
Statistics        Avg      Stdev        Max
  Reqs/sec      1496.73     287.61    4500.00
  Latency       83.52ms    15.73ms      0.86s
  HTTP codes:
    1xx - 0, 2xx - 100000, 3xx - 0, 4xx - 0, 5xx - 0
    others - 0
  Throughput:   474.86KB/s

已启用请求/响应主体日志记录

❯ .\bombardier.exe -c 125 -n 100000 http://localhost:5000/api/weather
Bombarding http://localhost:5000/api/weather with 100000 request(s) using 125 connection(s)
 100000 / 100000 [==========================================] 100.00% 1466/s 1m8s
Done!
Statistics        Avg      Stdev        Max
  Reqs/sec      1471.50     285.97    5090.86
  Latency       84.96ms    17.64ms      0.87s
  HTTP codes:
    1xx - 0, 2xx - 100000, 3xx - 0, 4xx - 0, 5xx - 0
    others - 0
  Throughput:   466.79KB/s

当打开HttpLogging时，吞吐量从 ~2300 req/s 变为 ~1500 req/
s.POST

没有 HttpLogging

❯ .\bombardier.exe -m POST http://localhost:5000/user/login -c 125 -n 100000 --duration 1s -l -H "Content-Type: application/json" -f "login.json"
Bombarding http://localhost:5000/user/login with 100000 request(s) using 125 connection(s)
 100000 / 100000 [==========================================] 100.00% 2112/s 47s
Done!
Statistics        Avg      Stdev        Max
  Reqs/sec      2118.76     444.32    6498.38
  Latency       59.03ms    14.41ms   576.00ms
  Latency Distribution
     50%    57.00ms
     75%    59.96ms
     90%    70.00ms
     95%    75.00ms
     99%    90.81ms
  HTTP codes:
    1xx - 0, 2xx - 100000, 3xx - 0, 4xx - 0, 5xx - 0
    others - 0
  Throughput:   771.02KB/s

HttpLogging 的默认设置

❯ .\bombardier.exe -m POST http://localhost:5000/user/login -c 125 -n 100000 --duration 1s -l -H "Content-Type: application/json" -f "login.json"
Bombarding http://localhost:5000/user/login with 100000 request(s) using 125 connection(s)
 100000 / 100000 [==========================================] 100.00% 1388/s 1m12s
Done!
Statistics        Avg      Stdev        Max
  Reqs/sec      1392.11     328.62    8995.50
  Latency       89.92ms    19.01ms      1.09s
  Latency Distribution
     50%    88.00ms
     75%    91.00ms
     90%    97.00ms
     95%   111.00ms
     99%   125.00ms
  HTTP codes:
    1xx - 0, 2xx - 100000, 3xx - 0, 4xx - 0, 5xx - 0
    others - 0
  Throughput:   506.18KB/s

已启用请求/响应主体日志记录

❯ .\bombardier.exe -m POST http://localhost:5000/user/login -c 125 -n 100000 --duration 1s -l -H "Content-Type: application/json" -f "login.json"
Bombarding http://localhost:5000/user/login with 100000 request(s) using 125 connection(s)
 100000 / 100000 [==========================================] 100.00% 1281/s 1m18s
Done!
Statistics        Avg      Stdev        Max
  Reqs/sec      1284.89     311.66    4550.25
  Latency       97.31ms    21.58ms      1.22s
  Latency Distribution
     50%    95.00ms
     75%    99.00ms
     90%   108.00ms
     95%   119.00ms
     99%   150.00ms
  HTTP codes:
    1xx - 0, 2xx - 100000, 3xx - 0, 4xx - 0, 5xx - 0
    others - 0
  Throughput:   467.73KB/s

我们可以看到，记录更多内容会对性能造成影响。记录请求/响应主体时，我们每秒会额外损失约 100 个请求。

如果您喜欢此文章，请收藏、点赞、评论，谢谢，祝您快乐每一天。