首页 > 其他分享 >将错就错:借助 YARP 转发 DNS 错乱解析造成的错误请求

将错就错:借助 YARP 转发 DNS 错乱解析造成的错误请求

时间:2023-01-19 19:44:58浏览次数:66  
标签:将错就错 private YARP readonly DNS host new httpContext public

最近园子在部署 IPv6 时遇到了一个非常奇怪的 dns 解析问题,当给非 www 二级域名(比如 q.cnblogs.com)添加 AAAA(IPv6) 记录后,部分用户访问 q.cnblogs.com 时会被错误地解析为 www.cnblogs.com 对应的 IPv4 地址,去掉 AAAA 解析就恢复正常。

为了对付这个不可控的奇怪问题(换了2家知名 dns 解析服务商都有同样的问题),我们采用了一个将错就错的变通方法,在 www 应用中借助反向代理将错误解析的请求转发给正确的应用处理,反向代理库用的是 YARP

由于我们只转发请求,所以只需使用 YARP 的 Direct Forwarding 功能。

安装 nuget 包 Yarp.ReverseProxy,然后在 Startup 中注册 HttpForwarder

services.AddHttpForwarder();

专门实现一个中间件 ForwardRequestsMiddleware 用于转发请求:

查看代码 ForwardRequestsMiddleware
public class ForwardRequestsMiddleware : IDisposable
{
    private static readonly string[] _forwardedSubdomains = new[] { "q", "news" };

    private readonly RequestDelegate _next;
    private readonly IHttpForwarder _forwarder;
    private readonly HttpMessageInvoker _httpClient;
    private readonly ForwarderRequestConfig _requestOptions;
    private bool _disposed;

    public static string[] Subdomains => _forwardedSubdomains;

    public ForwardRequestsMiddleware(
        RequestDelegate next,
        IHttpForwarder forwarder)
    {
        _next = next;
        _forwarder = forwarder;
        _httpClient = new HttpMessageInvoker(new SocketsHttpHandler()
        {
            UseProxy = false,
            AllowAutoRedirect = false,
            AutomaticDecompression = DecompressionMethods.None,
            UseCookies = false,
            ActivityHeadersPropagator = new ReverseProxyPropagator(DistributedContextPropagator.Current),
            ConnectTimeout = TimeSpan.FromSeconds(15),
        });
        _requestOptions = new ForwarderRequestConfig { ActivityTimeout = TimeSpan.FromSeconds(100) };
    }

    public async Task InvokeAsync(HttpContext context)
    {
        var host = context.Request.Host.Host;

        if (host.Equals(ConstStrings.DefaultHost, StringComparison.OrdinalIgnoreCase) ||
            !host.Contains(".") ||
            !_forwardedSubdomains.Any(d => host.Equals($"{d}.{ConstStrings.RootDomain}", StringComparison.OrdinalIgnoreCase)))
        {
            await _next(context);
            return;
        }

        var serviceName = host.Substring(0, host.IndexOf(".")) + "_web";
        await _forwarder.SendAsync(context, $"http://{serviceName}", _httpClient, _requestOptions);
    }

    public void Dispose()
    {
        Dispose(true);
        GC.SuppressFinalize(this);
    }

    protected virtual void Dispose(bool disposing)
    {
        if (_disposed) return;

        if (disposing) _httpClient?.Dispose();

        _disposed = true;
    }
}

实现后测试时发现转发时少了原始请求的 Host 请求头,而我们的应用需要这个请求头,进一步了解后得知默认使用的 StructuredTransformer 会自动去除原始 Host 请求头。

于是自己实现一个 CustomTransformer:

private class CustomTransformer : HttpTransformer
{
    public override ValueTask TransformRequestAsync(HttpContext httpContext, HttpRequestMessage proxyRequest, string destinationPrefix)
    {
        return base.TransformRequestAsync(httpContext, proxyRequest, destinationPrefix);
    }
}

通过 IHttpForwarder.SendAsync 方法第5个参数传入

await _forwarder.SendAsync(context, $"http://{serviceName}", _httpClient, _requestOptions, _transformer);

继续测试时发现转发时少了 X-Forwarded-ForX-Forwarded-Proto 请求头,StructuredTransformer 转发时会加上这些请求头,但 StructuredTransformer 被标记为 internal,无法重用也无法定制,只能自己想方法在 CustomTransformer 中实现。

参考 YARP 的源码经过一番折腾,终于实现了!

查看代码 CustomTransformer
private class CustomTransformer : HttpTransformer
{
    private readonly RequestTransform[] _requestTransforms;

    public CustomTransformer()
    {
        _requestTransforms = new RequestTransform[]
        {
            new RequestHeaderXForwardedForTransform("X-Forwarded-For", ForwardedTransformActions.Set),
            new RequestHeaderXForwardedProtoTransform("X-Forwarded-Proto", ForwardedTransformActions.Set)
        };
    }

    public override async ValueTask TransformRequestAsync(HttpContext httpContext, HttpRequestMessage proxyRequest, string destinationPrefix)
    {
        await base.TransformRequestAsync(httpContext, proxyRequest, destinationPrefix);

        if (_requestTransforms.Length == 0)
        {
            return;
        }

        var transformContext = new RequestTransformContext()
        {
            DestinationPrefix = destinationPrefix,
            HttpContext = httpContext,
            ProxyRequest = proxyRequest,
            Path = httpContext.Request.Path,
            Query = new QueryTransformContext(httpContext.Request),
            HeadersCopied = true
        };

        foreach (var requestTransform in _requestTransforms)
        {
            await requestTransform.ApplyAsync(transformContext);
        }
    }
}

进一步改进中间件,改用 IForwarderHttpClientFactory 创建 HttpMessageInvoker,之前的实现是手动 new,因此需要实现 IDisposable 接口。

使用 IForwarderHttpClientFactory 需要在 Startup 中注册 ForwarderHttpClientFactory

services.TryAddSingleton<IForwarderHttpClientFactory, ForwarderHttpClientFactory>();

中间件的最终实现如下:

查看代码 ForwardRequestsMiddleware
public class ForwardRequestsMiddleware
{
    private static readonly string[] _forwardedSubdomains = new[] { "q", "news" };

    private readonly RequestDelegate _next;
    private readonly IHttpForwarder _forwarder;
    private readonly IForwarderHttpClientFactory _httpClientFactory;
    private readonly ForwarderRequestConfig _requestOptions;
    private readonly HttpTransformer _transformer;

    public ForwardRequestsMiddleware(
        RequestDelegate next,
        IHttpForwarder forwarder,
        IForwarderHttpClientFactory httpClientFactory)
    {
        _next = next;
        _forwarder = forwarder;
        _httpClientFactory = httpClientFactory;
        _requestOptions = new ForwarderRequestConfig { ActivityTimeout = TimeSpan.FromSeconds(300) };
        _transformer = new CustomTransformer();
    }

    public async Task InvokeAsync(HttpContext context)
    {
        var host = context.Request.Host.Host;

        if (host.Equals(ConstStrings.DefaultHost, StringComparison.OrdinalIgnoreCase) ||
            !host.Contains(".") ||
            !_forwardedSubdomains.Any(d => host.Equals($"{d}.{ConstStrings.RootDomain}", StringComparison.OrdinalIgnoreCase)))
        {
            await _next(context);
            return;
        }

        var serviceName = host.Substring(0, host.IndexOf(".")) + "_web";

        var httpClient = _httpClientFactory.CreateClient(new ForwarderHttpClientContext
        {
            NewConfig = HttpClientConfig.Empty
        });

        await _forwarder.SendAsync(context, $"http://{serviceName}", httpClient, _requestOptions, _transformer);
    }

    private class CustomTransformer : HttpTransformer
    {
        private readonly RequestTransform[] _requestTransforms;

        public CustomTransformer()
        {
            _requestTransforms = new RequestTransform[]
            {
                new RequestHeaderXForwardedForTransform("X-Forwarded-For", ForwardedTransformActions.Set),
                new RequestHeaderXForwardedProtoTransform("X-Forwarded-Proto", ForwardedTransformActions.Set)
            };
        }

        public override async ValueTask TransformRequestAsync(HttpContext httpContext, HttpRequestMessage proxyRequest, string destinationPrefix)
        {
            await base.TransformRequestAsync(httpContext, proxyRequest, destinationPrefix);

            if (_requestTransforms.Length == 0)
            {
                return;
            }

            var transformContext = new RequestTransformContext()
            {
                DestinationPrefix = destinationPrefix,
                HttpContext = httpContext,
                ProxyRequest = proxyRequest,
                Path = httpContext.Request.Path,
                Query = new QueryTransformContext(httpContext.Request),
                HeadersCopied = true
            };

            foreach (var requestTransform in _requestTransforms)
            {
                await requestTransform.ApplyAsync(transformContext);
            }
        }
    }
}

(搞定)

标签:将错就错,private,YARP,readonly,DNS,host,new,httpContext,public
From: https://www.cnblogs.com/dudu/p/17061088.html

相关文章

  • DNS欺骗:网站克隆实现网站钓鱼攻击
    1DNS1.1DNS是什么?域名系统(DomainNameSystem)是互联网使用的命名系统,用来将主机域名转换为ip地址,属于应用层协议,使用UDP传输。1.2为什么需要DNS?DNS协议提供域......
  • Linux下开启防火墙放行nfs,ssh,httpd,dns,chrony服务(iptables,firewalld),firewalld端口转发
     环境:CentOS7.9什么是防火墙防火墙:防火墙是位于内部网和外部网之间的屏障,它按照系统管理员预先定义好的规则来控制数据包的进出。防火墙又可以分为硬件防火墙与软件......
  • Java反序列化-URLDNS利用链分析
    前言URLDNS链是Java反序列化中比较简单的一个链子,由于URLDNS不依赖第三方包和不限制jdk版本,所以经常用于检测反序列化漏洞。URLDNS并不能执行命令,只能发送DNS请求。(应该......
  • linux添加dns配置
    named.conf是DNS服务器bind的配置文件resolv.conf是系统的DNS配置系统的DNS配置1、编辑DNS配置文件vim/etc/resolv.conf没有resolv.conf文件,touch创建一个2......
  • JVM DNS 缓存配置(转)
    原文:https://www.jianshu.com/p/048e8bd3ea46作者:EricAlpha域名解析并非一个简单的过程,其解析结果可能会被层层缓存,如浏览器DNS缓存、操作系统DNS缓存、ISP的DNS......
  • Linux DNS --- Bind多域名配置
    一、承上启下https://www.cnblogs.com/eagle6688/p/17026162.html上一篇我们配置了example.com的解析,本文我们尝试添加第二个域名test.com还有它的二级子域名mail.test.c......
  • JAVA中DNS缓存设置(转)
    原文:https://blog.csdn.net/guanfengliang1988/article/details/92813431作者:夜风_BLOG我们上网的原点就是打开浏览器,在上方地址栏输入网址的那一刻,这个回车按了之后,发生......
  • Centos 7 搭建DNS服务器
    1.DNS原理看这篇文章https://www.cnblogs.com/kubixuesheng/p/6260195.html2.服务器准备192.168.1.100 DNS服务器192.168.1.101 DNS客户端192.168.1.102 DNS客......
  • 什么是域名? 什么是DNS?
    域名关于域名,百度百科是这样介绍的:百度百科(https://baike.baidu.com/item/域名/86062):域名(英语:DomainName),又称网域,是由一串用点分隔的名字组成的Internet上某一台计算......
  • 超详细 DNS 协议解析(转)
    原文:https://xie.infoq.cn/article/9584c7089e5f966a90c1c5d9a?source=app_share作者:飞天小牛肉 addbyzhj: 个人不认同作者对“权威DNS服务器”的解释,它并不是电脑......