首页 > 编程语言 >python请求有关ja3指纹问题

python请求有关ja3指纹问题

时间:2023-06-21 14:24:07浏览次数:50  
标签:CIPHERS DH python self ciphers ECDH ja3 指纹 context

遇见一个网站采集,无论怎样都返回空数据(实际上是有数据的),但是抓包下来又确实是那样的,请教了一些人推测是指纹验证,拜读了网上其他大佬的博客文章后实验了一下,发现确实是这个问题!
第一次知道tcp还有这个东西,让我大受震撼,值此搬运一下。

参考链接及来源:
Python 爬虫进阶必备 | JA3 指纹在爬虫中的应用与定向突破
python爬虫 requests、httpx、aiohttp、scrapy突破ja3指纹识别

实例:

requests

from requests.adapters import HTTPAdapter
from requests.packages.urllib3.util.ssl_ import create_urllib3_context
import requests
import random
ORIGIN_CIPHERS = ('ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+HIGH:'
                  'DH+HIGH:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+HIGH:RSA+3DES')


class DESAdapter(HTTPAdapter):
    def __init__(self, *args, **kwargs):
        """
        A TransportAdapter that re-enables 3DES support in Requests.
        """
        CIPHERS = ORIGIN_CIPHERS.split(':')
        random.shuffle(CIPHERS)
        CIPHERS = ':'.join(CIPHERS)
        self.CIPHERS = CIPHERS + ':!aNULL:!eNULL:!MD5'
        super().__init__(*args, **kwargs)

    def init_poolmanager(self, *args, **kwargs):
        context = create_urllib3_context(ciphers=self.CIPHERS)
        kwargs['ssl_context'] = context
        return super(DESAdapter, self).init_poolmanager(*args, **kwargs)

    def proxy_manager_for(self, *args, **kwargs):
        context = create_urllib3_context(ciphers=self.CIPHERS)
        kwargs['ssl_context'] = context
        return super(DESAdapter, self).proxy_manager_for(*args, **kwargs)

headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36 Edg/92.0.902.67'}
s = requests.Session()
s.headers.update(headers)
s.mount('https://ja3er.com', DESAdapter())
resp = s.get('https://ja3er.com/json').json()
print(resp)

aiohttp

import random
import ssl
import asyncio
import aiohttp

# ssl._create_default_https_context = ssl._create_unverified_context


ORIGIN_CIPHERS = ('ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+HIGH:'
                  'DH+HIGH:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+HIGH:RSA+3DES')


class SSLFactory:
    def __init__(self):
        self.ciphers = ORIGIN_CIPHERS.split(":")

    def __call__(self) -> ssl.SSLContext:
        random.shuffle(self.ciphers)
        ciphers = ":".join(self.ciphers)
        ciphers = ciphers + ":!aNULL:!eNULL:!MD5"
        context = ssl.create_default_context()
        context.set_ciphers(ciphers)
        return context


sslgen = SSLFactory()
async def main():
    async with aiohttp.ClientSession() as session:
         async with session.get("https://ja3er.com/json", headers={}, ssl=sslgen()) as resp:
                data = await resp.json()
                print(data)

asyncio.get_event_loop().run_until_complete(main())

httpx:

异步模式:
import httpx
import asyncio
import random
import ssl

ORIGIN_CIPHERS = ('ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+HIGH:'
                  'DH+HIGH:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+HIGH:RSA+3DES')


class SSLFactory:
    def __init__(self):
        self.ciphers = ORIGIN_CIPHERS.split(":")

    def __call__(self) -> ssl.SSLContext:
        random.shuffle(self.ciphers)
        ciphers = ":".join(self.ciphers)
        ciphers = ciphers + ":!aNULL:!eNULL:!MD5"
        context = ssl.create_default_context()
        context.set_ciphers(ciphers)
        return context


sslgen = SSLFactory()
async def main():
    async with httpx.AsyncClient(verify=sslgen()) as client:
        resp = await client.get('https://ja3er.com/json')
        result = resp.json()
        print(result)


asyncio.run(main())
同步模式:
import httpx
import asyncio
import random
import ssl

ORIGIN_CIPHERS = ('ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+HIGH:'
                  'DH+HIGH:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+HIGH:RSA+3DES')


class SSLFactory:
    def __init__(self):
        self.ciphers = ORIGIN_CIPHERS.split(":")

    def __call__(self) -> ssl.SSLContext:
        random.shuffle(self.ciphers)
        ciphers = ":".join(self.ciphers)
        ciphers = ciphers + ":!aNULL:!eNULL:!MD5"
        context = ssl.create_default_context()
        context.set_ciphers(ciphers)
        return context


sslgen = SSLFactory()
with httpx.Client(headers={}, http2=True, verify=sslgen()) as client:
    response = client.get('https://ja3er.com/json')
    print(response.text)

scrapy

class MyHTTPDownloadHandler(HTTPDownloadHandler):
    def shuffle_ciphers(self):
        self.ORIGIN_CIPHERS = ('ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+HIGH:'
                               'DH+HIGH:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+HIGH:RSA+3DES')
        CIPHERS = self.ORIGIN_CIPHERS.split(':')
        random.shuffle(CIPHERS)
        CIPHERS = ':'.join(CIPHERS) + ':!aNULL:!eNULL:!MD5'
        return CIPHERS

    def download_request(self, request, spider):
        tls_ciphers = self.shuffle_ciphers()
        self._contextFactory = ScrapyClientContextFactory(tls_ciphers=tls_ciphers)
        return super().download_request(request, spider)

爬虫配置文件

custom_settings = {
        "CONCURRENT_REQUESTS": 5,
        "DOWNLOAD_DELAY": 1,
        "DOWNLOAD_TIMEOUT": 10,
        "RETRY_TIMES": 3,
           "DOWNLOAD_HANDLERS": {
            'http': 'scrapy_project.middlewares.MyHTTPDownloadHandler',
            'https': 'scrapy_project.middlewares.MyHTTPDownloadHandler',
        }
    }

 



标签:CIPHERS,DH,python,self,ciphers,ECDH,ja3,指纹,context
From: https://www.cnblogs.com/tjp40922/p/17496102.html

相关文章

  • python.制图.饼状图
    1defsendMsgtemplate_card(fail,passnum,casename,casetime,path):importpandasaspdimportmatplotlib.pyplotasplt#将测试结果转换为数值类型data1={'errors':'0','failures':fail,'skipped':'1'......
  • Ubuntu安装管理不同版本的python
    ubuntu多个版本安装背景:本地环境Ubuntu22.0464-bit,默认安装python3.10.6,未安装pip,venv需求:安装python3.8,并安装两版本对应pip,venv1增加PPA软件源deadsnakessudoaptinstallsoftware-properties-commonsudoadd-apt-repositoryppa:deadsnakes/ppa2跟新系统的......
  • 通用密钥,无需密码,在无密码元年实现Passkeys通用密钥登录(基于Django4.2/Python3.10)
    毋庸讳言,密码是极其伟大的发明,但拜病毒和黑客所赐,一旦密码泄露,我们就得绞尽脑汁再想另外一个密码,但记忆力并不是一个靠谱的东西,一旦遗忘密码,也会造成严重的后果,2023年业界巨头Google已经率先支持了Passkeys登录方式,只须在设备上利用PIN码解锁、指纹或面部辨识等生物识别方式,即可验......
  • PostgreSQL 通过python 监控逻辑复制
    上期是讲逻辑复制,本期是通过PYTHON来对逻辑复制中的配置参数,publication定义,打印不适合进行逻辑复制的表,打印没有在使用的复制槽,另外包含当前发布端和接收端两边的LSN对比。以下是代码,对于逻辑复制中主要的监控点有1 是不是存在复制槽不使用的情况2 是不是存在主库和从库之......
  • 十五、python文件IO操作
    十五、python文件IO操作python文件操作的步骤python文件的操作就三个步骤:1.先open打开一个要操作的文件2.操作此文件(读,写,追加等)3.close关闭此文件python文件访问模式简单格式:file_object=open(file_path,mode="")mode:r只读模式,不能写(文件必须存在,不存在会......
  • Python 修改ha配置文件
    Python修改ha配置文件任务要求1、用户输入字符串{"backend":"test.oldboy.org","record":{"server":"100.1.7.9","weight":20,"maxconn":30}}2、在对应的backend下,添加一条新记录backend不存在时,创建3、删除一条记录ba......
  • Porting Code to Python 3 with 2to3
    参考https://www.cmi.ac.in/~madhavan/courses/prog2-2012/docs/diveintopython3/porting-code-to-python-3-with-2to3.html......
  • 根据ubuntu:20.04制作python环境docker镜像
    因为有个算法是python写的,要在服务器上调用,之前是直接根据jdk镜像制作的环境,现在要装python,jdk双环境,只能自己制作一个镜像出来了,命令如下FROMubuntu:20.04ENVTZ=Asia/ShanghaiENVLANGC.UTF-8RUNmv/etc/apt/sources.list/etc/apt/sources.list.bakCOPYsources.li......
  • 【python基础】类-类属性
    在初始类中,我们介绍了如何访问类属性,除了访问类属性外还有其他操作类属性的情况,我们将在这里做详细介绍:1.给类属性指定默认值类中的每个属性都必须有初始值,哪怕这个值是0或者空字符串。在有些情况下,如设置默认值时,在方法__init__方法内指定这种初始值是可行的,如果对某个属性这样......
  • python 生成小学计算练习 docx
    python3生成二年级下计算练习,有口算,有竖式。 importrandomimportosfromdocx.sharedimportPtfromdocximportDocumentdefcreate_page(document):#口算document.add_paragraph('一、口算')operators='+-×÷'columnsNumber=3rowsNumbe......