Python - Scrapy

1.安装第三方包

pip install Scrapy

2.创建项目

# 生成文件夹 scrapydemo1
PS E:\PyProject> scrapy startproject scrapydemo1
New Scrapy project 'scrapydemo1', using template directory 'E:\code_tool\python\lib\site-packages\scrapy\templates\project', created in:
    E:\PyProject\scrapydemo1

You can start your first spider with:
    cd scrapydemo1
    scrapy genspider example example.com


PS E:\PyProject> cd scrapydemo1
PS E:\PyProject\scrapydemo1> ls


# 文件目录结果如下：
    目录: E:\PyProject\scrapydemo1


Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d-----         2022/12/4     10:43                scrapydemo1   # 项目的模块，引入的时候需要从这里引入
-a----         2022/12/4     10:43            265 scrapy.cfg # Scrapy部署时的文件配置


PS E:\PyProject\scrapydemo1> cd .\scrapydemo1\
PS E:\PyProject\scrapydemo1\scrapydemo1> ls


    目录: E:\PyProject\scrapydemo1\scrapydemo1


Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d-----         2022/12/4     10:42                spiders
-a----         2022/12/4     10:43            267 items.py
-a----         2022/12/4     10:43           3658 middlewares.py
-a----         2022/12/4     10:43            365 pipelines.py
-a----         2022/12/4     10:43           3306 settings.py
-a----         2022/12/4     10:42              0 __init__.py

3.创建Spider

进入刚创建的scrapydemo1文件夹，使用genspider命令，第一个参数是Spider名称
第二个是网站名称

PS E:\PyProject\scrapydemo1\scrapydemo1> scrapy genspider quotes quotes.toscrape.com
Created spider 'quotes' using template 'basic' in module:
  scrapydemo1.spiders.quotes

https://cuiqingcai.com/31047.html