首页 > 其他分享 >制作一个谷歌浏览器插件,实现网页数据爬虫

制作一个谷歌浏览器插件,实现网页数据爬虫

时间:2022-12-08 13:56:40浏览次数:76  
标签:插件 浏览器 runner image 爬虫 access entrypoint container Docker

Override the entrypoint of an image

Introduced in GitLab and GitLab Runner 9.4. Read more about the extended configuration options.

Before explaining the available entrypoint override methods, let’s describe how the runner starts. It uses a Docker image for the containers used in the CI/CD jobs:

  1. The runner starts a Docker container using the defined entrypoint. The default from Dockerfile that may be overridden in the .gitlab-ci.yml file.
  2. The runner attaches itself to a running container.
  3. The runner prepares a script (the combination of before_scriptscript, and after_script).
  4. The runner sends the script to the container’s shell stdin and receives the output.

To override the entrypoint of a Docker image, define an empty entrypoint in the .gitlab-ci.yml file, so the runner does not start a useless shell layer. However, that does not work for all Docker versions.

  • For Docker 17.06 and later, the entrypoint can be set to an empty value.
  • For Docker 17.03 and earlier, the entrypoint can be set to /bin/sh -c/bin/bash -c, or an equivalent shell available in the image.

The syntax of image:entrypoint is similar to Dockerfile’s ENTRYPOINT.

Let’s assume you have a super/sql:experimental image with a SQL database in it. You want to use it as a base image for your job because you want to execute some tests with this database binary. Let’s also assume that this image is configured with /usr/bin/super-sql run as an entrypoint. When the container starts without additional options, it runs the database’s process. The runner expects that the image has no entrypoint or that the entrypoint is prepared to start a shell command.

With the extended Docker configuration options, instead of:

  • Creating your own image based on super/sql:experimental.
  • Setting the ENTRYPOINT to a shell.
  • Using the new image in your CI job.

You can now define an entrypoint in the .gitlab-ci.yml file.

For Docker 17.06 and later:

image:
  name: super/sql:experimental
  entrypoint: [""]

For Docker 17.03 and earlier:

image:
  name: super/sql:experimental
  entrypoint: ["/bin/sh", "-c"]

Define image and services in config.toml

Look for the [runners.docker] section:

[runners.docker]
  image = "ruby:latest"
  services = ["mysql:latest", "postgres:latest"]

The image and services defined this way are added to all jobs run by that runner.

Access an image from a private Container Registry

To access private container registries, the GitLab Runner process can use:

To define which option should be used, the runner process reads the configuration in this order:

  • DOCKER_AUTH_CONFIG CI/CD variable.
  • DOCKER_AUTH_CONFIG environment variable set in the runner’s config.toml file.
  • config.json file in $HOME/.docker directory of the user running the process. If the --user flag is provided to run the child processes as unprivileged user, the home directory of the main runner process user is used.

Requirements and limitations

  • Available for Kubernetes executor in GitLab Runner 13.1 and later.
  • Credentials Store and Credential Helpers require binaries to be added to the GitLab Runner $PATH, and require access to do so. Therefore, these features are not available on shared runners, or any other runner where the user does not have access to the environment where the runner is installed.

Use statically-defined credentials

There are two approaches that you can take to access a private registry. Both require setting the CI/CD variable DOCKER_AUTH_CONFIG with appropriate authentication information.

  1. Per-job: To configure one job to access a private registry, add DOCKER_AUTH_CONFIG as a CI/CD variable.
  2. Per-runner: To configure a runner so all its jobs can access a private registry, add DOCKER_AUTH_CONFIG as an environment variable in the runner’s configuration.

标签:插件,浏览器,runner,image,爬虫,access,entrypoint,container,Docker
From: https://www.cnblogs.com/sdfasdf/p/16965872.html

相关文章

  • 浏览器桌面通知Notification实践
    一言不合就上图: 最近常常在浏览器看到这样的消息推送,还有QQ.com的推送,现在我对这个不了解,不知道叫消息自动推送对不对,这个时chrome浏览器的截图,出现在右下角,其他浏览器的样......
  • laravel9-原生注解路由插件-laravel-route-notes
    laravel-route-noteslaravel框架扩展,原生注解生成路由优点是直接生成路由文件,不在运行中解析路由,提升效率使用环境[PHP]>=8.0[Laravel]>=9.0如何安装直接使用co......
  • layui多选下拉框插件地址
    地址:https://maplemei.gitee.io/xm-select/#/component/install示例:vardemo1=xmSelect.render({el:"#select_user_box",l......
  • java爬虫笔记:使用WebCollector增量采集www.baiduyunsousou.com
    WebCollector可以配置短点爬取,历史数据根据Key去重,也就是url 最近在采集百度云网盘,记录一下 /***@authorLiu*@create2022-08-0211:48*/@Component@Slf......
  • 企业如何用爬虫做舆情监控
    数据采集通俗点来说就是通过爬虫代码访问目标网站的API链接获取有用的信息。爬虫程序就是模拟人工从网页中获取需要的信息,并自动保存在文档里面,应用十分广泛。例如图片、视......
  • vue3 Cropperimage插件写入默认网络图片跨域解决办法----- 图片转换成base64
    最近项目中有一个裁切图片的需求,百度了一番最后选用cropperImage插件。由于项目中图片是存放在阿里云上,cropperImage插件在初始化默认图的时候会存在跨域问题,百度经验我选......
  • ArcGIS插件-太乐地图
    在很多行业,打败你的往往不是竞争对手,而是你意想不到的其他因素。回想十年前,如果能预料到现在的处境,也许当时的心态和做法会有所不同。在多年前,市场上出现包括水经注、太乐......
  • 【2022-12-07】爬虫从入门到入狱(五)
    scrapy架构介绍#引擎(EGINE)引擎负责控制系统所有组件之间的数据流,并在某些动作发生时触发事件。有关详细信息,请参见上面的数据流部分。#调度器(SCHEDULER)用来接......
  • 关于JQuery validate表单校验插件对级联下拉框的校验问题
         今天在使用JQueryvalidate表单校验插件时,遇见了一个非常诡异的问题了。在此记录一下:     问题描述:               ......
  • 【提前爆料】FL Studio 21新版将引入4款全新插件!创作力UP!
    FLStudio21全称FruityLoopsStudio,就是大家熟悉的水果编曲软件,一个全能的音乐制作软件,包括编曲、录音、剪辑和混音等诸多功能,让你的电脑编程一个全能的录音室。FLStudio......