标签：Containerd MiB nerdctl nginx containerd docker 高阶 alpine

1.前言

对于用惯了docker cli的用户来说，containerd的命令行工具ctr使用起来不是很顺手，此时别慌，还有另外一个命令行工具项目nerdctl可供我们选择。 nerdctl是一个与docker cli风格兼容的containerd的cli工具。 nerdctl已经作为子项目加入了containerd项目，它的github地址是https://github.com/containerd/nerdctl，而且从最近的nerdctl 0.8开始，nerdctl直接兼容了docker compose的语法(不包含swarm)，这很大提高了直接将containerd作为本地开发、测试和单机容器部署使用的体验。本来k8s后续将不再支持dockershim，docker在k8s社区的地位急剧下降，现在单机直接使用containerd易用性也不断被完善，也许docker的辉煌已经远去了。

实际上nerdctl compose实现的是Compose Specification规范，这个规范是从自Docker Compose file version 3 specification规范发展而来的。

2.安装nerdctl

宿主机安装了 cri-containerd-cni-1.7.0-linux-amd64.tar.gz 下载链接：https://github.com/containerd/containerd/releases/download/v1.7.0/cri-containerd-cni-1.7.0-linux-amd64.tar.gz

Containerd 安装请参考前面的博文。

nerdctl 官方发布包含两个安装版本：

Minimal：仅包含 nerdctl 二进制文件及 rootless 模式下的辅助安装脚本；
Full：全量包，其中包含了 Containerd、CNI、runc、BuildKit 等完整组件。

建议采用 Full版本安装。

2.1 下载nerdctl

下载地址：https://github.com/containerd/nerdctl/releases

wget https://github.com/containerd/nerdctl/releases/download/v1.2.1/nerdctl-full-1.2.1-linux-amd64.tar.gz

2.2 解压安装

安装注意：containerd 和 nerdctl-full 中同时包含了 runc 和 ctr ，如果直接覆盖可能会造成版本不一致的问题。

本次版本如下：

软件包	版本号
cri-containerd-cni	1.7.0
nerdctl-full	1.2.1

错误示范：

安装 containerd
>tar xf cri-containerd-cni-1.7.0-linux-amd64.tar.gz -C /

安装runc
>cp -a runc.amd64 /usr/local/sbin/runc

安装 nerdctl
>tar xf nerdctl-full-1.2.1-linux-amd64.tar.gz -C /usr/local/


这样，nerdctl 的 runc 和 ctr 会覆盖 containerd

>whereis runc
runc: /usr/local/bin/runc /usr/local/sbin/runc
>whereis ctr
ctr: /usr/local/bin/ctr

通过 ctr 查看版本：

>ctr version
Client:
  Version:  v1.6.19
  Revision: 1e1ea6e986c6c86565bc33d52e34b81b3e2bc71f
  Go version: go1.20.1

Server:
  Version:  v1.7.0
  Revision: 1fbd70374134b891f97ce19c70b6e50c7b9f4e0d
  UUID: 7e1b4345-bfdb-47e2-852f-7f663d73d71e
WARNING: version mismatch
WARNING: revision mismatch

出现了告警信息，提示版本不符。

为了规避这个问题，在安装 nerdctl 时，可以先移除 runc 和 ctr 再次拷贝到对应的目录。

正确示范：

安装 containerd
>tar xf cri-containerd-cni-1.7.0-linux-amd64.tar.gz -C /
>systemctl enable containerd ; systemctl start containerd

安装runc
>cp -a runc.amd64 /usr/local/sbin/runc

创建nerdctl目录
>mkdir /usr/local/nerdctl
解压到该目录
>tar xf nerdctl-full-1.2.1-linux-amd64.tar.gz -C /usr/local/nerdctl/
移除两个重复的命令
>rm -rf  bin/{ctr,runc}
拷贝对应位置的文件
>cp -a * /usr/local/

然后查看 ctr 和 runc

>ctr version
Client:
  Version:  v1.7.0
  Revision: 1fbd70374134b891f97ce19c70b6e50c7b9f4e0d
  Go version: go1.20.2

Server:
  Version:  v1.7.0
  Revision: 1fbd70374134b891f97ce19c70b6e50c7b9f4e0d
  UUID: a2488cb6-5f27-41f7-89b9-477d762806f6

>runc -v
runc version 1.1.4
commit: v1.1.4-0-g5fd4c4d1
spec: 1.0.2-dev
go: go1.17.10
libseccomp: 2.5.4

>nerdctl version
Client:
 Version:       v1.2.1
 OS/Arch:       linux/amd64
 Git commit:    a0bbfd75ba92bcb11ac6059bf4f6f4e50c6da0b8
 buildctl:
  Version:      v0.11.3
  GitCommit:    4ddee42a32aac4cd33bf9c2be4c87c2ffd34747b

Server:
 containerd:
  Version:      v1.7.0
  GitCommit:    1fbd70374134b891f97ce19c70b6e50c7b9f4e0d
 runc:
  Version:      1.1.4
  GitCommit:    v1.1.4-0-g5fd4c4d1

3.nerdctl使用

nerdctl 是 containerd 的命令行界面的工具。nerdctl 兼容 docker ，如果会使用 docker-cli 就等于掌握了 nerdctl 80% 的使用方法。nerdctl 不但兼容docker-cli 甚至还兼容了 docker-compose的功能点。

3.1 更名docker

甚至可以直接将nerdctl更名为 docker

cat << 'EOF' > /usr/local/bin/docker
#!/bin/bash
/usr/local/bin/nerdctl $@
EOF

chmod +x /usr/local/bin/docker

3.2 nerdctl bash自动补全

>yum install bash-completion -y
>nerdctl completion bash > /etc/bash_completion.d/nerdctl
>source /etc/bash_completion.d/nerdctl

上面补全的是 nerdctl 的命令，而当 nerdctl 重命名 docker 后，没有 docker 的自动补全。

添加 docker 别名的自动补全：

生成自动补全文件
>nerdctl completion bash > /etc/bash_completion.d/nerdctl
>nerdctl completion bash > /etc/bash_completion.d/docker

生效
>source /etc/bash_completion.d/nerdctl
>source /etc/bash_completion.d/docker

测试：

输入 docker image + 两下 tab

>docker image
image   (Manage images)  images  (List images)

>nerdctl image
image   (Manage images)  images  (List images)

3.3 安装常用插件

安装docker常用扩展插件

docker run --rm -v /usr/local/bin:/sysdir registry.cn-beijing.aliyuncs.com/k7scn/tools tar zxf /pkg.tgz -C /sysdir

安装扩展命令如下：

>docker run -it registry.cn-beijing.aliyuncs.com/k7scn/tools bash
bash-5.2# mkdir /sysdir
bash-5.2# tar xf pkg.tgz -C /sysdir/
bash-5.2# cd /sysdir/
bash-5.2# ls
cclear          ctop            docker-compose  ergoget         iclear          kdtoken         upgrade-tools
crictl          din             dps             helminit        kbtoken         scope

执行完成后，就已经拷贝到 /usr/local/bin 目录下。

4 镜像管理

4.1 查看镜像

>nerdctl images
REPOSITORY                                      TAG       IMAGE ID        CREATED              PLATFORM       SIZE        BLOB SIZE
registry.cn-beijing.aliyuncs.com/k7scn/tools    latest    71442d19f1f3    About an hour ago    linux/amd64    54.5 MiB    45.7 MiB

或者

>nerdctl image ls
REPOSITORY                                      TAG       IMAGE ID        CREATED              PLATFORM       SIZE        BLOB SIZE
registry.cn-beijing.aliyuncs.com/k7scn/tools    latest    71442d19f1f3    About an hour ago    linux/amd64    54.5 MiB    45.7 MiB

4.2 下载镜像

>nerdctl pull nginx:alpine
>nerdctl images
REPOSITORY    TAG       IMAGE ID        CREATED           PLATFORM       SIZE        BLOB SIZE
nginx         alpine    c94a22b036af    19 seconds ago    linux/amd64    42.7 MiB    16.0 MiB

4.3 查看镜像详细信息

>nerdctl inspect nginx:alpine

4.4 构建镜像

>mkdir dockerfile
~/dockerfile>vim Dockerfile
FROM nginx:alpine
RUN echo "hello world." > /usr/share/nginx/html/index.html

~/dockerfile>nerdctl build -t mynginx:alpine ./
ERRO[0000] `buildctl` needs to be installed and `buildkitd` needs to be running, see https://github.com/moby/buildkit  error="2 errors occurred:\n\t* failed to ping to host unix:///run/buildkit-default/buildkitd.sock: exit status 1\n\t* failed to ping to host unix:///run/buildkit/buildkitd.sock: exit status 1\n\n"
FATA[0000] no buildkit host is available, tried 2 candidates: 2 errors occurred:
        * failed to ping to host unix:///run/buildkit-default/buildkitd.sock: exit status 1
        * failed to ping to host unix:///run/buildkit/buildkitd.sock: exit status 1

当构建镜像时，出现如上报错信息，是因为 buildkit.service 服务没有启动，启动服务：

>systemctl enable buildkit.service ; systemctl start buildkit.service

BuildKit 是由 docker 公司开发的下一代 docker build 工具，具有更高效、更安全、易于扩展等特点。BuildKit 是由 buildkitd 守护程序和 buildctl 客户端组成。

buildkitd 作为服务端，连接容器运行时，目前支持 runc 和 containerd 作为镜像构建环境，默认是 runc
buildctl 作为客户端，负责解析 Dockerfile 文件，并向 buildkitd 发出构建请求。由于命令复杂，使用 nerdctl 替代

再次进行构建

>nerdctl build -t myapp:v1 ./
[+] Building 3.5s (5/6)
[+] Building 3.5s (6/6) FINISHED
 => [internal] load .dockerignore                                                                                                                                                   0.0s
 => => transferring context: 2B                                                                                                                                                     0.0s
 => [internal] load build definition from Dockerfile                                                                                                                                0.0s
 => => transferring dockerfile: 114B                                                                                                                                                0.0s
 => [internal] load metadata for docker.io/library/nginx:alpine                                                                                                                     2.5s
 => [1/2] FROM docker.io/library/nginx:alpine@sha256:c94a22b036afa972426b82d5b0a49c959786005b4f6f81ac7467ca5538d0158f                                                               0.0s
 => => resolve docker.io/library/nginx:alpine@sha256:c94a22b036afa972426b82d5b0a49c959786005b4f6f81ac7467ca5538d0158f                                                               0.0s
 => CACHED [2/2] RUN echo "hello world." > /usr/share/nginx/html/index.html                                                                                                         0.0s
 => exporting to docker image format                                                                                                                                                0.9s
 => => exporting layers                                                                                                                                                             0.0s
 => => exporting manifest sha256:4c33b3b11666960d0ac4216730378f70a897623e9888b77066189a1dea54aeb4                                                                                   0.0s
 => => exporting config sha256:a2470d7093fd645ea5adf7055d38110b4d1e4f1221cb2d5feb48c58f47c94b07                                                                                     0.0s
 => => sending tarball                                                                                                                                                              0.9s
Loaded image: docker.io/library/myapp:v1

查看镜像

>nerdctl images
REPOSITORY    TAG       IMAGE ID        CREATED           PLATFORM       SIZE        BLOB SIZE
myapp         v1        4c33b3b11666    25 seconds ago    linux/amd64    42.7 MiB    16.0 MiB
nginx         alpine    c94a22b036af    49 seconds ago    linux/amd64    42.7 MiB    16.0 MiB

注意：

nerdctl 构建的机制和 docker 是完全不同的。

docker 首先会检查本地是否有 Dockerfile 中 FROM 的镜像。如果有，直接使用。没有则通过网络下载镜像;
nerdctl 会根据 Dockerfile FROM参数指定镜像的域名去网上找这个镜像，找到后确认和本地同名镜像校验无误之后，才会使用本地的镜像构建新镜像。

举例：

通过tag 打标一个不存在域名的镜像
>ctr i tag docker.io/library/nginx:alpine abc.com/library/nginx:alpine

查看镜像
>nerdctl images
REPOSITORY               TAG       IMAGE ID        CREATED           PLATFORM       SIZE        BLOB SIZE
abc.com/library/nginx    alpine    c94a22b036af    1 minutes ago     linux/amd64    42.7 MiB    16.0 MiB
nginx                    alpine    c94a22b036af    17 minutes ago    linux/amd64    42.7 MiB    16.0 MiB

通过abc.com/library/nginx:alpine构建新镜像
>vim Dockerfile
FROM abc.com/library/nginx:alpine
RUN echo "hello world." > /usr/share/nginx/html/index.html

root@containerd(192.168.199.101)~/dockerfile>nerdctl build -t myapp:v1 ./
[+] Building 0.8s (3/3) FINISHED
 => [internal] load build definition from Dockerfile                                                                                                                                0.0s
 => => transferring dockerfile: 130B                                                                                                                                                0.0s
 => [internal] load .dockerignore                                                                                                                                                   0.0s
 => => transferring context: 2B                                                                                                                                                     0.0s
 => ERROR [internal] load metadata for abc.com/library/nginx:alpine                                                                                                                 0.7s
------
 > [internal] load metadata for abc.com/library/nginx:alpine:
------
Dockerfile:1
--------------------
   1 | >>> FROM abc.com/library/nginx:alpine
   2 |     RUN echo "hello world." > /usr/share/nginx/html/index.html
   3 |
--------------------
error: failed to solve: abc.com/library/nginx:alpine: abc.com/library/nginx:alpine: not found
FATA[0001] no image was built

构建时，直接就抛出了错误信息，这里要非常注意！

4.5 镜像标签TAG

>nerdctl images
REPOSITORY    TAG       IMAGE ID        CREATED               PLATFORM       SIZE        BLOB SIZE
nginx         alpine    c94a22b036af    About a minute ago    linux/amd64    42.7 MiB    16.0 MiB

>nerdctl tag nginx:alpine myapp:v1

>nerdctl images
REPOSITORY    TAG       IMAGE ID        CREATED               PLATFORM       SIZE        BLOB SIZE
myapp         v1        c94a22b036af    1 second ago          linux/amd64    42.7 MiB    16.0 MiB
nginx         alpine    c94a22b036af    About a minute ago    linux/amd64    42.7 MiB    16.0 MiB

4.6 删除镜像

>nerdctl images
REPOSITORY    TAG       IMAGE ID        CREATED               PLATFORM       SIZE        BLOB SIZE
myapp         v1        c94a22b036af    1 second ago          linux/amd64    42.7 MiB    16.0 MiB
nginx         alpine    c94a22b036af    About a minute ago    linux/amd64    42.7 MiB    16.0 MiB

>nerdctl rmi myapp:v1
Untagged: docker.io/library/myapp:v1@sha256:c94a22b036afa972426b82d5b0a49c959786005b4f6f81ac7467ca5538d0158f
Deleted: sha256:f1417ff83b319fbdae6dd9cd6d8c9c88002dcd75ecf6ec201c8c6894681cf2b5
Deleted: sha256:1003ff723696bfd596cd65592fa26554840e90780f6937e6ddccc909b8ed1443
Deleted: sha256:1d54586a1706c0af48668c10cbd8246626acb4fec01287be54cd9b26d72df15d
Deleted: sha256:c1cd5c8c68ef2336b2504336206d58931e9215a863a35a741f66aa3f4970b0f5
Deleted: sha256:f0fb842dea4179a94f1b8c2ac178e72690fa2b30e25e03a7a7893794fe9520a5
Deleted: sha256:f9cb3f1f1d3d7c591c4ab02118816fe6761a8f2f7b2500a5ec7421a42b8a5ea2
Deleted: sha256:31531248c7cbf5b31a8d9695c20041b9b3749b8c04b9831331ad93333fcf1474

>nerdctl images
REPOSITORY    TAG       IMAGE ID        CREATED          PLATFORM       SIZE        BLOB SIZE
nginx         alpine    c94a22b036af    2 minutes ago    linux/amd64    42.7 MiB    16.0 MiB

4.7 导出镜像

导出，不压缩
>nerdctl save nginx:alpine -o nginx.tar

导出且压缩
>nerdctl save nginx:alpine | gzip > nginx-alpine-image.tar.gz

比较两者大小，越大的镜像越压缩后越明显
>du -sh *
16M     nginx-alpine-image.tar.gz
17M     nginx.tar

4.8 导入镜像

>nerdctl load < nginx-alpine-image.tar.gz
或
>nerdctl load -i nginx-alpine-image.tar.gz

通过上面的展示，基本和docker无差别，其他镜像管理的功能不再赘述。

5 网络

在安装 cri-containerd-cni 时，网络插件也安装了。

>tar tvf cri-containerd-cni-1.7.0-linux-amd64.tar.gz
...
-rw-r--r-- root/root        57 2023-03-11 02:14 etc/crictl.yaml
drwxr-xr-x root/root         0 2023-03-11 02:13 etc/cni/
drwxr-xr-x root/root         0 2023-03-11 02:13 etc/cni/net.d/
-rw-r--r-- root/root       604 2023-03-11 02:13 etc/cni/net.d/10-containerd-net.conflist
...

5.1 查看网络

>nerdctl network ls
NETWORK ID      NAME              FILE
                containerd-net    /etc/cni/net.d/10-containerd-net.conflist
17f29b073143    bridge            /etc/cni/net.d/nerdctl-bridge.conflist
                host

5.2 创建桥接网络

>nerdctl network create -d bridge --subnet 10.244.0.0/16 mynet

>nerdctl network  ls
NETWORK ID      NAME              FILE
                containerd-net    /etc/cni/net.d/10-containerd-net.conflist
17f29b073143    bridge            /etc/cni/net.d/nerdctl-bridge.conflist
11c844f95e28    mynet             /etc/cni/net.d/nerdctl-mynet.conflist
                host
                none
                

查看创建的网络的配置文件                
>cat /etc/cni/net.d/nerdctl-mynet.conflist
{
  "cniVersion": "1.0.0",
  "name": "mynet",
  "nerdctlID": "11c844f95e2862126712e209cd3acbc68c137931c639633da9dfc17b3a464bde",
  "nerdctlLabels": {},
  "plugins": [
    {
      "type": "bridge",
      "bridge": "br-11c844f95e28",
      "isGateway": true,
      "ipMasq": true,
      "hairpinMode": true,
      "ipam": {
        "ranges": [
          [
            {
              "gateway": "10.244.0.1",
              "subnet": "10.244.0.0/16"
            }
          ]
        ],
        "routes": [
          {
            "dst": "0.0.0.0/0"
          }
        ],
        "type": "host-local"
      }
    },
    {
      "type": "portmap",
      "capabilities": {
        "portMappings": true
      }
    },
    {
      "type": "firewall",
      "ingressPolicy": "same-bridge"
    },
    {
      "type": "tuning"
    }
  ]
}

nerdctl 所使用的网络及模式和 docker 完全一致，这里不再赘述。

6 容器管理

nerdctl 和 dockerc-cli 如出一辙，nerdctl 出现的原因之一就是为了兼容 docker-cli，所以用法一致，这里只列举几个，其他使用请直接参考 docker-cli

6.1 启动容器

>nerdctl run --name ngx -d -p 80:80 nginx:alpine
20629b70f5c444dd6c379012a6519e21d16b38cc6affa0457e844c6bb3aaedc8

启动容器并指定特定网络(使用宿主机网络直接启动容器)
>nerdctl run --name ngx --net host -d nginx:alpine
49d08525ad53e8a1b85a8daf87b15d703768cbe1d61c598bdc4e39135e1c2b4e

6.2 查看容器

>nerdctl ps
CONTAINER ID    IMAGE                             COMMAND                   CREATED               STATUS    PORTS                 NAMES
0c671d5ac3f1    docker.io/library/nginx:alpine    "/docker-entrypoint.…"    About a minute ago    Up        0.0.0.0:80->80/tcp    ngx

查看所有容器
>nerdctl ps -a
CONTAINER ID    IMAGE                             COMMAND                   CREATED               STATUS                         PORTS                 NAMES
0c671d5ac3f1    docker.io/library/nginx:alpine    "/docker-entrypoint.…"    About a minute ago    Up                             0.0.0.0:80->80/tcp    ngx
5ff17a6ba473    docker.io/library/nginx:alpine    "/docker-entrypoint.…"    19 seconds ago        Exited (137) 14 seconds ago    0.0.0.0:80->80/tcp    ngx-1

查看容器详细信息
>nerdctl inspect ngx
[
    {
        "Id": "49d08525ad53e8a1b85a8daf87b15d703768cbe1d61c598bdc4e39135e1c2b4e",
        "Created": "2023-04-06T01:45:57.035984068Z",
        "Path": "/docker-entrypoint.sh",
        "Args": [
            "nginx",
            "-g",
            "daemon off;"
        ],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "Pid": 17233,
            "ExitCode": 0,
            "FinishedAt": "0001-01-01T00:00:00Z"
        },
        "Image": "docker.io/library/nginx:alpine",
        "ResolvConfPath": "",
        "HostnamePath": "/var/lib/nerdctl/1935db59/containers/default/49d08525ad53e8a1b85a8daf87b15d703768cbe1d61c598bdc4e39135e1c2b4e/hostname",
        "LogPath": "/var/lib/nerdctl/1935db59/containers/default/49d08525ad53e8a1b85a8daf87b15d703768cbe1d61c598bdc4e39135e1c2b4e/49d08525ad53e8a1b85a8daf87b15d703768cbe1d61c598bdc4e39135e1c2b4e-json.log",
        "Name": "ngx",
        "RestartCount": 0,
        "Driver": "overlayfs",
        "Platform": "linux",
        "AppArmorProfile": "",
        "Mounts": null,
        "Config": {
            "Hostname": "49d08525ad53",
            "AttachStdin": false,
            "Labels": {
                "io.containerd.image.config.stop-signal": "SIGQUIT",
                "nerdctl/extraHosts": "null",
                "nerdctl/hostname": "49d08525ad53",
                "nerdctl/log-uri": "binary:///usr/local/bin/nerdctl?_NERDCTL_INTERNAL_LOGGING=%2Fvar%2Flib%2Fnerdctl%2F1935db59",
                "nerdctl/name": "ngx",
                "nerdctl/namespace": "default",
                "nerdctl/networks": "[\"host\"]",
                "nerdctl/platform": "linux/amd64",
                "nerdctl/state-dir": "/var/lib/nerdctl/1935db59/containers/default/49d08525ad53e8a1b85a8daf87b15d703768cbe1d61c598bdc4e39135e1c2b4e"
            }
        },
        "NetworkSettings": {
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "192.168.199.101",
            "IPPrefixLen": 24,
            "MacAddress": "52:54:00:e8:88:2b",
            "Networks": {
                "unknown-br-11c844f95e28": {
                    "IPAddress": "10.244.0.1",
                    "IPPrefixLen": 16,
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "56:2d:35:b5:47:9c"
                },
                "unknown-eth0": {
                    "IPAddress": "192.168.199.101",
                    "IPPrefixLen": 24,
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "52:54:00:e8:88:2b"
                },
                "unknown-nerdctl0": {
                    "IPAddress": "10.4.0.1",
                    "IPPrefixLen": 24,
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "86:26:27:ea:8a:a2"
                }
            }
        }
    }
]

6.3 删除容器

1.stop 容器
2.删除 容器
或者
强制删除容器

>nerdctl stop ngx
>nerdctl rm ngx
or
>nerdctl rm -f ngx

--- EOF ---

标签：Containerd,MiB,nerdctl,nginx,containerd,docker,高阶,alpine
From： https://www.cnblogs.com/hukey/p/17291811.html

Containerd高阶命令行工具 - nerdctl 【2】