使用 Docker Registry API 获取托管在 Docker Hub 上的公共 Docker 映像的清单

Question

我正在尝试找出正确的 URL 用于此目的。例如，假设我想获取 alpine:3.9 标签的清单。我已经尝试过https://hub.docker.com/v2/repositories/library/alpine/manifests/3.9，但这会产生 404 错误。

我发现 Docker Hub 的注册表实现与他们的文档并不匹配。比如https://docs.docker.com/registry/spec/api/#tags表示获取标签列表的URL是v2//tags/list，但是在查询Docker Hub的时候，其实需要去掉URL的“列表”部分：@ 987654323@。因此，在查询 Docker Hub 注册表时，这让我对他们的文档产生了疑问。

Answer 1

以下是一些使用 V2 端点的 cURL 命令。我对hub.docker.com 端点的用途（https://hub.docker.com/v2/users/login、https://hub.docker.com/v2/repositories/library/ 等）感到非常困惑，但我认为/v2/ 完全是红鲱鱼，与 registry V2 API？使用hub.docker.com 的This 文章可以获得标签，但不能获得清单。

DOCKERHUB_USERNAME=$(jq -r '.username' < ~/.secrets/docker.json)
DOCKERHUB_PASSWORD=$(jq -r '.password' < ~/.secrets/docker.json)

TARGET_NS_REPO=library/debian

# yes, you need a new token for each repository, maybe you can have multiple scopes though?
PARAMS="service=registry.docker.io&scope=repository:$TARGET_NS_REPO:pull"
TOKEN=$(curl --user "$DOCKERHUB_USERNAME:$DOCKERHUB_PASSWORD" \
    "https://auth.docker.io/token?$PARAMS" \
    | jq -r '.token'
)

curl "https://registry-1.docker.io/v2/$TARGET_NS_REPO/tags/list" \
    -H "Authorization:Bearer $TOKEN" \
    | jq '.tags[:10]'

TAG="10-slim"
curl "https://registry-1.docker.io/v2/$TARGET_NS_REPO/manifests/$TAG" \
    -H "Authorization:Bearer $TOKEN" \
    | jq '.fsLayers'

输出：

[
  "10-slim",
  "10.0-slim",
  "10.0",
  "10",
  "6.0.10",
  "6.0.8",
  "6.0.9",
  "6.0",
  "6",
  "7-slim"
]
[
  {
    "blobSum": "sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4"
  },
  {
    "blobSum": "sha256:1ab2bdfe97783562315f98f94c0769b1897a05f7b0395ca1520ebee08666703b"
  }
]

对其进行逆向工程

我基本上不得不使用mitmproxy 对此进行逆向工程。如果您想知道其他方法是如何工作的：

1. 安装/运行mitmproxy。通过以下方式检查它是否正常工作：

curl -x localhost:8080 http://mitm.it/cert/pem  # should print out a cert

1. 获取/安装它的证书（给 MITM 自己）：

# Ubuntu 18.04, other distros may vary
MITM_CERT_PATH=/usr/local/share/ca-certificates/mitmproxy.crt
sudo cp ~/.mitmproxy/mitmproxy-ca-cert.cer "$MITM_CERT_PATH"
sudo chown root:root "$MITM_CERT_PATH"
sudo chmod 644 "$MITM_CERT_PATH"
sudo update-ca-certificates

# Verify MITM root cert accepted
curl -x localhost:8080 https://sha256.badssl.com/

# Troubleshooting
# - see if installed (https://unix.stackexchange.com/a/97252/42385)
awk -v cmd='openssl x509 -noout -subject' \
    '/BEGIN/{close(cmd)};{print | cmd}' \
    < /etc/ssl/certs/ca-certificates.crt \
    | grep -i mitmproxy

# - print the cert used (OpenSSL 1.1.0+)
openssl s_client -proxy localhost:8080 -showcerts -connect sha256.badssl.com:443 </dev/null

如果需要，稍后卸载证书

sudo rm /usr/local/share/ca-certificates/mitmproxy.crt
sudo update-ca-certificates

Check not in the list
awk -v cmd='openssl x509 -noout -subject' \
    '/BEGIN/{close(cmd)};{print | cmd}' \
    < /etc/ssl/certs/ca-certificates.crt \
    | grep -i mitmproxy

# Double-check MITM root cert rejected
curl -x localhost:8080 https://sha256.badssl.com/

1. 设置HTTPS_PROXY 运行dockerd（如果它已经在运行，则停止服务）

sudo HTTPS_PROXY=http://localhost:8080/ dockerd  # bash
# sudo env HTTPS_PROXY=http://localhost:8080/ dockerd  # fish

4. 告诉 Docker 守护进程做某事，例如docker pull alpine。在 mitmproxy 中，您会看到类似

Flows
   GET https://registry-1.docker.io/v2/
       ← 401 application/json 87b 213ms
   GET https://auth.docker.io/token?account=youraccount&scope=repository%3Alibrary%2Fal
       pine%3Apull&service=registry.docker.io
       ← 200 application/json 4.18k 245ms
>> GET https://registry-1.docker.io/v2/library/alpine/manifests/latest
       ← 200 application/vnd.docker.distribution.manifest.list.v2+json 1.6k 294ms
   GET https://registry-1.docker.io/v2/library/alpine/manifests/sha256:57334c50959f26ce
       1ee025d08f136c2292c128f84e7b229d1b0da5dac89e9866
       ← 200 application/vnd.docker.distribution.manifest.v2+json 528b 326ms
   GET https://registry-1.docker.io/v2/library/alpine/blobs/sha256:b7b28af77ffec6054d13
       378df4fdf02725830086c7444d9c278af25312aa39b9
       ← 307 text/html 242b 288ms
   GET https://registry-1.docker.io/v2/library/alpine/blobs/sha256:0503825856099e6adb39
       c8297af09547f69684b7016b7f3680ed801aa310baaa
       ← 307 text/html 242b 322ms
   GET https://production.cloudflare.docker.com/registry-v2/docker/registry/v2/blobs/sh
       a256/b7/b7b28af77ffec6054d13378df4fdf02725830086c7444d9c278af25312aa39b9/data?…
       ← 200 application/octet-stream 1.48k 191ms
   GET https://production.cloudflare.docker.com/registry-v2/docker/registry/v2/blobs/sh
       a256/05/0503825856099e6adb39c8297af09547f69684b7016b7f3680ed801aa310baaa/data?…
       ← 200 application/octet-stream 2.66m 207ms
⇩  [27/32]                                                                     [*:8080]

5. 检查请求。选择...manifests/latest 请求查看：

Flow Details
2019-08-20 13:43:44 GET https://registry-1.docker.io/v2/library/alpine/manifests/latest
         ← 200 OK application/vnd.docker.distribution.manifest.list.v2+json 1.6k 294ms
       [[ Request ]]             Response                  Detail
Host:             registry-1.docker.io
User-Agent:       docker/19.03.1 go/go1.12.5 git-commit/74b1e89 kernel/4.15.0-55-generic
                  os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.1\\(linux\\))
Accept:           application/vnd.docker.distribution.manifest.v2+json
Accept:           application/vnd.docker.distribution.manifest.list.v2+json
Accept:           application/vnd.oci.image.index.v1+json
Accept:           application/vnd.docker.distribution.manifest.v1+prettyjws
Accept:           application/json
Accept:           application/vnd.oci.image.manifest.v1+json
Authorization:    Bearer eyJhbGci...(a big JWT returned by the auth.docker.io req.)
Accept-Encoding:  gzip
Connection:       close

Answer 2

我完全无法让 Docker 的 Registry API 与 DockerHub 一起用于清单 :-(

如果你找到方法，请告诉我。

很遗憾，DockerHub 的 UI 也不包含清单详细信息。

如果您有权访问 Google Container Registry (GCR)，那么它的 Docker Registry API 实现就可以工作。 GCR 的 UI 还包括清单和摘要。

我在这里总结了我的经验：

https://medium.com/google-cloud/adventures-w-docker-manifests-78f255d662ff