为什么 K3s 系统升级控制器失败并显示“not found, requeuing”？答案

【问题标题】：Why does the K3s system upgrade controller fail with "not found, requeuing"?为什么 K3s 系统升级控制器失败并显示“not found, requeuing”？
【发布时间】：2022-10-25 04:42:05
【问题描述】：

我安装了系统升级控制器并应用了这个计划清单：

apiVersion: upgrade.cattle.io/v1
kind: Plan
metadata:
  name: master-plan
  namespace: system-upgrade
spec:
  concurrency: 1
  cordon: true
  nodeSelector:
    matchExpressions:
    - key: k3s-master-upgrade
      operator: In
      values:
      - "true"
  serviceAccountName: system-upgrade
  upgrade:
    image: rancher/k3s-upgrade
  channel: https://update.k3s.io/v1-release/channels/stable
---
apiVersion: upgrade.cattle.io/v1
kind: Plan
metadata:
  name: worker-plan
  namespace: system-upgrade
spec:
  concurrency: 1
  cordon: true
  nodeSelector:
    matchExpressions:
    - key: k3s-worker-upgrade
      operator: In
      values:
      - "true"
  prepare:
    args:
    - prepare
    - master-plan
    image: rancher/k3s-upgrade
  serviceAccountName: system-upgrade
  upgrade:
    image: rancher/k3s-upgrade
  channel: https://update.k3s.io/v1-release/channels/stable

我应用并检查了标签：

$ kubectl label node crux k3s-worker-upgrade=true
$ kubectl describe nodes crux | grep k3s-worker-upgrade
                    k3s-worker-upgrade=true

$ kubectl label node nemo k3s-master-upgrade=true
$ kubectl describe nodes nemo | grep k3s-master-upgrade
                    k3s-master-upgrade=true

根据kubectl get nodes，我还在v1.23.6+k3s1，但稳定频道在v1.24.4+k3s1。

我收到以下错误：

$ kubectl -n system-upgrade logs deployment.apps/system-upgrade-controller

time="2022-09-12T11:29:31Z" level=error msg="error syncing 'system-upgrade/apply-worker-plan-on-crux-with-4190e4adda3866e909fc7735c1-f0dff': handler system-upgrade-controller: jobs.batch \"apply-worker-plan-on-crux-with-4190e4adda3866e909fc7735c1-f0dff\" not found, requeuing"
time="2022-09-12T11:30:35Z" level=error msg="error syncing 'system-upgrade/apply-master-plan-on-nemo-with-4190e4adda3866e909fc7735c1-9cf4f': handler system-upgrade-controller: jobs.batch \"apply-master-plan-on-nemo-with-4190e4adda3866e909fc7735c1-9cf4f\" not found, requeuing"

$ kubectl -n system-upgrade get jobs -o yaml

- apiVersion: batch/v1
  kind: Job
  metadata:
    labels:
      upgrade.cattle.io/controller: system-upgrade-controller
      upgrade.cattle.io/node: crux
      upgrade.cattle.io/plan: worker-plan
      upgrade.cattle.io/version: v1.24.4-k3s1
  status:
    conditions:
    - lastProbeTime: "2022-09-12T12:14:31Z"
      lastTransitionTime: "2022-09-12T12:14:31Z"
      message: Job was active longer than specified deadline
      reason: DeadlineExceeded
      status: "True"
      type: Failed
    failed: 1
    startTime: "2022-09-12T11:59:31Z"
    uncountedTerminatedPods: {}

【问题讨论】：

标签： k3s

【解决方案1】：

同样在这里。我只能通过使用二进制文件进行手动升级来升级 k3s。

wget https://github.com/k3s-io/k3s/releases/download/v1.24.4%2Bk3s1/k3s

systemctl stop k3s
cp ./k3s  /usr/local/bin/
systemctl start k3s

我已经在所有节点上完成了。

这似乎比自动升级、没有排水管、没有警戒线要快得多、容易得多……

【讨论】：