在 minikube 上设置 kubeflow 管道时的问题

Question

我有一个在 macos 上运行的 minikube。尝试设置 kubeflow 管道时，我得到以下输出：

(base) ~/ml $ export PIPELINE_VERSION=1.7.0
(base) ~/ml $ kubectl apply -k "github.com/kubeflow/pipelines/manifests/kustomize/cluster-scoped-resources?ref=$PIPELINE_VERSION"
namespace/kubeflow created
customresourcedefinition.apiextensions.k8s.io/clusterworkflowtemplates.argoproj.io unchanged
customresourcedefinition.apiextensions.k8s.io/cronworkflows.argoproj.io unchanged
customresourcedefinition.apiextensions.k8s.io/workfloweventbindings.argoproj.io unchanged
customresourcedefinition.apiextensions.k8s.io/workflows.argoproj.io unchanged
customresourcedefinition.apiextensions.k8s.io/workflowtemplates.argoproj.io unchanged
serviceaccount/kubeflow-pipelines-cache-deployer-sa created
clusterrole.rbac.authorization.k8s.io/kubeflow-pipelines-cache-deployer-clusterrole unchanged
clusterrolebinding.rbac.authorization.k8s.io/kubeflow-pipelines-cache-deployer-clusterrolebinding unchanged
unable to recognize "github.com/kubeflow/pipelines/manifests/kustomize/cluster-scoped-resources?ref=1.7.0": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"
unable to recognize "github.com/kubeflow/pipelines/manifests/kustomize/cluster-scoped-resources?ref=1.7.0": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"
unable to recognize "github.com/kubeflow/pipelines/manifests/kustomize/cluster-scoped-resources?ref=1.7.0": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"

(base) ~/ml $ kubectl wait --for condition=established --timeout=60s crd/applications.app.k8s.io
Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "applications.app.k8s.io" not found
(base) ~/ml $  
(base) ~/ml $ kubectl get crd -A
NAME                                   CREATED AT
clusterworkflowtemplates.argoproj.io   2021-12-18T15:28:31Z
cronworkflows.argoproj.io              2021-12-18T15:28:31Z
workfloweventbindings.argoproj.io      2021-12-18T15:28:31Z
workflows.argoproj.io                  2021-12-18T15:28:31Z
workflowtemplates.argoproj.io          2021-12-18T15:28:31Z

具体是什么意思：

unable to recognize "github.com/kubeflow/pipelines/manifests/kustomize/cluster-scoped-resources?ref=1.7.0": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"

是不是下面这个错误的根本原因：

Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "applications.app.k8s.io" not found

 (base) ~/ml $ minikube version
minikube version: v1.24.0
commit: 76b94fb3c4e8ac5062daf70d60cf03ddcc0a741b

(base) ~/ml $ kubectl api-resources --api-group=apiextensions.k8s.io -o wide
NAME                        SHORTNAMES   APIVERSION                NAMESPACED   KIND                       VERBS
customresourcedefinitions   crd,crds     apiextensions.k8s.io/v1   false        CustomResourceDefinition   [create delete deletecollection get list patch update watch]

Answer 1

事实证明，kubeflow 管道 1.7.0 不适用于高于 1.22 的 kubernetes 版本。我用的是 kubernetes 1.21.8 和 minikube，安装 kubeflow 管道 1.7.0 没有问题。

是的，这是正确的行为。

你提到过：

具体是什么意思：

unable to recognize "github.com/kubeflow/pipelines/manifests/kustomize/cluster-scoped-resources?ref=1.7.0": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"

这是直接连接到Kubernetes version 1.22 api changes：

v1.22 版本将停止提供我们在下面列出的 API 版本。这些都是以前被弃用的测试版 API，有利于更新和更稳定的 API 版本。

• ValidatingWebhookConfiguration 和 MutatingWebhookConfiguration API 的测试版（admissionregistration.k8s.io/v1beta1 API 版本）
• 测试版CustomResourceDefinition API (apiextensions.k8s.io/v1beta1)
• 测试版APIService API (apiregistration.k8s.io/v1beta1)
• 测试版TokenReview API (authentication.k8s.io/v1beta1)
• SubjectAccessReview、LocalSubjectAccessReview、SelfSubjectAccessReview 的 Beta API 版本（API 版本来自 authorization.k8s.io/v1beta1）
• 测试版CertificateSigningRequest API（certificates.k8s.io/v1beta1）
• 测试版Lease API (coordination.k8s.io/v1beta1)
• 所有 beta Ingress API（extensions/v1beta1 和 networking.k8s.io/v1beta1 API 版本）

从 1.22 版开始，无法使用 apiextensions.k8s.io/v1beta1（此 API 不再可用），如果您想使用此类 API 安装管道，则只能使用 Kubernetes 1.21 版。