Spinnaker：AKS 帐户未显示在 UI 上答案

【问题标题】：Spinnaker:AKS account not showing on UISpinnaker：AKS 帐户未显示在 UI 上
【发布时间】：2019-12-02 19:39:41
【问题描述】：

我已使用以下命令将 Spinnaker 云提供商配置为 kubernetes

hal config provider kubernetes enable
kubectl config current-context
CONTEXT=$(kubectl config current-context)
hal config provider kubernetes account add my-k8s-v2-account     --provider-version v2     --context $CONTEXT
hal config features edit --artifacts true

但此帐户在 Spinnaker UI 上不可见

并在日志中显示如下错误

Nov 29 12:07:43 47184UW2DDevLVM2 gate[34594]: 2019-11-29 12:07:43.860 ERROR 34594 --- [TaskScheduler-5] c.n.s.g.s.DefaultProviderLookupService   : Unable to refresh account details cache, reason: timeout

请指教..谢谢..

这是我的 hal deploy diff 命令输出

+ Get current deployment
  Success
+ Determine config diff
  Success
~ EDITED
default.persistentStorage.redis
- port 6379 -> null
- host localhost -> null
~ EDITED
telemetry

我已经配置了新的虚拟机并从头开始完成了所有安装过程，但仍然是同样的问题:(

这里是 ~/.kube/config 文件

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: xxx
    server: https://xxx:443
  name:xxx
contexts:
- context:
    cluster: xxx
    user: xxx
  name: xxx
current-context: xxx
kind: Config
preferences: {}
users:
- name: xxx
  user:
    client-certificate-data: xxx
    client-key-data: xxx
    token: xxx

这里是 ~/.hal/config 文件

currentDeployment: default
deploymentConfigurations:
- name: default
  version: 1.17.2
  providers:
    appengine:
      enabled: false
      accounts: []
    aws:
      enabled: false
      accounts: []
      bakeryDefaults:
        baseImages: []
      defaultKeyPairTemplate: '{{name}}-keypair'
      defaultRegions:
      - name: xxx
      defaults:
        iamRole: BaseIAMRole
    ecs:
      enabled: false
      accounts: []
    azure:
      enabled: false
      accounts: []
      bakeryDefaults:
        templateFile: azure-linux.json
        baseImages: []
    dcos:
      enabled: false
      accounts: []
      clusters: []
    dockerRegistry:
      enabled: false
      accounts: []
    google:
      enabled: false
      accounts: []
      bakeryDefaults:
        templateFile: gce.json
        baseImages: []
        zone: us-central1-f
        network: default
        useInternalIp: false
    kubernetes:
      enabled: true
      accounts:
      - name: xxx
        requiredGroupMembership: []
        providerVersion: V2
        permissions: {}
        dockerRegistries: []
        context: xxx
        configureImagePullSecrets: true
        cacheThreads: 1
        namespaces: []
        omitNamespaces: []
        kinds: []
        omitKinds: []
        customResources: []
        cachingPolicies: []
        kubeconfigFile: /home/xxx/.kube/config
        oAuthScopes: []
        onlySpinnakerManaged: false
      primaryAccount: xxx
    oracle:
      enabled: false
      accounts: []
      bakeryDefaults:
        templateFile: oci.json
        baseImages: []
    cloudfoundry:
      enabled: false
      accounts: []
  deploymentEnvironment:
    size: SMALL
    type: LocalDebian
    imageVariant: SLIM
    updateVersions: true
    consul:
      enabled: false
    vault:
      enabled: false
    customSizing: {}
    sidecars: {}
    initContainers: {}
    hostAliases: {}
    affinity: {}
    tolerations: {}
    nodeSelectors: {}
    gitConfig:
      upstreamUser: spinnaker
    livenessProbeConfig:
      enabled: false
    haServices:
      clouddriver:
        enabled: false
        disableClouddriverRoDeck: false
      echo:
        enabled: false
  persistentStorage:
    persistentStoreType: azs
    azs:
      storageAccountName: xxx
      storageAccountKey: xxx
      storageContainerName: xxx
    gcs:
      rootFolder: front50
    redis: {}
    s3:
      rootFolder: front50
    oracle: {}
  features:
    auth: false
    fiat: false
    chaos: false
    entityTags: false
    artifacts: true
  metricStores:
    datadog:
      enabled: false
      tags: []
    prometheus:
      enabled: false
      add_source_metalabels: true
    stackdriver:
      enabled: false
    newrelic:
      enabled: false
      tags: []
    period: 30
    enabled: false
  notifications:
    slack:
      enabled: false
    twilio:
      enabled: false
      baseUrl: https://api.twilio.com/
    github-status:
      enabled: false
  timezone: America/Los_Angeles
  ci:
    jenkins:
      enabled: false
      masters: []
    travis:
      enabled: false
      masters: []
    wercker:
      enabled: false
      masters: []
    concourse:
      enabled: false
      masters: []
    gcb:
      enabled: false
      accounts: []
  repository:
    artifactory:
      enabled: false
      searches: []
  security:
    apiSecurity:
      ssl:
        enabled: false
      overrideBaseUrl: http://xxx:8084/
    uiSecurity:
      ssl:
        enabled: false
      overrideBaseUrl: http://xxx:9000/
    authn:
      oauth2:
        enabled: false
        client: {}
        resource: {}
        userInfoMapping: {}
      saml:
        enabled: false
        userAttributeMapping: {}
      ldap:
        enabled: false
      x509:
        enabled: false
      iap:
        enabled: false
      enabled: false
    authz:
      groupMembership:
        service: EXTERNAL
        google:
          roleProviderType: GOOGLE
        github:
          roleProviderType: GITHUB
        file:
          roleProviderType: FILE
        ldap:
          roleProviderType: LDAP
      enabled: false
  artifacts:
    bitbucket:
      enabled: false
      accounts: []
    gcs:
      enabled: false
      accounts: []
    oracle:
      enabled: false
      accounts: []
    github:
      enabled: false
      accounts: []
    gitlab:
      enabled: false
      accounts: []
    gitrepo:
      enabled: false
      accounts: []
    http:
      enabled: false
      accounts: []
    helm:
      enabled: false
      accounts: []
    s3:
      enabled: false
      accounts: []
    maven:
      enabled: false
      accounts: []
    templates: []
  pubsub:
    enabled: false
    google:
      enabled: false
      pubsubType: GOOGLE
      subscriptions: []
      publishers: []
  canary:
    enabled: false
    serviceIntegrations:
    - name: google
      enabled: false
      accounts: []
      gcsEnabled: false
      stackdriverEnabled: false
    - name: prometheus
      enabled: false
      accounts: []
    - name: datadog
      enabled: false
      accounts: []
    - name: signalfx
      enabled: false
      accounts: []
    - name: aws
      enabled: false
      accounts: []
      s3Enabled: false
    - name: newrelic
      enabled: false
      accounts: []
    reduxLoggerEnabled: true
    defaultJudge: NetflixACAJudge-v1.0
    stagesEnabled: true
    templatesEnabled: true
    showAllConfigsEnabled: true
  plugins:
    plugins: []
    enabled: false
    downloadingEnabled: false
    pluginConfigurations:
      plugins: {}
  webhook:
    trust:
      enabled: false
  telemetry:
    enabled: false
    endpoint: https://stats.spinnaker.io
    instanceId: xxx
    connectionTimeoutMillis: 3000
    readTimeoutMillis: 5000

这里是用于安装 spinnaker 的命令

az login
az aks get-credentials --resource-group xxx --name xxx
curl -O https://raw.githubusercontent.com/spinnaker/halyard/master/install/debian/InstallHalyard.sh
sudo bash InstallHalyard.sh --user xxx
hal config provider kubernetes enable
CONTEXT=$(kubectl config current-context)
hal config provider kubernetes account add xxx \
    --provider-version v2 \
    --context $CONTEXT
hal config features edit --artifacts true  
hal config deploy edit --type localdebian
hal config storage azs edit --storage-account-name xxx --storage-account-key xxx
hal config storage edit --type azs
hal version list
hal config version edit --version 1.17.2
sudo hal deploy apply
echo "host: 0.0.0.0" | tee \
    ~/.hal/default/service-settings/gate.yml \
    ~/.hal/default/service-settings/deck.yml
hal config security ui edit \
    --override-base-url http://xxx:9000/
hal config security api edit \
    --override-base-url http://xxx:8084/
sudo hal deploy apply

在异常日志下方找到

Dec  2 11:12:07 47184UW2DDevLVM2 clouddriver[23908]: 2019-12-02 11:12:07.424 ERROR 23908 --- [1-7002-exec-105] c.n.s.k.w.e.GenericExceptionHandlers     : Internal Server Error
Dec  2 11:12:07 47184UW2DDevLVM2 clouddriver[23908]: java.lang.NullPointerException: null
Dec  2 11:12:07 47184UW2DDevLVM2 clouddriver[23908]: #011at com.netflix.spinnaker.clouddriver.kubernetes.health.KubernetesHealthIndicator.health(KubernetesHealthIndicator.java:48) ~[clouddriver-kubernetes-6.4.1-20191111102213.jar:6.4.1-20191111102213]
Dec  2 11:12:07 47184UW2DDevLVM2 clouddriver[23908]: #011at org.springframework.boot.actuate.health.CompositeHealthIndicator.health(CompositeHealthIndicator.java:95) ~[spring-boot-actuator-2.1.7.RELEASE.jar:2.1.7.RELEASE]
Dec  2 11:12:07 47184UW2DDevLVM2 clouddriver[23908]: #011at org.springframework.boot.actuate.health.HealthEndpoint.health(HealthEndpoint.java:50) ~[spring-boot-actuator-2.1.7.RELEASE.jar:2.1.7.RELEASE]
Dec  2 11:12:07 47184UW2DDevLVM2 clouddriver[23908]: #011at org.springframework.boot.actuate.health.HealthEndpointWebExtension.health(HealthEndpointWebExtension.java:53) ~[spring-boot-actuator-2.1.7.RELEASE.jar:2.1.7.RELEASE]

加上 localhost 7002 没有响应

hexunix@47184UW2DDevLVM2:~$ curl -v http://localhost:7002/credentials
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 7002 (#0)
> GET /credentials HTTP/1.1
> Host: localhost:7002
> User-Agent: curl/7.58.0
> Accept: */*
>

【问题讨论】：

标签： kubernetes spinnaker spinnaker-halyard

【解决方案1】：

这就是我在我的环境中所做的事情

kubeconfig_path="/home/root/.hal/kube-config"
kubernetes_account="my-account"
docker_registry="docker.io"
hal config provider kubernetes account add $kubernetes_account --provider-version v2 \
  --kubeconfig-file "$kubeconfig_path" \
  --context $(kubectl config current-context --kubeconfig "$kubeconfig_path") \
  --omit-namespaces=kube-system,kube-public \
  --docker-registries "$docker_registry"

进行必要的更新并应用更改。它应该可以工作。

从 hal config 中可以清楚地看到添加了 kubernetes 帐户。

    kubernetes:
      enabled: true
      accounts:
      - name: xxx
        requiredGroupMembership: []
        providerVersion: V2
        permissions: {}
        dockerRegistries: []
        context: xxx
        configureImagePullSecrets: true
        cacheThreads: 1
        namespaces: []
        omitNamespaces: []
        kinds: []
        omitKinds: []
        customResources: []
        cachingPolicies: []
        kubeconfigFile: /home/xxx/.kube/config
        oAuthScopes: []
        onlySpinnakerManaged: false
      primaryAccount: xxx

【讨论】：

@P Ekambaram - kubeconfig_path 不应该是 ~/.kube/config 而不是 ~/.hal/config 吗？因为 ~/.hal/config 在这个命令上没有响应 k8s custername - kubectl config current-context --kubeconfig "$kubeconfig_path"... 但是我用 ~/.kube/config 路径再次尝试了这个命令，但它仍然没有工作......我也执行了“sudo hal deploy apply”......
@p Ekambaram - 我已经用更新的路径尝试过这个，但它仍然无法正常工作。我已经在虚拟机中完成了至少 4-5 次来回配置，并启用了其他云提供商禁用，所以可能是下周我会创建一个新的虚拟机，所有配置都是从头开始的，然后会看到..谢谢你的回复..
分享/home/spinnaker/.hal/config
@P Ekambaram - 我已经配置了新的虚拟机并从头开始完成了所有安装过程，但仍然是同样的问题 :( 我用 ~/.kube/config 和 ~/.hal/config 编辑了问题文件..
分享/home/spinnaker/.hal/config