您的理解是正确的：使用 kops，您无法更改现有集群的 CIDR 块；它卡在创建它的 VPC 中，you can't change the CIDR block of a VPC:
VPC 的 IP 地址范围由关联的 CIDR 块组成
用它。创建 VPC 时选择一个 CIDR 块，然后
以后可以添加或删除辅助 CIDR 块。 您设置的 CIDR 块
无法更改创建 VPC 时添加的内容，但您可以添加和
删除辅助 CIDR 块以更改 IP 地址范围
专有网络。 （强调我的）
这就引出了第二点：迁移集群。这可以分为两个阶段：
- 迁移由
kops 管理的基础架构
- 迁移集群上的工作负载
1.迁移kops管理的基础架构
您将需要迁移（即重新创建）kops 集群本身：ec2 实例、kops InstanceGroups 和 Cluster 对象、各种 AWS 基础设施等。为此，您可以使用 kops toolbox template 命令：
kops toolbox template --values /path/to/values.yaml --template /path/to/cluster/template.yaml > /path/to/output/cluster.yaml
kops create -f /path/to/output/cluster.yaml
这是一个类似 Helm 的工具，允许您模板化您的 kops 集群配置并传入不同的 values.yaml 文件。您可能希望将此命令包含在一个小型 shell 脚本包装器或 Makefile 中，以创建一键式集群部署，从而轻松且可重复地设置您的 k8s 集群基础设施。
示例集群 template.yaml 文件和 values.yaml 文件可能如下所示，其中包括 Cluster 以及 master、worker 和 autoscale InstanceGroups 的规范。
# template.yaml
{{ $clusterSubdomain := (env "CLUSTER_SUBDOMAIN") }}
{{ $subnetCidr := (env "SUBNET_CIDR") }}
apiVersion: kops/v1alpha2
kind: Cluster
metadata:
name: {{ $clusterSubdomain }}.k8s.example.io
spec:
hooks:
- manifest: |
[Unit]
Description=Create example user
ConditionPathExists=!/home/example/.ssh/authorized_keys
[Service]
Type=oneshot
ExecStart=/bin/sh -c 'useradd example && echo "{{ .examplePublicKey }}" > /home/example/.ssh/authorized_keys'
name: useradd-example.service
roles:
- Node
- Master
- manifest: |
Type=oneshot
ExecStart=/usr/bin/coreos-cloudinit --from-file=/home/core/cloud-config.yaml
name: reboot-window.service
roles:
- Node
- Master
kubeAPIServer:
authorizationRbacSuperUser: admin
featureGates:
TaintBasedEvictions: "true"
kubeControllerManager:
featureGates:
TaintBasedEvictions: "true"
horizontalPodAutoscalerUseRestClients: false
kubeScheduler:
featureGates:
TaintBasedEvictions: "true"
kubelet:
featureGates:
TaintBasedEvictions: "true"
fileAssets:
- content: |
yes
name: docker-1.12
path: /etc/coreos/docker-1.12
roles:
- Node
- Master
- content: |
#cloud-config
coreos:
update:
reboot-strategy: "etcd-lock"
locksmith:
window-start: {{ .locksmith.windowStart }}
window-length: {{ .locksmith.windowLength }}
name: cloud-config.yaml
path: /home/core/cloud-config.yaml
roles:
- Node
- Master
api:
dns: {}
authorization:
rbac: {}
channel: stable
cloudProvider: aws
configBase: s3://my-bucket.example.io/{{ $clusterSubdomain }}.k8s.example.io
etcdClusters:
- etcdMembers:
- instanceGroup: master-{{ .zone }}
name: a
name: main
- etcdMembers:
- instanceGroup: master-{{ .zone }}
name: a
name: events
iam:
allowContainerRegistry: true
legacy: false
kubernetesApiAccess:
- {{ .apiAccessCidr }}
kubernetesVersion: {{ .k8sVersion }}
masterPublicName: api.{{ $clusterSubdomain }}.k8s.example.io
networkCIDR: {{ .vpcCidr }}
networkID: {{ .vpcId }}
networking:
canal: {}
nonMasqueradeCIDR: 100.64.0.0/10
sshAccess:
- {{ .sshAccessCidr }}
subnets:
- cidr: {{ $subnetCidr }}
name: {{ .zone }}
type: Public
zone: {{ .zone }}
topology:
dns:
type: Public
masters: public
nodes: public
---
apiVersion: kops/v1alpha2
kind: InstanceGroup
metadata:
labels:
kops.k8s.io/cluster: {{ $clusterSubdomain }}.k8s.example.io
name: master-{{ .zone }}
spec:
{{- if .additionalSecurityGroups }}
additionalSecurityGroups:
{{- range .additionalSecurityGroups }}
- {{ . }}
{{- end }}
{{- end }}
image: {{ .image }}
machineType: {{ .awsMachineTypeMaster }}
maxSize: 1
minSize: 1
nodeLabels:
kops.k8s.io/instancegroup: master-{{ .zone }}
role: Master
subnets:
- {{ .zone }}
---
apiVersion: kops/v1alpha2
kind: InstanceGroup
metadata:
labels:
kops.k8s.io/cluster: {{ $clusterSubdomain }}.k8s.example.io
name: nodes
spec:
{{- if .additionalSecurityGroups }}
additionalSecurityGroups:
{{- range .additionalSecurityGroups }}
- {{ . }}
{{- end }}
{{- end }}
image: {{ .image }}
machineType: {{ .awsMachineTypeNode }}
maxSize: {{ .nodeCount }}
minSize: {{ .nodeCount }}
nodeLabels:
kops.k8s.io/instancegroup: nodes
role: Node
subnets:
- {{ .zone }}
---
apiVersion: kops/v1alpha2
kind: InstanceGroup
metadata:
name: ag.{{ $clusterSubdomain }}.k8s.example.io
labels:
kops.k8s.io/cluster: {{ $clusterSubdomain }}.k8s.example.io
spec:
{{- if .additionalSecurityGroups }}
additionalSecurityGroups:
{{- range .additionalSecurityGroups }}
- {{ . }}
{{- end }}
{{- end }}
image: {{ .image }}
machineType: {{ .awsMachineTypeAg }}
maxSize: 10
minSize: 1
nodeLabels:
kops.k8s.io/instancegroup: ag.{{ $clusterSubdomain }}.k8s.example.io
role: Node
subnets:
- {{ .zone }}
还有 values.yaml 文件：
# values.yaml:
region: us-west-2
zone: us-west-2a
environment: staging
image: ami-abc123
awsMachineTypeNode: c5.large
awsMachineTypeMaster: m5.xlarge
awsMachineTypeAg: c5.large
nodeCount: "2"
k8sVersion: "1.9.3"
vpcId: vpc-abc123
vpcCidr: 172.23.0.0/16
apiAccessCidr: <e.g. office ip>
sshAccessCidr: <e.g. office ip>
additionalSecurityGroups:
- sg-def234 # kubernetes-standard
- sg-abc123 # example scan engine targets
examplePublicKey: "ssh-rsa ..."
locksmith:
windowStart: Mon 16:00 # 8am Monday PST
windowLength: 4h
2。迁移集群上的工作负载
没有任何 Ark 的实际操作经验，it does seem to fit your use case well：
集群迁移
使用备份和恢复
Heptio Ark 可以帮助您将资源从一个集群移植到
另一个，只要将每个 Ark Config 指向同一个云对象
贮存。在这种情况下，我们还假设您的集群是
由同一云提供商托管。请注意，Heptio Ark 没有
支持跨云提供商迁移持久卷。
(Cluster 1) Assuming you haven’t already been checkpointing your data with the Ark schedule operation, you need to first back up your
整个集群（根据需要更换）：
ark backup create <BACKUP-NAME>
The default TTL is 30 days (720 hours); you can use the --ttl flag to change this as necessary.
(Cluster 2) Make sure that the persistentVolumeProvider and backupStorageProvider fields in the Ark Config match the ones from
集群 1，以便您的新 Ark 服务器实例指向
同一个桶。
(Cluster 2) Make sure that the Ark Backup object has been created. Ark resources are synced with the backup files available in cloud
存储。
(Cluster 2) Once you have confirmed that the right Backup (<BACKUP-NAME>) is now present, you can restore everything with:
ark restore create --from-backup <BACKUP-NAME>
在 AWS 集群上配置 Ark 似乎很简单：https://github.com/heptio/ark/blob/master/docs/aws-config.md。
使用 kops 工具箱脚本和 Ark 配置进行一些初始设置后，您应该有一种干净、可重复的方式来迁移集群并将您的宠物变成牛，正如 meme 所说的那样。