Multi Cluster Best Effort
On vSphere
Cloud Provider Interface is VMware vSphere
Pre
1.
Kubernetes
a.
HAproxy
b.
Master 3EA
c.
Node 5EA
2.
Cloud Provider
a.
VMware vSphere
3.
CLI
a.
kubectl
b.
helm
c.
clusterctl
START
•
VMware Cloud Provider를 위한 환경구성
# Cluster API Path : ~/.cluster-api/clusterctl.yaml
## -- Controller settings -- ##
VSPHERE_USERNAME: "altair"
VSPHERE_PASSWORD: "Megazone00!"
## -- Required workload cluster default settings -- ##
VSPHERE_SERVER: "vcsa01.ama.local"
VSPHERE_DATACENTER: "AMA DC"
VSPHERE_DATASTORE: "vsanDatastore"
VSPHERE_NETWORK: "Service PG_vlan3201"
VSPHERE_RESOURCE_POOL: "99-Workload"
VSPHERE_FOLDER: "Altair"
VSPHERE_TEMPLATE: "ubuntu-2204-kube-v1.28.0"
CONTROL_PLANE_ENDPOINT_IP: "30.20.1.241"
VIP_NETWORK_INTERFACE: "ens192"
VSPHERE_TLS_THUMBPRINT: "7E:3E:5F:75:43:A9:1B:96:49:E6:88:3A:3B:AD:7C:97:03:51:2C:53"
EXP_CLUSTER_RESOURCE_SET: "true"
VSPHERE_SSH_AUTHORIZED_KEY: "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCxM/cNmXnjWV87tRC/ySC22Ng/CA5jx79LR/A38i+lERQ1jbg+3G1OOfzxNyNscQylBpmMV0xJ2POelm8yYGlbjge5IA4y8Fqjh3nUwvk8HDcw2rg6O6FWNnQSBd9mO6I5np1hV+RptSzL2RxihxoRDVDqQxlAcjFQsYs6gSUSES/iWywQxdsOsyCO5KwSxVvSxYNbkswzReEllOW6jb5I1btU7jAAV9Hmc9Fvcz1S0jNeK4K7u7ic7kJ4t1q4dkImmFYHM7C+/RBNHDr+IX2YhAhWBPyJx3gZ6X/dkg1Hb34KTVaXKtWcjLFiQA7fd6leWG9eAqPgy6juDsVYZiLXOozONacOLFibFcUcy09w440reesxZXLx7q9uVyTcHXON4IhhxVuj8+XldRdJEQ+dxs9onhME4+PwUyLX5auWjDaHakofCZS9g2mQNtxPEfSc9SaYltRzDtqDUatUwPOPl9ir4Xc/+DaZWFuaT2pMG8tSmY0B9GtKCngXkJ7nId8= root@server"
VSPHERE_STORAGE_POLICY: "tkg-csi"
CPI_IMAGE_K8S_VERSION: "v1.28.0"
Shell
복사
•
CAPI / CAPV 구성요소 설치
# Install
clusterctl init -i vsphere -v 6
Using configuration File="/root/.cluster-api/clusterctl.yaml"
Fetching providers
Fetching File="infrastructure-components.yaml" Provider="vsphere" Type="InfrastructureProvider" Version="v1.8.7"
Fetching File="metadata.yaml" Provider="cluster-api" Type="CoreProvider" Version="v1.6.0"
Creating Namespace="cert-manager-test"
Creating Issuer="test-selfsigned" Namespace="cert-manager-test"
Creating Certificate="selfsigned-cert" Namespace="cert-manager-test"
Deleting Namespace="cert-manager-test"
Deleting Issuer="test-selfsigned" Namespace="cert-manager-test"
Deleting Certificate="selfsigned-cert" Namespace="cert-manager-test"
Skipping installing cert-manager as it is already installed
Using configuration File="/root/.cluster-api/clusterctl.yaml"
Shell
복사
•
Workload Template 생성
clusterctl generate cluster workload-01 \
--kubernetes-version=1.28.4 \
--control-plane-machine-count 3 \
--worker-machine-count 3 \
-v 6 > workload-01.yaml
Shell
복사
workload-01.yaml
kubectl apply -f workload-01.yaml
YAML
복사
ClusterResourceSet
이 전까지 진행 할 시에 클러스터 생성이 되지만 CNI 는 설치가 안되어 있음.
따라서, 클러스터 생성 이 후 설치하거나 ClusterResourceSet 으로 바인딩하여 설치해야한다.
•
구성 파일 가져오기
curl -L https://docs.projectcalico.org/manifests/calico.yaml -o calico.yaml
Bash
복사
•
해당 구성파일로 Configmap 만들기
kubectl create configmap calico-configmap \
--from-file=calico.yaml \
-o yaml \
--dry-run=client > calico-configmap.yaml
# 적용
kubectl apply -f calico-configmap.yaml
Bash
복사
•
ClusterResourceSet 적용
---
apiVersion: addons.cluster.x-k8s.io/v1alpha3
kind: ClusterResourceSet
metadata:
name: calico
spec:
clusterSelector:
matchLabels:
cni: calico ## 이 분으로 Binding 됨
resources:
- kind: ConfigMap
name: calico-configmap
# 적용
YAML
복사
•
Cluster에 적용
apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
name: cluster-50
labels:
cni: calico ## 라벨로 적용
YAML
복사
