云野 k8s集群部署
Kubernetes部署
官网：https://kubernetes.io/docs/setup/
Docker配置修改(一定要注意json格式，网站有很多可以检测一下JSON在线 | JSON解析格式化—SO JSON在线工具)
[root@k8s-master ~]# cat /etc/docker/daemon.json
{
"registry-mirrors": [
"https://xx4bwyg2.mirror.aliyuncs.com",
"http://f1361db2.m.daocloud.io",
"https://registry.docker-cn.com",
"http://hub-mirror.c.163.com",
"https://docker.mirrors.ustc.edu.cn"
],
"exec-opts": ["native.cgroupdriver=systemd"],
"log-driver": "json-file",
"log-opts": {
"max-size": "100m"
}
}
[root@k8s-master ~]# mkdir -p /etc/systemd/system/docker.service.d
[root@k8s-master ~]# systemctl daemon-reload
[root@k8s-master ~]# systemctl restart docker
[root@k8s-master ~]# scp /etc/docker/daemon.json 10.220.180.130:/etc/docker/
root@10.220.180.130's password:
daemon.json 100% 317 300.3KB/s 00:00
[root@k8s-master ~]# scp /etc/docker/daemon.json 10.220.180.131:/etc/docker/
root@10.220.180.131's password:
daemon.json 100% 317 330.2KB/s 00:00
[root@k8s-node1 ~]# mkdir -p /etc/systemd/system/docker.service.d
[root@k8s-node1 ~]# systemctl daemon-reload
[root@k8s-node1 ~]# systemctl restart docker
[root@k8s-node2 ~]# mkdir -p /etc/systemd/system/docker.service.d
[root@k8s-node2 ~]# systemctl daemon-reload
[root@k8s-node2 ~]# systemctl restart docker
添加k8s软件源
[root@k8s-master ~]# cat /etc/yum.repos.d/kubernetes.repo [kubernetes] name=Kubernetes baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64 enabled=1 gpgcheck=0 repo_gpgcheck=0 gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg [root@k8s-master ~]# yum repolist Loaded plugins: fastestmirror, langpacks Loading mirror speeds from cached hostfile * base: mirrors.ustc.edu.cn * extras: mirrors.ustc.edu.cn * updates: mirrors.ustc.edu.cn repo id repo name status base/7/x86_64 CentOS-7 - Base 10,072 docker-ce-stable/x86_64 Docker CE Stable - x86_64 290 extras/7/x86_64 CentOS-7 - Extras 518 kubernetes Kubernetes 1,022 mysql-innovation-community/x86_64 MySQL Innovation Release Community Server 33 mysql80-community/x86_64 MySQL 8.0 Community Server 465 updates/7/x86_64 CentOS-7 - Updates 5,704 repolist: 18,104 [root@k8s-master ~]# scp /etc/yum.repos.d/kubernetes.repo 10.220.180.130:/etc/yum.repos.d/ root@10.220.180.130's password: kubernetes.repo 100% 280 186.7KB/s 00:00 [root@k8s-master ~]# scp /etc/yum.repos.d/kubernetes.repo 10.220.180.131:/etc/yum.repos.d/ root@10.220.180.131's password: kubernetes.repo 100% 280 222.3KB/s 00:00
所有节点安装k8s
[root@k8s-master ~]# yum install -y kubelet kubeadm kubectl [root@k8s-master ~]# systemctl enable --now kubelet [root@k8s-node1 ~]# yum install -y kubelet kubeadm kubectl [root@k8s-node1 ~]# systemctl enable --now kubelet [root@k8s-node2 ~]# yum install -y kubelet kubeadm kubectl [root@k8s-node2 ~]# systemctl enable --now kubelet
查看默认配置信息
[root@k8s-master ~]# kubeadm config print init-defaults
apiVersion: kubeadm.k8s.io/v1beta3
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
token: abcdef.0123456789abcdef
ttl: 24h0m0s
usages:
- signing
- authentication
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 1.2.3.4
bindPort: 6443
nodeRegistration:
criSocket: unix:///var/run/containerd/containerd.sock
imagePullPolicy: IfNotPresent
name: node
taints: null
---
apiServer:
timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta3
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns: {}
etcd:
local:
dataDir: /var/lib/etcd
imageRepository: registry.k8s.io
kind: ClusterConfiguration
kubernetesVersion: 1.28.0
networking:
dnsDomain: cluster.local
serviceSubnet: 10.96.0.0/12
scheduler: {}
默认从k8s.gcr.io上下载组件镜像，需要翻墙才可以，所以需要修改镜像仓库(这步操作只在master节点执行)：
[root@k8s-master ~]# kubeadm config images list --image-repository registry.aliyuncs.com/google_containers //列出所需镜像 I0202 11:50:23.099141 15226 version.go:256] remote version is much newer: v1.29.1; falling back to: stable-1.28 registry.aliyuncs.com/google_containers/kube-apiserver:v1.28.6 registry.aliyuncs.com/google_containers/kube-controller-manager:v1.28.6 registry.aliyuncs.com/google_containers/kube-scheduler:v1.28.6 registry.aliyuncs.com/google_containers/kube-proxy:v1.28.6 registry.aliyuncs.com/google_containers/pause:3.9 registry.aliyuncs.com/google_containers/etcd:3.5.9-0 registry.aliyuncs.com/google_containers/coredns:v1.10.1 [root@k8s-master ~]# kubeadm config images pull --image-repository registry.aliyuncs.com/google_containers //拉取镜像 I0202 15:10:43.433299 25267 version.go:256] remote version is much newer: v1.29.1; falling back to: stable-1.28 [config/images] Pulled registry.aliyuncs.com/google_containers/kube-apiserver:v1.28.6 [config/images] Pulled registry.aliyuncs.com/google_containers/kube-controller-manager:v1.28.6 [config/images] Pulled registry.aliyuncs.com/google_containers/kube-scheduler:v1.28.6 [config/images] Pulled registry.aliyuncs.com/google_containers/kube-proxy:v1.28.6 [config/images] Pulled registry.aliyuncs.com/google_containers/pause:3.9 [config/images] Pulled registry.aliyuncs.com/google_containers/etcd:3.5.9-0 [config/images] Pulled registry.aliyuncs.com/google_containers/coredns:v1.10.1 [root@k8s-master ~]# kubeadm init --pod-network-cidr=10.220.180.0/16 --image-repository registry.aliyuncs.com/google_containers //初始化集群 ...... [kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key [addons] Applied essential addon: CoreDNS [addons] Applied essential addon: kube-proxy Your Kubernetes control-plane has initialized successfully! To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config Alternatively, if you are the root user, you can run: export KUBECONFIG=/etc/kubernetes/admin.conf You should now deploy a pod network to the cluster. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/ Then you can join any number of worker nodes by running the following on each as root: kubeadm join 10.220.180.120:6443 --token 7m1j99.44fjdyw4u7jadxo7 \ --discovery-token-ca-cert-hash sha256:ace63743c8f6da2784f4646f3aacd12c735c8cf1066042d48fa8a11ea470d35c
配置kubectl(所有节点)
[root@k8s-master ~]# useradd kubeadm [root@k8s-master ~]# vim /etc/sudoers ...... 98 ## 99 ## Allow root to run any commands anywhere 100 root ALL=(ALL) ALL 101 kubeadm ALL=(ALL) NOPASSWD: ALL 102 ## Allows members of the 'sys' group to run networking, software, 103 ## service management apps and more. ...... [root@k8s-master ~]# scp /etc/kubernetes/admin.conf 10.220.180.131:/etc/kubernetes/ root@10.220.180.131's password: admin.conf 100% 5650 4.0MB/s 00:00 [root@k8s-master ~]# scp /etc/kubernetes/admin.conf 10.220.180.130:/etc/kubernetes/ root@10.220.180.130's password: admin.conf 100% 5650 2.6MB/s 00:00 [root@k8s-master ~]# mkdir -p $HOME/.kube [root@k8s-master ~]# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config [root@k8s-master ~]# sudo chown $(id -u):$(id -g) $HOME/.kube/config
配置kubectl命令补齐功能（所有节点）
[root@k8s-master ~]# echo "source <(kubectl completion bash)" >> ~/.bashrc [root@k8s-master ~]# source ~/.bashrc
节点扩容（node节点执行）
[root@k8s-node1 ~]# kubeadm join 10.220.180.120:6443 --token kci9oa.xdhbo6sjbuith7r3 --discovery-token-ca-cert-hash sha256:dd9efcbef65f50ed8aaa3fd62af9abc5223a6a6035efdf506f0a513353b9aa02
[preflight] Running pre-flight checks [preflight] Reading configuration from the cluster... [preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml' [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml" [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env" [kubelet-start] Starting the kubelet [kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap... This node has joined the cluster: * Certificate signing request was sent to apiserver and a response was received. * The Kubelet was informed of the new secure connection details. Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
[root@k8s-node2 ~]# kubeadm join 10.220.180.120:6443 --token kci9oa.xdhbo6sjbuith7r3 --discovery-token-ca-cert-hash sha256:dd9efcbef65f50ed8aaa3fd62af9abc5223a6a6035efdf506f0a513353b9aa02
[preflight] Running pre-flight checks [preflight] Reading configuration from the cluster... [preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml' [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml" [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env" [kubelet-start] Starting the kubelet [kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap... This node has joined the cluster: * Certificate signing request was sent to apiserver and a response was received. * The Kubelet was informed of the new secure connection details. Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
安装flannel网络组件：https://github.com/coreos/flannel
[root@k8s-master ~]# wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
[root@k8s-master ~]# vim kube-flannel.ym
......
98 net-conf.json: |
99 {
100 "Network": "10.220.0.0/16",
101 "Backend": {
102 "Type": "vxlan"
103 }
104 }
...... [root@k8s-master ~]# kubectl apply kube-flannel.yml
其他网络组件：
Master查看状态:
[root@k8s-master ~]# kubectl get cs Warning: v1 ComponentStatus is deprecated in v1.19+ NAME STATUS MESSAGE ERROR etcd-0 Healthy ok scheduler Healthy ok controller-manager Healthy ok [root@k8s-master ~]# kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-master Ready control-plane 5h48m v1.28.2 k8s-node1 Ready <none> 5h33m v1.28.2 k8s-node2 Ready <none> 5h33m v1.28.2 [root@k8s-master ~]# kubectl get pods -o wide -A NAMESPACE NAME R EADY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES kube-flannel kube-flannel-ds-5r9dj 1/1 Running 0 7s 10.220.180.120 k8s-master <none> <none> kube-flannel kube-flannel-ds-cqt49 1/1 Running 0 7s 10.220.180.130 k8s-node1 <none> <none> kube-flannel kube-flannel-ds-p7b4n 1/1 Running 0 7s 10.220.180.131 k8s-node2 <none> <none> kube-system coredns-66f779496c-b4nqp 1/1 Running 0 43h 10.244.0.3 k8s-master <none> <none> kube-system coredns-66f779496c-bx6sk 1/1 Running 0 43h 10.244.0.2 k8s-master <none> <none> kube-system etcd-k8s-master 1/1 Running 0 43h 10.220.180.120 k8s-master <none> <none> kube-system kube-apiserver-k8s-master 1/1 Running 0 43h 10.220.180.120 k8s-master <none> <none> kube-system kube-controller-manager-k8s-master 1/1 Running 1 43h 10.220.180.120 k8s-master <none> <none> kube-system kube-proxy-btqzp 1/1 Running 0 42h 10.220.180.130 k8s-node1 <none> <none> kube-system kube-proxy-hb6n8 1/1 Running 0 43h 10.220.180.120 k8s-master <none> <none> kube-system kube-proxy-nlxlh 1/1 Running 0 42h 10.220.180.131 k8s-node2 <none> <none> kube-system kube-scheduler-k8s-master 1/1 Running 0 43h 10.220.180.120 k8s-master <none> <none>
kubectl命令指南：
