1 v1.26 安装

安装视频参见：kubernetes 1.26版本上线，90分钟入门精讲在新窗口打开
20220106 未安装成功，改为 v1.26+containerd方式

环境配置

# vi /etc/hosts
10.0.0.216 k8s-m1
10.0.0.217 k8s-n1
10.0.0.218 k8s-n2

防火墙配置

所有主机均需要操作。

关闭现有防火墙firewalld
# systemctl disable firewalld
# systemctl stop firewalld
# firewall-cmd --state
not running

SELINUX配置

所有主机均需要操作。修改SELinux配置需要重启操作系统。

# sed -ri 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config

时间同步配置

所有主机均需要操作。最小化安装系统需要安装ntpdate软件。

systemctl start chronyd
systemctl enable chronyd

升级操作系统内核

所有主机均需要操作。

导入elrepo gpg key
# rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org

安装elrepo YUM源仓库
# yum -y install https://www.elrepo.org/elrepo-release-7.0-4.el7.elrepo.noarch.rpm

安装kernel-ml版本，ml为长期稳定版本，lt为长期维护版本
# yum --enablerepo="elrepo-kernel" -y install kernel-lt.x86_64

设置grub2默认引导为0
# grub2-set-default 0

重新生成grub2引导文件
# grub2-mkconfig -o /boot/grub2/grub.cfg

更新后，需要重启，使用升级的内核生效。
# reboot

重启后，需要验证内核是否为更新对应的版本
# uname -r

配置内核转发及网桥过滤

所有主机均需要操作。

添加网桥过滤及内核转发配置文件
# cat c
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
vm.swappiness = 0

加载br_netfilter模块
# modprobe br_netfilter

查看是否加载
# lsmod | grep br_netfilter
br_netfilter           22256  0
bridge                151336  1 br_netfilter

安装ipset及ipvsadm

所有主机均需要操作。

安装ipset及ipvsadm
# yum -y install ipset ipvsadm

配置ipvsadm模块加载方式
添加需要加载的模块
# cat > /etc/sysconfig/modules/ipvs.modules <<EOF
#!/bin/bash
modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
modprobe -- nf_conntrack
EOF

授权、运行、检查是否加载
# chmod 755 /etc/sysconfig/modules/ipvs.modules && bash /etc/sysconfig/modules/ipvs.modules && lsmod | grep -e ip_vs -e nf_conntrack

关闭SWAP分区

修改完成后需要重启操作系统，如不重启，可临时关闭，命令为swapoff -a

永远关闭swap分区，需要重启操作系统
# cat /etc/fstab
......
# /dev/mapper/centos-swap swap                    swap    defaults        0 0
在上一行中行首添加#
# 临时关闭
swapoff -a  # 禁用swap
free -h # 查看分区

1.1 DOCKER准备

1.1.1 docker安装源准备

wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -O /etc/yum.repos.d/docker-ce.repo

1.1.2 安装docker-ce

yum -y install docker-ce --allowerasing

1.1.3 启动docker

systemctl enable docker
systemctl start docker

1.1.4 修改cgroup方式

vi /etc/docker/daemon.json
{
 "registry-mirrors": ["https://c07oywfn.mirror.aliyuncs.com"],
 "exec-opts": ["native.cgroupdriver=systemd"]
}
# 重启docker
systemctl restart docker

1.1.5 cri-dockerd

clone cri-dockerd源码

git clone https://github.com/Mirantis/cri-dockerd.git

安装go环境

wget https://storage.googleapis.com/golang/getgo/installer_linux
chmod +x ./installer_linux
# 安装可能出现： Getting current Go version failed: Get https://golang.org/VERSION?m=text: dial tcp 142.251.43.17:443: i/o timeout
# 访问不了google网址
./installer_linux
source ~/.bash_profile

自行安装：https://golang.google.cn/doc/install

# 删除之前版本，解压最新的安装包（go1.19.4.linux-amd64.tar.gz）至 /usr/local目录下
rm -rf /usr/local/go && tar -C /usr/local -xzf go1.19.4.linux-amd64.tar.gz
# 配置环境变量
vi .bash_profile
export PATH=$PATH:/usr/local/go/bin
# 刷新环境变量
source $HOME/.bash_profile
# 查看版本
go version

# 配置 GOPROXY 环境变量, .bash_profile 或者 命令行
export GOPROXY=https://proxy.golang.com.cn,direct
# 还可以设置不走 proxy 的私有仓库或组，多个用逗号相隔（可选）
export GOPRIVATE=git.mycompany.com,github.com/my/private

cri-dockerd

cd cri-dockerd
mkdir bin
go build -o bin/cri-dockerd
mkdir -p /usr/local/bin
install -o root -g root -m 0755 bin/cri-dockerd /usr/local/bin/cri-dockerd
cp -a packaging/systemd/* /etc/systemd/system
sed -i -e 's,/usr/bin/cri-dockerd,/usr/local/bin/cri-dockerd,' /etc/systemd/system/cri-docker.service
systemctl daemon-reload
systemctl enable cri-docker.service
systemctl enable --now cri-docker.socket

1.2 K8s 1.26 集群部署

1.2.1 阿里云yum源

cat > /etc/yum.repos.d/kubernetes.repo  << EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

查看是否在最新版本k8s: yum list kubeadm.x86_64 --showduplicates | sort -f

1.2.2 安装

安装最新版本/指定版本二选一

# 安装最新版本
yum -y install kubelet kubeadm kubectl
# 安装指定版本
yum -y install kubelet-1.26.X kubeadm-1.26.X kubectl-1.26.X

1.2.3 配置kubelet

为了实现docker使用的cgroupdriver与kubelet使用的cgroup的一致性，建议修改如下文件内容。

# vim /etc/sysconfig/kubelet
KUBELET_EXTRA_ARGS="--cgroup-driver=systemd"

设置kubelet为开机自启动即可，由于没有生成配置文件，集群初始化后自动启动
# systemctl enable kubelet

1.2.4 集群初始化

离线环境可以提前准备镜像

# kubeadm config images list --kubernetes-version=v1.26.X

# cat image_download.sh
#!/bin/bash
images_list='
镜像列表'

for i in $images_list
do
        docker pull $i
done

docker save -o k8s-1-24-X.tar $images_list

直接初始化

kubeadm init --image-repository registry.aliyuncs.com/google_containers --kubernetes-version v1.26.0 --pod-network-cidr=10.224.0.0/16 --apiserver-advertise-address=10.0.0.216  --cri-socket unix:///var/run/cri-dockerd.sock

重置：

kubeadm reset --cri-socket unix:///var/run/cri-dockerd.sock

20230106 cri-dockerd 方式，k8s初始化失败

Unfortunately, an error has occurred:
	timed out waiting for the condition

This error is likely caused by:
	- The kubelet is not running
	- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)

If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
	- 'systemctl status kubelet'
	- 'journalctl -xeu kubelet'

Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
	- 'crictl --runtime-endpoint unix:///var/run/cri-dockerd.sock ps -a | grep kube | grep -v pause'
	Once you have found the failing container, you can inspect its logs with:
	- 'crictl --runtime-endpoint unix:///var/run/cri-dockerd.sock logs CONTAINERID'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher

systemctl status kubelet

[root@k8s-m1 cri-dockerd]# systemctl status kubelet
● kubelet.service - kubelet: The Kubernetes Node Agent
   Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled)
  Drop-In: /usr/lib/systemd/system/kubelet.service.d
           └─10-kubeadm.conf
   Active: active (running) since Fri 2023-01-06 12:47:51 CST; 6min ago
     Docs: https://kubernetes.io/docs/
 Main PID: 12073 (kubelet)
    Tasks: 12 (limit: 11379)
   Memory: 91.0M
   CGroup: /system.slice/kubelet.service
           └─12073 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --config=/var/lib/kubelet/config.yam>

1月 06 12:54:34 k8s-m1 kubelet[12073]: W0106 12:54:34.137616   12073 reflector.go:424] vendor/k8s.io/client-go/informers/factory.go:150: failed to list *v1.Node: Get "https://1>
1月 06 12:54:34 k8s-m1 kubelet[12073]: E0106 12:54:34.137764   12073 reflector.go:140] vendor/k8s.io/client-go/informers/factory.go:150: Failed to watch *v1.Node: failed to lis>
1月 06 12:54:36 k8s-m1 kubelet[12073]: E0106 12:54:36.164912   12073 remote_runtime.go:176] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = fa>
1月 06 12:54:36 k8s-m1 kubelet[12073]: E0106 12:54:36.165029   12073 kuberuntime_sandbox.go:72] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed >
1月 06 12:54:36 k8s-m1 kubelet[12073]: E0106 12:54:36.165122   12073 kuberuntime_manager.go:782] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed >
1月 06 12:54:36 k8s-m1 kubelet[12073]: E0106 12:54:36.165274   12073 pod_workers.go:965] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"etcd-k8s-m1_kub>
1月 06 12:54:37 k8s-m1 kubelet[12073]: E0106 12:54:37.116565   12073 controller.go:146] failed to ensure lease exists, will retry in 7s, error: Get "https://10.0.0.216:6443/api>
1月 06 12:54:38 k8s-m1 kubelet[12073]: I0106 12:54:38.313197   12073 kubelet_node_status.go:70] "Attempting to register node" node="k8s-m1"
1月 06 12:54:38 k8s-m1 kubelet[12073]: E0106 12:54:38.313949   12073 kubelet_node_status.go:92] "Unable to register node with API server" err="Post \"https://10.0.0.216:6443/ap>
1月 06 12:54:39 k8s-m1 kubelet[12073]: E0106 12:54:39.215592   12073 event.go:276] Unable to write event: '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1>
lin

20230207 cri-dockerd 方式,k8s安装成功

参见：14_Docker+k8s教程.md

cri-dockerd 改为二进制安装

# 1 v1.26 安装

# 环境配置

# 防火墙配置

# SELINUX配置

# 时间同步配置

# 升级操作系统内核

# 配置内核转发及网桥过滤

# 安装ipset及ipvsadm

# 关闭SWAP分区

# 1.1 DOCKER准备

# 1.1.1 docker安装源准备

# 1.1.2 安装docker-ce

# 1.1.3 启动docker

# 1.1.4 修改cgroup方式

# 1.1.5 cri-dockerd

# 1.2 K8s 1.26 集群部署

# 1.2.1 阿里云yum源

# 1.2.2 安装

# 1.2.3 配置kubelet

# 1.2.4 集群初始化

# 20230106 cri-dockerd 方式，k8s初始化失败

# 20230207 cri-dockerd 方式,k8s安装成功

1 v1.26 安装

环境配置

防火墙配置

SELINUX配置

时间同步配置

升级操作系统内核

配置内核转发及网桥过滤

安装ipset及ipvsadm

关闭SWAP分区

1.1 DOCKER准备

1.1.1 docker安装源准备

1.1.2 安装docker-ce

1.1.3 启动docker

1.1.4 修改cgroup方式

1.1.5 cri-dockerd

1.2 K8s 1.26 集群部署

1.2.1 阿里云yum源

1.2.2 安装

1.2.3 配置kubelet

1.2.4 集群初始化

20230106 cri-dockerd 方式，k8s初始化失败

20230207 cri-dockerd 方式,k8s安装成功