二进制安装Kubernetes（k8s） v1.23.5 ---（下）

接上文

Github：https://github.com/cby-chen/Kubernetes/

1.23.3 和 1.23.4 和 1.23.5 文档以及安装包已生成。

后续尽可能第一时间更新新版本文档

https://github.com/cby-chen/Kubernetes/releases

6.k8s组件配置（区别于第4点）

所有k8s节点创建以下目录

代码语言：shell

复制

mkdir -p /etc/kubernetes/manifests/ /etc/systemd/system/kubelet.service.d /var/lib/kubelet /var/log/kubernetes

6.1.创建apiserver（所有master节点）

6.1.1master01节点配置

代码语言：shell

复制

cat > /usr/lib/systemd/system/kube-apiserver.service << EOF [Unit] Description=Kubernetes API Server Documentation=https://github.com/kubernetes/kubernetes After=network.target [Service] ExecStart=/usr/local/bin/kube-apiserver --v=2 --logtostderr=true --allow-privileged=true --bind-address=0.0.0.0 --secure-port=6443 --insecure-port=0 --advertise-address=192.168.1.61 --service-cluster-ip-range=10.96.0.0/12 --service-node-port-range=30000-32767 --etcd-servers=https://192.168.1.61//192.168.1.62:2379,https://192.168.1.63:2379 --etcd-cafile=/etc/etcd/ssl/etcd-ca.pem --etcd-certfile=/etc/etcd/ssl/etcd.pem --etcd-keyfile=/etc/etcd/ssl/etcd-key.pem --client-ca-file=/etc/kubernetes/pki/ca.pem --tls-cert-file=/etc/kubernetes/pki/apiserver.pem --tls-private-key-file=/etc/kubernetes/pki/apiserver-key.pem --kubelet-client-certificate=/etc/kubernetes/pki/apiserver.pem --kubelet-client-key=/etc/kubernetes/pki/apiserver-key.pem --service-account-key-file=/etc/kubernetes/pki/sa.pub --service-account-signing-key-file=/etc/kubernetes/pki/sa.key --service-account-issuer=https://kubernetes.default.svc.cluster.local --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,ResourceQuota --authorization-mode=Node,RBAC --enable-bootstrap-token-auth=true --requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.pem --proxy-client-cert-file=/etc/kubernetes/pki/front-proxy-client.pem --proxy-client-key-file=/etc/kubernetes/pki/front-proxy-client-key.pem --requestheader-allowed-names=aggregator --requestheader-group-headers=X-Remote-Group --requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-username-headers=X-Remote-User --enable-aggregator-routing=true # --token-auth-file=/etc/kubernetes/token.csv Restart=on-failure RestartSec=10s LimitNOFILE=65535 [Install] WantedBy=multi-user.target

EOF

6.1.2master02节点配置

代码语言：shell

复制

cat > /usr/lib/systemd/system/kube-apiserver.service << EOF [Unit] Description=Kubernetes API Server Documentation=https://github.com/kubernetes/kubernetes After=network.target [Service] ExecStart=/usr/local/bin/kube-apiserver --v=2 --logtostderr=true --allow-privileged=true --bind-address=0.0.0.0 --secure-port=6443 --insecure-port=0 --advertise-address=192.168.1.62 --service-cluster-ip-range=10.96.0.0/12 --service-node-port-range=30000-32767 --etcd-servers=https://192.168.1.61//192.168.1.62:2379,https://192.168.1.63:2379 --etcd-cafile=/etc/etcd/ssl/etcd-ca.pem --etcd-certfile=/etc/etcd/ssl/etcd.pem --etcd-keyfile=/etc/etcd/ssl/etcd-key.pem --client-ca-file=/etc/kubernetes/pki/ca.pem --tls-cert-file=/etc/kubernetes/pki/apiserver.pem --tls-private-key-file=/etc/kubernetes/pki/apiserver-key.pem --kubelet-client-certificate=/etc/kubernetes/pki/apiserver.pem --kubelet-client-key=/etc/kubernetes/pki/apiserver-key.pem --service-account-key-file=/etc/kubernetes/pki/sa.pub --service-account-signing-key-file=/etc/kubernetes/pki/sa.key --service-account-issuer=https://kubernetes.default.svc.cluster.local --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,ResourceQuota --authorization-mode=Node,RBAC --enable-bootstrap-token-auth=true --requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.pem --proxy-client-cert-file=/etc/kubernetes/pki/front-proxy-client.pem --proxy-client-key-file=/etc/kubernetes/pki/front-proxy-client-key.pem --requestheader-allowed-names=aggregator --requestheader-group-headers=X-Remote-Group --requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-username-headers=X-Remote-User --enable-aggregator-routing=true # --token-auth-file=/etc/kubernetes/token.csv Restart=on-failure RestartSec=10s LimitNOFILE=65535 [Install] WantedBy=multi-user.target

EOF

6.1.3master03节点配置

代码语言：shell

复制

cat > /usr/lib/systemd/system/kube-apiserver.service << EOF [Unit] Description=Kubernetes API Server Documentation=https://github.com/kubernetes/kubernetes After=network.target [Service] ExecStart=/usr/local/bin/kube-apiserver --v=2 --logtostderr=true --allow-privileged=true --bind-address=0.0.0.0 --secure-port=6443 --insecure-port=0 --advertise-address=192.168.1.63 --service-cluster-ip-range=10.96.0.0/12 --service-node-port-range=30000-32767 --etcd-servers=https://192.168.1.61//192.168.1.62:2379,https://192.168.1.63:2379 --etcd-cafile=/etc/etcd/ssl/etcd-ca.pem --etcd-certfile=/etc/etcd/ssl/etcd.pem --etcd-keyfile=/etc/etcd/ssl/etcd-key.pem --client-ca-file=/etc/kubernetes/pki/ca.pem --tls-cert-file=/etc/kubernetes/pki/apiserver.pem --tls-private-key-file=/etc/kubernetes/pki/apiserver-key.pem --kubelet-client-certificate=/etc/kubernetes/pki/apiserver.pem --kubelet-client-key=/etc/kubernetes/pki/apiserver-key.pem --service-account-key-file=/etc/kubernetes/pki/sa.pub --service-account-signing-key-file=/etc/kubernetes/pki/sa.key --service-account-issuer=https://kubernetes.default.svc.cluster.local --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,ResourceQuota --authorization-mode=Node,RBAC --enable-bootstrap-token-auth=true --requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.pem --proxy-client-cert-file=/etc/kubernetes/pki/front-proxy-client.pem --proxy-client-key-file=/etc/kubernetes/pki/front-proxy-client-key.pem --requestheader-allowed-names=aggregator --requestheader-group-headers=X-Remote-Group --requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-username-headers=X-Remote-User --enable-aggregator-routing=true # --token-auth-file=/etc/kubernetes/token.csv Restart=on-failure RestartSec=10s LimitNOFILE=65535 [Install] WantedBy=multi-user.target

EOF

6.1.4启动apiserver（所有master节点）

代码语言：shell

复制

systemctl daemon-reload && systemctl enable --now kube-apiserver 注意查看状态是否启动正常

systemctl status kube-apiserver

6.2.配置kube-controller-manager service

代码语言：shell

复制

所有master节点配置，且配置相同

172.16.0.0/12为pod网段，按需求设置你自己的网段
cat > /usr/lib/systemd/system/kube-controller-manager.service << EOF
[Unit]

Description=Kubernetes Controller Manager

Documentation=https://github.com/kubernetes/kubernetes

After=network.target
[Service]

ExecStart=/usr/local/bin/kube-controller-manager 

--v=2 

--logtostderr=true 

--address=127.0.0.1 

--root-ca-file=/etc/kubernetes/pki/ca.pem 

--cluster-signing-cert-file=/etc/kubernetes/pki/ca.pem 

--cluster-signing-key-file=/etc/kubernetes/pki/ca-key.pem 

--service-account-private-key-file=/etc/kubernetes/pki/sa.key 

--kubeconfig=/etc/kubernetes/controller-manager.kubeconfig 

--leader-elect=true 

--use-service-account-credentials=true 

--node-monitor-grace-period=40s 

--node-monitor-period=5s 

--pod-eviction-timeout=2m0s 

--controllers=*,bootstrapsigner,tokencleaner 

--allocate-node-cidrs=true 

--cluster-cidr=172.16.0.0/12 

--requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.pem 

--node-cidr-mask-size=24
Restart=always

RestartSec=10s
[Install]

WantedBy=multi-user.target
EOF

6.2.1启动kube-controller-manager，并查看状态

代码语言：shell

复制

systemctl daemon-reload

systemctl enable --now kube-controller-manager

systemctl  status kube-controller-manager

6.3.配置kube-scheduler service

6.3.1所有master节点配置，且配置相同

代码语言：shell

复制

cat > /usr/lib/systemd/system/kube-scheduler.service << EOF
[Unit]

Description=Kubernetes Scheduler

Documentation=https://github.com/kubernetes/kubernetes

After=network.target
[Service]

ExecStart=/usr/local/bin/kube-scheduler 

--v=2 

--logtostderr=true 

--address=127.0.0.1 

--leader-elect=true 

--kubeconfig=/etc/kubernetes/scheduler.kubeconfig
Restart=always

RestartSec=10s
[Install]

WantedBy=multi-user.target
EOF

6.3.2启动并查看服务状态

代码语言：shell

复制

systemctl daemon-reload

systemctl enable --now kube-scheduler

systemctl status kube-scheduler

7.TLS Bootstrapping配置

7.1在master01上配置

代码语言：shell

复制

cd /root/Kubernetes/bootstrap kubectl config set-cluster kubernetes --certificate-authority=/etc/kubernetes/pki/ca.pem --embed-certs=true --server=https://192.168.1.59 --kubeconfig=/etc/kubernetes/bootstrap-kubelet.kubeconfig kubectl config set-credentials tls-bootstrap-token-user --token=c8ad9c.2e4d610cf3e7426e --kubeconfig=/etc/kubernetes/bootstrap-kubelet.kubeconfig kubectl config set-context tls-bootstrap-token-user@kubernetes --cluster=kubernetes --user=tls-bootstrap-token-user --kubeconfig=/etc/kubernetes/bootstrap-kubelet.kubeconfig kubectl config use-context tls-bootstrap-token-user@kubernetes --kubeconfig=/etc/kubernetes/bootstrap-kubelet.kubeconfig token的位置在bootstrap.secret.yaml，如果修改的话到这个文件修改

mkdir -p /root/.kube ; cp /etc/kubernetes/admin.kubeconfig /root/.kube/config

7.2查看集群状态，没问题的话继续后续操作

代码语言：shell

复制

kubectl get cs Warning: v1 ComponentStatus is deprecated in v1.19+ NAME STATUS MESSAGE ERROR controller-manager Healthy ok etcd-0 Healthy {"health":"true","reason":""} scheduler Healthy ok etcd-1 Healthy {"health":"true","reason":""} etcd-2 Healthy {"health":"true","reason":""}

kubectl create -f bootstrap.secret.yaml

8.node节点配置

8.1.在master01上将证书复制到node节点

代码语言：shell

复制

cd /etc/kubernetes/

for NODE in k8s-master02 k8s-master03 k8s-node01 k8s-node02 k8s-node03 k8s-node04 k8s-node05; do ssh $NODE mkdir -p /etc/kubernetes/pki for FILE in pki/ca.pem pki/ca-key.pem pki/front-proxy-ca.pem bootstrap-kubelet.kubeconfig; do scp /etc/kubernetes/FILE NODE:/etc/kubernetes/${FILE} done done

8.2.kubelet配置

8.2.1所有k8s节点创建相关目录

代码语言：shell

复制

mkdir -p /var/lib/kubelet /var/log/kubernetes /etc/systemd/system/kubelet.service.d /etc/kubernetes/manifests/
所有k8s节点配置kubelet service

cat > /usr/lib/systemd/system/kubelet.service << EOF
[Unit]

Description=Kubernetes Kubelet

Documentation=https://github.com/kubernetes/kubernetes

After=docker.service

Requires=docker.service
[Service]

ExecStart=/usr/local/bin/kubelet 

--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.kubeconfig  

--kubeconfig=/etc/kubernetes/kubelet.kubeconfig 

--config=/etc/kubernetes/kubelet-conf.yml 

--network-plugin=cni  

--cni-conf-dir=/etc/cni/net.d  

--cni-bin-dir=/opt/cni/bin  

--container-runtime=remote  

--runtime-request-timeout=15m  

--container-runtime-endpoint=unix:///run/containerd/containerd.sock  

--cgroup-driver=systemd 

--node-labels=node.kubernetes.io/node=''
Restart=always

StartLimitInterval=0

RestartSec=10
[Install]

WantedBy=multi-user.target

EOF

8.2.2所有k8s节点创建kubelet的配置文件

代码语言：shell

复制

cat > /etc/kubernetes/kubelet-conf.yml <<EOF

apiVersion: kubelet.config.k8s.io/v1beta1

kind: KubeletConfiguration

address: 0.0.0.0

port: 10250

readOnlyPort: 10255

authentication:

anonymous:

enabled: false

webhook:

cacheTTL: 2m0s

enabled: true

x509:

clientCAFile: /etc/kubernetes/pki/ca.pem

authorization:

mode: Webhook

webhook:

cacheAuthorizedTTL: 5m0s

cacheUnauthorizedTTL: 30s

cgroupDriver: systemd

cgroupsPerQOS: true

clusterDNS:

10.96.0.10

clusterDomain: cluster.local

containerLogMaxFiles: 5

containerLogMaxSize: 10Mi

contentType: application/vnd.kubernetes.protobuf

cpuCFSQuota: true

cpuManagerPolicy: none

cpuManagerReconcilePeriod: 10s

enableControllerAttachDetach: true

enableDebuggingHandlers: true

enforceNodeAllocatable:
pods

eventBurst: 10

eventRecordQPS: 5

evictionHard:

imagefs.available: 15%

memory.available: 100Mi

nodefs.available: 10%

nodefs.inodesFree: 5%

evictionPressureTransitionPeriod: 5m0s

failSwapOn: true

fileCheckFrequency: 20s

hairpinMode: promiscuous-bridge

healthzBindAddress: 127.0.0.1

healthzPort: 10248

httpCheckFrequency: 20s

imageGCHighThresholdPercent: 85

imageGCLowThresholdPercent: 80

imageMinimumGCAge: 2m0s

iptablesDropBit: 15

iptablesMasqueradeBit: 14

kubeAPIBurst: 10

kubeAPIQPS: 5

makeIPTablesUtilChains: true

maxOpenFiles: 1000000

maxPods: 110

nodeStatusUpdateFrequency: 10s

oomScoreAdj: -999

podPidsLimit: -1

registryBurst: 10

registryPullQPS: 5

resolvConf: /etc/resolv.conf

rotateCertificates: true

runtimeRequestTimeout: 2m0s

serializeImagePulls: true

staticPodPath: /etc/kubernetes/manifests

streamingConnectionIdleTimeout: 4h0m0s

syncFrequency: 1m0s

volumeStatsAggPeriod: 1m0s

EOF

8.2.3启动kubelet

代码语言：shell

复制

systemctl daemon-reload

systemctl restart kubelet

systemctl enable --now kubelet

8.2.4查看集群

代码语言：shell

复制

[root@k8s-master01 ~]# kubectl  get node

NAME           STATUS     ROLES    AGE   VERSION

k8s-master01   NotReady   <none>   14h   v1.23.5

k8s-master02   NotReady   <none>   14h   v1.23.5

k8s-master03   NotReady   <none>   14h   v1.23.5

k8s-node01     NotReady   <none>   14h   v1.23.5

k8s-node02     NotReady   <none>   14h   v1.23.5

k8s-node03     NotReady   <none>   14h   v1.23.5

k8s-node04     NotReady   <none>   14h   v1.23.5

k8s-node05     NotReady   <none>   14h   v1.23.5

[root@k8s-master01 ~]#

8.3.kube-proxy配置

8.3.1此配置只在master01操作

代码语言：shell

复制

cd /root/Kubernetes/

kubectl -n kube-system create serviceaccount kube-proxy

kubectl create clusterrolebinding system:kube-proxy         --clusterrole system:node-proxier         --serviceaccount kube-system:kube-proxy
SECRET=$(kubectl -n kube-system get sa/kube-proxy 

--output=jsonpath='{.secrets[0].name}')
JWT_TOKEN=(kubectl -n kube-system get secret/SECRET 

--output=jsonpath='{.data.token}' | base64 -d)
PKI_DIR=/etc/kubernetes/pki

K8S_DIR=/etc/kubernetes
kubectl config set-cluster kubernetes     --certificate-authority=/etc/kubernetes/pki/ca.pem     --embed-certs=true     --server=https://192.168.1.59     --kubeconfig=${K8S_DIR}/kube-proxy.kubeconfig
kubectl config set-credentials kubernetes     --token=${JWT_TOKEN}     --kubeconfig=/etc/kubernetes/kube-proxy.kubeconfig
kubectl config set-context kubernetes     --cluster=kubernetes     --user=kubernetes     --kubeconfig=/etc/kubernetes/kube-proxy.kubeconfig
kubectl config use-context kubernetes     --kubeconfig=/etc/kubernetes/kube-proxy.kubeconfig

8.3.2将kubeconfig发送至其他节点

代码语言：shell

复制

for NODE in k8s-master02 k8s-master03; do scp /etc/kubernetes/kube-proxy.kubeconfig $NODE:/etc/kubernetes/kube-proxy.kubeconfig done

for NODE in k8s-node01 k8s-node02 k8s-node03 k8s-node04 k8s-node05; do scp /etc/kubernetes/kube-proxy.kubeconfig $NODE:/etc/kubernetes/kube-proxy.kubeconfig done

8.3.3所有k8s节点添加kube-proxy的配置和service文件

代码语言：shell

复制

cat >  /usr/lib/systemd/system/kube-proxy.service << EOF

[Unit]

Description=Kubernetes Kube Proxy

Documentation=https://github.com/kubernetes/kubernetes

After=network.target
[Service]

ExecStart=/usr/local/bin/kube-proxy 

--config=/etc/kubernetes/kube-proxy.yaml 

--v=2
Restart=always

RestartSec=10s
[Install]

WantedBy=multi-user.target
EOF

代码语言：shell

复制

cat > /etc/kubernetes/kube-proxy.yaml << EOF apiVersion: kubeproxy.config.k8s.io/v1alpha1 bindAddress: 0.0.0.0 clientConnection: acceptContentTypes: "" burst: 10 contentType: application/vnd.kubernetes.protobuf kubeconfig: /etc/kubernetes/kube-proxy.kubeconfig qps: 5 clusterCIDR: 172.16.0.0/12 configSyncPeriod: 15m0s conntrack: max: null maxPerCore: 32768 min: 131072 tcpCloseWaitTimeout: 1h0m0s tcpEstablishedTimeout: 24h0m0s enableProfiling: false healthzBindAddress: 0.0.0.0:10256 hostnameOverride: "" iptables: masqueradeAll: false masqueradeBit: 14 minSyncPeriod: 0s syncPeriod: 30s ipvs: masqueradeAll: true minSyncPeriod: 5s scheduler: "rr" syncPeriod: 30s kind: KubeProxyConfiguration metricsBindAddress: 127.0.0.1:10249 mode: "ipvs" nodePortAddresses: null oomScoreAdj: -999 portRange: "" udpIdleTimeout: 250ms

EOF

8.3.4启动kube-proxy

代码语言：shell

复制

 systemctl daemon-reload

systemctl enable --now kube-proxy

9.安装Calico

9.1以下步骤只在master01操作

9.1.1更改calico网段

代码语言：shell

复制

cd /root/Kubernetes/calico/ sed -i "s#POD_CIDR#172.16.0.0/12#g" calico.yaml grep "IPV4POOL_CIDR" calico.yaml -A 1 - name: CALICO_IPV4POOL_CIDR value: "172.16.0.0/12" 创建

kubectl apply -f calico.yaml

9.1.2查看容器状态

代码语言：shell

复制

[root@k8s-master01 ~]# kubectl get pod -A NAMESPACE NAME READY STATUS RESTARTS AGE kube-system calico-kube-controllers-6f6595874c-nb95g 1/1 Running 0 2m54s kube-system calico-node-67dn4 1/1 Running 0 2m54s kube-system calico-node-79zxj 1/1 Running 0 2m54s kube-system calico-node-85bsf 1/1 Running 0 2m54s kube-system calico-node-8trsm 1/1 Running 0 2m54s kube-system calico-node-dvz72 1/1 Running 0 2m54s kube-system calico-node-qqzwx 1/1 Running 0 2m54s kube-system calico-node-rngzq 1/1 Running 0 2m55s kube-system calico-node-w8gqp 1/1 Running 0 2m54s kube-system calico-typha-6b6cf8cbdf-2b454 1/1 Running 0 2m55s [root@k8s-master01 ~]#

[root@k8s-master01 ~]# kubectl get node NAME STATUS ROLES AGE VERSION k8s-master01 Ready <none> 14h v1.23.5 k8s-master02 Ready <none> 14h v1.23.5 k8s-master03 Ready <none> 14h v1.23.5 k8s-node01 Ready <none> 14h v1.23.5 k8s-node02 Ready <none> 14h v1.23.5 k8s-node03 Ready <none> 14h v1.23.5 k8s-node04 Ready <none> 14h v1.23.5 k8s-node05 Ready <none> 14h v1.23.5 [root@k8s-master01 ~]#

10.安装CoreDNS

10.1以下步骤只在master01操作

10.1.1修改文件

代码语言：shell

复制

cd /root/Kubernetes/CoreDNS/ sed -i "s#KUBEDNS_SERVICE_IP#10.96.0.10#g" coredns.yaml

cat coredns.yaml | grep clusterIP: clusterIP: 10.96.0.10

10.1.2安装

代码语言：shell

复制

kubectl  create -f coredns.yaml

serviceaccount/coredns created

clusterrole.rbac.authorization.k8s.io/system:coredns created

clusterrolebinding.rbac.authorization.k8s.io/system:coredns created

configmap/coredns created

deployment.apps/coredns created

service/kube-dns created

11.安装Metrics Server

11.1以下步骤只在master01操作

11.1.1安装Metrics-server

在新版的Kubernetes中系统资源的采集均使用Metrics-server，可以通过Metrics采集节点和Pod的内存、磁盘、CPU和网络的使用率

代码语言：shell

复制

安装metrics server cd /root/Kubernetes/metrics-server/ kubectl create -f .

serviceaccount/metrics-server created clusterrole.rbac.authorization.k8s.io/system:aggregated-metrics-reader created clusterrole.rbac.authorization.k8s.io/system:metrics-server created rolebinding.rbac.authorization.k8s.io/metrics-server-auth-reader created clusterrolebinding.rbac.authorization.k8s.io/metrics-server:system:auth-delegator created clusterrolebinding.rbac.authorization.k8s.io/system:metrics-server created service/metrics-server created deployment.apps/metrics-server created apiservice.apiregistration.k8s.io/v1beta1.metrics.k8s.io created

11.1.2稍等片刻查看状态

代码语言：shell

复制

kubectl  top node

NAME           CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%

k8s-master01   154m         1%     1715Mi          21%

k8s-master02   151m         1%     1274Mi          16%

k8s-master03   523m         6%     1345Mi          17%

k8s-node01     84m          1%     671Mi           8%

k8s-node02     73m          0%     727Mi           9%

k8s-node03     96m          1%     769Mi           9%

k8s-node04     68m          0%     673Mi           8%

k8s-node05     82m          1%     679Mi           8%

12.集群验证

12.1部署pod资源

代码语言：shell

复制

cat<<EOF | kubectl apply -f - apiVersion: v1 kind: Pod metadata: name: busybox namespace: default spec: containers: name: busybox image: busybox:1.28 command: sleep "3600" imagePullPolicy: IfNotPresent restartPolicy: Always EOF 查看

kubectl get pod NAME READY STATUS RESTARTS AGE busybox 1/1 Running 0 17s

12.2用pod解析默认命名空间中的kubernetes

代码语言：shell

复制

kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 17h kubectl exec busybox -n default -- nslookup kubernetes 3Server: 10.96.0.10 Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local

Name: kubernetes Address 1: 10.96.0.1 kubernetes.default.svc.cluster.local

12.3测试跨命名空间是否可以解析

代码语言：shell

复制

kubectl exec busybox -n default -- nslookup kube-dns.kube-system Server: 10.96.0.10 Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local

Name: kube-dns.kube-system Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local

12.4每个节点都必须要能访问Kubernetes的kubernetes svc 443和kube-dns的service 53

代码语言：shell

复制

telnet 10.96.0.1 443 Trying 10.96.0.1... Connected to 10.96.0.1. Escape character is '^]'. telnet 10.96.0.10 53 Trying 10.96.0.10... Connected to 10.96.0.10. Escape character is '^]'.

curl 10.96.0.10:53 curl: (52) Empty reply from server

12.5Pod和Pod之前要能通

代码语言：shell

复制

kubectl get po -owide

NAME      READY   STATUS    RESTARTS   AGE   IP              NODE         NOMINATED NODE   READINESS GATES

busybox   1/1     Running   0          17m   172.27.14.193   k8s-node02   <none>           <none>
kubectl get po -n kube-system -owide

NAME                                       READY   STATUS    RESTARTS      AGE   IP               NODE           NOMINATED NODE   READINESS GATES

calico-kube-controllers-5dffd5886b-4blh6   1/1     Running   0             77m   172.25.244.193   k8s-master01   <none>           <none>

calico-node-fvbdq                          1/1     Running   1 (75m ago)   77m   192.168.1.61     k8s-master01   <none>           <none>

calico-node-g8nqd                          1/1     Running   0             77m   192.168.1.64     k8s-node01     <none>           <none>

calico-node-mdps8                          1/1     Running   0             77m   192.168.1.65     k8s-node02     <none>           <none>

calico-node-nf4nt                          1/1     Running   0             77m   192.168.1.63     k8s-master03   <none>           <none>

calico-node-sq2ml                          1/1     Running   0             77m   192.168.1.62     k8s-master02   <none>           <none>

calico-typha-8445487f56-mg6p8              1/1     Running   0             77m   192.168.1.65     k8s-node02     <none>           <none>

calico-typha-8445487f56-pxbpj              1/1     Running   0             77m   192.168.1.61     k8s-master01   <none>           <none>

calico-typha-8445487f56-tnssl              1/1     Running   0             77m   192.168.1.64     k8s-node01     <none>           <none>

coredns-5db5696c7-67h79                    1/1     Running   0             63m   172.25.92.65     k8s-master02   <none>           <none>

metrics-server-6bf7dcd649-5fhrw            1/1     Running   0             61m   172.18.195.1     k8s-master03   <none>           <none>
进入busybox ping其他节点上的pod
kubectl exec -ti busybox -- sh

/ # ping 192.168.1.64

PING 192.168.1.64 (192.168.1.64): 56 data bytes

64 bytes from 192.168.1.64: seq=0 ttl=63 time=0.358 ms

64 bytes from 192.168.1.64: seq=1 ttl=63 time=0.668 ms

64 bytes from 192.168.1.64: seq=2 ttl=63 time=0.637 ms

64 bytes from 192.168.1.64: seq=3 ttl=63 time=0.624 ms

64 bytes from 192.168.1.64: seq=4 ttl=63 time=0.907 ms
可以连通证明这个pod是可以跨命名空间和跨主机通信的

12.6创建三个副本，可以看到3个副本分布在不同的节点上（用完可以删了）

代码语言：shell

复制

cat > deployments.yaml << EOF apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment labels: app: nginx spec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.14.2 ports: - containerPort: 80 EOF kubectl apply -f deployments.yaml deployment.apps/nginx-deployment created kubectl get pod NAME READY STATUS RESTARTS AGE busybox 1/1 Running 0 6m25s nginx-deployment-9456bbbf9-4bmvk 1/1 Running 0 8s nginx-deployment-9456bbbf9-9rcdk 1/1 Running 0 8s nginx-deployment-9456bbbf9-dqv8s 1/1 Running 0 8s 删除nginx

[root@k8s-master01 ~]# kubectl delete -f deployments.yaml

13.安装dashboard

代码语言：shell

复制

cd /root/Kubernetes/dashboard/

kubectl create -f . serviceaccount/admin-user created clusterrolebinding.rbac.authorization.k8s.io/admin-user created namespace/kubernetes-dashboard created serviceaccount/kubernetes-dashboard created service/kubernetes-dashboard created secret/kubernetes-dashboard-certs created secret/kubernetes-dashboard-csrf created secret/kubernetes-dashboard-key-holder created configmap/kubernetes-dashboard-settings created role.rbac.authorization.k8s.io/kubernetes-dashboard created clusterrole.rbac.authorization.k8s.io/kubernetes-dashboard created rolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created clusterrolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created deployment.apps/kubernetes-dashboard created service/dashboard-metrics-scraper created deployment.apps/dashboard-metrics-scraper created

13.1创建管理员用户

代码语言：shell

复制

cat > admin.yaml << EOF apiVersion: v1 kind: ServiceAccount metadata: name: admin-user namespace: kube-system apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: admin-user annotations: rbac.authorization.kubernetes.io/autoupdate: "true" roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: cluster-admin subjects: kind: ServiceAccount name: admin-user namespace: kube-system

EOF

13.2执行yaml文件

代码语言：shell

复制

kubectl apply -f admin.yaml -n kube-system

serviceaccount/admin-user created clusterrolebinding.rbac.authorization.k8s.io/admin-user created

13.3更改dashboard的svc为NodePort，如果已是请忽略

代码语言：shell

复制

kubectl edit svc kubernetes-dashboard -n kubernetes-dashboard

type: NodePort

13.4查看端口号

代码语言：shell

复制

kubectl get svc kubernetes-dashboard -n kubernetes-dashboard

NAME                   TYPE       CLUSTER-IP     EXTERNAL-IP   PORT(S)         AGE

kubernetes-dashboard   NodePort   10.98.201.22   <none>        443:31245/TCP   10m

13.5查看token

代码语言：shell

复制

kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep admin-user | awk '{print $1}') Name: admin-user-token-5vfk4 Namespace: kube-system Labels: <none> Annotations: kubernetes.io/service-account.name: admin-user kubernetes.io/service-account.uid: fc2535ae-8760-4037-9026-966f03ab9bf9 Type: kubernetes.io/service-account-token Data

ca.crt: 1363 bytes namespace: 11 bytes token: eyJhbGciOiJSUzI1NiIsImtpZCI6InVOMnhMdHFTRWxweUlfUm93VmhMZTVXZW1FXzFrT01nQ0dTcE5uYjJlNWMifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJhZG1pbi11c2VyLXRva2VuLTV2Zms0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImFkbWluLXVzZXIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiJmYzI1MzVhZS04NzYwLTQwMzctOTAyNi05NjZmMDNhYjliZjkiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZS1zeXN0ZW06YWRtaW4tdXNlciJ9.HSU1FeqY6pDVoXVIv4Lu27TDhCYHM-FzGsGybYL5QPJ5-P0b3tQqUH9i3AQlisiGPB--jCFT5CUeOeXneOyfV7XkC7frbn6VaQoh51n6ztkIvjUm8Q4xj_LQ2OSFfWlFUnaZsaYTdD-RCldwh63pX362T_FjgDknO4q1wtKZH5qR0mpL1dOjas50gnOSyBY0j-nSPrifhnNq3_GcDLE4LxjuzO1DfGNTEHZ6TojPJ_5ZElMolaYJsVejn2slfeUQEWdiD5AHFZlRd4exODCHyvUhRpzb9jO2rovN2LMqdE_vxBtNgXp19evQB9AgZyMMSmu1Ch2C2UAi4NxjKw8HNA

13.6登录dashboard

https://192.168.1.61/

eyJhbGciOiJSUzI1NiIsImtpZCI6InYzV2dzNnQzV3hHb2FQWnYzdnlOSmpudmtpVmNjQW5VM3daRi12SFM4dEEifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJhZG1pbi11c2VyLXRva2VuLWs1NDVrIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImFkbWluLXVzZXIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiJjMzA4MDcxYy00Y2Y1LTQ1ODMtODNhMi1lYWY3ODEyNTEyYjQiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZS1zeXN0ZW06YWRtaW4tdXNlciJ9.pshvZPi9ZJkXUWuWilcYs1wawTpzV-nMKesgF3d_l7qyTPaK2N5ofzIThd0SjzU7BFNb4_rOm1dw1Be5kLeHjY_YW5lDnM5TAxVPXmZQ0HJ2pAQ0pjQqCHFnPD0bZFIYkeyz8pZx0Hmwcd3ZdC1yztr0ADpTAmMgI9NC2ZFIeoFFo4Ue9ZM_ulhqJQjmgoAlI_qbyjuKCNsWeEQBwM6HHHAsH1gOQIdVxqQ83OQZUuynDQRpqlHHFIndbK2zVRYFA3GgUnTu2-VRQ-DXBFRjvZR5qArnC1f383jmIjGT6VO7l04QJteG_LFetRbXa-T4mcnbsd8XutSgO0INqwKpjw

14.安装命令行自动补全功能

代码语言：shell

复制

yum install bash-completion -y

source /usr/share/bash-completion/bash_completion

source <(kubectl completion bash)

echo "source <(kubectl completion bash)" >> ~/.bashrc

附录：

配置kube-controller-manager有效期100年（能不能生效的先配上再说）

代码语言：shell

复制

vim /usr/lib/systemd/system/kube-controller-manager.service [Service]下找个地方加上 --cluster-signing-duration=876000h0m0s \ 重启

systemctl daemon-reload systemctl restart kube-controller-manager

防止漏洞扫描

代码语言：shell

复制

vim /etc/systemd/system/kubelet.service.d/10-kubelet.conf
[Service]

Environment="KUBELET_KUBECONFIG_ARGS=--kubeconfig=/etc/kubernetes/kubelet.kubeconfig --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.kubeconfig"

Environment="KUBELET_SYSTEM_ARGS=--network-plugin=cni --cni-conf-dir=/etc/cni/net.d --cni-bin-dir=/opt/cni/bin"

Environment="KUBELET_CONFIG_ARGS=--config=/etc/kubernetes/kubelet-conf.yml  --pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.6"

Environment="KUBELET_EXTRA_ARGS=--tls-cipher-suites=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384    --image-pull-progress-deadline=30m"

ExecStart=

ExecStart=/usr/local/bin/kubelet KUBELET_KUBECONFIG_ARGS KUBELET_CONFIG_ARGS KUBELET_SYSTEM_ARGS KUBELET_EXTRA_ARGS

预留空间，按需分配

代码语言：shell

复制

vim /etc/kubernetes/kubelet-conf.yml
rotateServerCertificates: true

allowedUnsafeSysctls:

"net.core*"
"net.ipv4.*"

kubeReserved:

cpu: "1"

memory: 1Gi

ephemeral-storage: 10Gi

systemReserved:

cpu: "1"

memory: 1Gi

ephemeral-storage: 10Gi

数据盘要与系统盘分开；etcd使用ssd磁盘