Kubernetes 生产环境安装部署基于 Kubernetes v1.14.0 之 etcd集群

时间：2019-05-31 14:12:55 阅读：154 评论：0 收藏：0 [点我收藏+]

说明：没有明确注明在某台服务器，都是在k8s-operation 工作服务器完成 K8S node 节点数大于2000 节点

k8s-operation 目录规划，工作目录/apps/work/k8s 目录进行操作集群所用域名 niuke.local

etcd 集群准备

1、etcd 服务器配置

etcd 中心集群

对外ip地址	k8s集群连接ip地址	cpu	内存	硬盘
192.168.2.247	172.172.0.1	16	32	120G
192.168.2.248	172.172.0.2	16	32	120G
192.168.2.249	172.172.0.3	16	32	120G

etcd 事件集群

对外ip地址	k8s集群连接ip地址	cpu	内存	硬盘
192.168.2.250	172.172.0.4	16	32	120G
192.168.2.251	172.172.0.5	16	32	120G
192.168.2.252	172.172.0.6	16	32	120G

2、etcd 二进制准备

cd /apps/work/k8s
mkdir etcd
cd etcd
wgte https://github.com/etcd-io/etcd/releases/download/v3.3.12/etcd-v3.3.12-linux-arm64.tar.gz
tar -xvf etcd-v3.3.12-linux-arm64.tar.gz
mkdir bin conf data ssl
mv etcd* bin/
rm -f etcd-v3.3.12-linux-arm64.tar.gz

3、etcd 证书准备

3.1 创建证书配置文件

mkdir -p /apps/work/k8s/cfssl/ && \
cat << EOF | tee /apps/work/k8s/cfssl/ca-config.json
{
"signing": {
"default": {
"expiry": "87600h"
},
"profiles": {
"kubernetes": {
"usages": [
"signing",
"key encipherment",
"server auth",
"client auth"
],
"expiry": "87600h"
}
}
}
}
EOF

3.2 创建etcd ca证书配置

mkdir -p /apps/work/k8s/cfssl/etcd
cat << EOF | tee /apps/work/k8s/cfssl/etcd/etcd-ca-csr.json
{
"CN": "etcd",
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "GuangDong",
"L": "GuangZhou",
"O": "niuke",
"OU": "niuke"
}
]
}
EOF

生成 ETCD CA 证书和私钥

mkdir -p /apps/work/k8s/cfssl/pki/etcd
cfssl gencert -initca /apps/work/k8s/cfssl/etcd/etcd-ca-csr.json | cfssljson -bare /apps/work/k8s/cfssl/pki/etcd/etcd-ca

3.3 创建 ETCD Server 证书中心集群

3.3.1 创建 ETCD Server 配置文件

export ETCD_SERVER_IPS=" \
\"172.172.0.1\", \
\"172.172.0.2\", \
\"172.172.0.3\" \
" && \
export ETCD_SERVER_HOSTNAMES=" \
\"etcd01\", \
\"etcd02\", \
\"etcd03\" \
" && \
cat << EOF | tee /apps/work/k8s/cfssl/etcd/etcd_server.json
{
"CN": "etcd",
"hosts": [
"127.0.0.1",
${ETCD_SERVER_IPS},
${ETCD_SERVER_HOSTNAMES}
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "GuangDong",
"L": "GuangZhou",
"O": "niuke",
"OU": "niuke"
}
]
}
EOF

生成 ETCD Server 证书和私钥

cfssl gencert \
-ca=/apps/work/k8s/cfssl/pki/etcd/etcd-ca.pem \
-ca-key=/apps/work/k8s/cfssl/pki/etcd/etcd-ca-key.pem \
-config=/apps/work/k8s/cfssl/ca-config.json \
-profile=kubernetes \
/apps/work/k8s/cfssl/etcd/etcd_server.json | \
cfssljson -bare /apps/work/k8s/cfssl/pki/etcd/etcd_server

3.4 创建 ETCD Member 证书中心集群

3.4.1 创建 ETCD Member 1 配置文件

export ETCD_MEMBER_1_IP=" \
\"172.172.0.1\" \
" && \
export ETCD_MEMBER_1_HOSTNAMES="etcd01\
" && \
cat << EOF | tee /apps/work/k8s/cfssl/etcd/${ETCD_MEMBER_1_HOSTNAMES}.json
{
"CN": "etcd",
"hosts": [
"127.0.0.1",
${ETCD_MEMBER_1_IP},
"${ETCD_MEMBER_1_HOSTNAMES}"
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "GuangDong",
"L": "GuangZhou",
"O": "niuke",
"OU": "niuke"
}
]
}
EOF

生成 ETCD Member 1 证书和私钥

cfssl gencert \
-ca=/apps/work/k8s/cfssl/pki/etcd/etcd-ca.pem \
-ca-key=/apps/work/k8s/cfssl/pki/etcd/etcd-ca-key.pem \
-config=/apps/work/k8s/cfssl/ca-config.json \
-profile=kubernetes \
/apps/work/k8s/cfssl/etcd/${ETCD_MEMBER_1_HOSTNAMES}.json | \
cfssljson -bare /apps/work/k8s/cfssl/pki/etcd/etcdmember${ETCD_MEMBER_1_HOSTNAMES}

3.4.2 创建 ETCD Member 2 配置文件

export ETCD_MEMBER_2_IP=" \
\"172.172.0.2\" \
" && \
export ETCD_MEMBER_2_HOSTNAMES=etcd02\
" && \
cat << EOF | tee /apps/work/k8s/cfssl/etcd/${ETCD_MEMBER_2_HOSTNAMES}.json
{
"CN": "etcd",
"hosts": [
"127.0.0.1",
${ETCD_MEMBER_2_IP},
"${ETCD_MEMBER_2_HOSTNAMES}"
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "GuangDong",
"L": "GuangZhou",
"O": "niuke",
"OU": "niuke"
}
]
}
EOF

生成 ETCD Member 2 证书和私钥

cfssl gencert \
-ca=/apps/work/k8s/cfssl/pki/etcd/etcd-ca.pem \
-ca-key=/apps/work/k8s/cfssl/pki/etcd/etcd-ca-key.pem \
-config=/apps/work/k8s/cfssl/ca-config.json \
-profile=kubernetes \
/apps/work/k8s/cfssl/etcd/${ETCD_MEMBER_2_HOSTNAMES}.json | \
cfssljson -bare /apps/work/k8s/cfssl/pki/etcd/etcdmember${ETCD_MEMBER_2_HOSTNAMES}

3.4.3 创建 ETCD Member 3 配置文件

export ETCD_MEMBER_3_IP=" \
\"172.172.0.3\" \
" && \
export ETCD_MEMBER_3_HOSTNAMES="etcd03\
" && \
cat << EOF | tee /apps/work/k8s/cfssl/etcd/${ETCD_MEMBER_3_HOSTNAMES}.json
{
"CN": "etcd",
"hosts": [
"127.0.0.1",
${ETCD_MEMBER_3_IP},
"${ETCD_MEMBER_3_HOSTNAMES}"
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "GuangDong",
"L": "GuangZhou",
"O": "niuke",
"OU": "niuke"
}
]
}
EOF

生成 ETCD Member 3 证书和私钥

cfssl gencert \
-ca=/apps/work/k8s/cfssl/pki/etcd/etcd-ca.pem \
-ca-key=/apps/work/k8s/cfssl/pki/etcd/etcd-ca-key.pem \
-config=/apps/work/k8s/cfssl/ca-config.json \
-profile=kubernetes \
/apps/work/k8s/cfssl/etcd/${ETCD_MEMBER_3_HOSTNAMES}.json | \
cfssljson -bare /apps/work/k8s/cfssl/pki/etcd/etcdmember${ETCD_MEMBER_3_HOSTNAMES}

3.5 创建 ETCD EVENTS Server 证书

3.5.1 创建 ETCD EVENTS Server 配置文件

export ETCD_EVENTS_IPS=" \
\"172.172.0.4\", \
\"172.172.0.5\", \
\"172.172.0.6\" \
" && \
export ETCD_EVENTS_HOSTNAMES=" \
\"etcd-even01\", \
\"etcd-even02\", \
\"etcd-even03\" \
" && \
cat << EOF | tee /apps/work/k8s/cfssl/etcd/etcd_events.json
{
"CN": "etcd",
"hosts": [
"127.0.0.1",
${ETCD_EVENTS_IPS},
${ETCD_EVENTS_HOSTNAMES}
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "GuangDong",
"L": "GuangZhou",
"O": "niuke",
"OU": "niuke"
}
]
}
EOF

生成 ETCD EVENTS 证书和私钥

cfssl gencert \
-ca=/apps/work/k8s/cfssl/pki/etcd/etcd-ca.pem \
-ca-key=/apps/work/k8s/cfssl/pki/etcd/etcd-ca-key.pem \
-config=/apps/work/k8s/cfssl/ca-config.json \
-profile=kubernetes \
/apps/work/k8s/cfssl/etcd/etcd_events.json | \
cfssljson -bare /apps/work/k8s/cfssl/pki/etcd/etcd_events

3.5.2 创建 ETCD EVENTS Member 1 配置文件

export ETCD_EVENTS_MEMBER_1_IP=" \
\"172.172.0.4\" \
" && \
export ETCD_EVENTS_MEMBER_1_HOSTNAMES="etcd-even01\
" && \
cat << EOF | tee /apps/work/k8s/cfssl/etcd/"$ETCD_EVENTS_MEMBER_1_HOSTNAMES".json
{
"CN": "etcd",
"hosts": [
"127.0.0.1",
${ETCD_EVENTS_MEMBER_1_IP},
"${ETCD_EVENTS_MEMBER_1_HOSTNAMES}"
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "GuangDong",
"L": "GuangZhou",
"O": "niuke",
"OU": "niuke"
}
]
}
EOF

生成 ETCD EVENTS Member 1 证书和私钥

cfssl gencert \
-ca=/apps/work/k8s/cfssl/pki/etcd/etcd-ca.pem \
-ca-key=/apps/work/k8s/cfssl/pki/etcd/etcd-ca-key.pem \
-config=/apps/work/k8s/cfssl/ca-config.json \
-profile=kubernetes \
/apps/work/k8s/cfssl/etcd/${ETCD_EVENTS_MEMBER_1_HOSTNAMES}.json | \
cfssljson -bare /apps/work/k8s/cfssl/pki/etcd/etcdevents${ETCD_EVENTS_MEMBER_1_HOSTNAMES}

3.5.3 创建 ETCD EVENTS Member 2 配置文件

export ETCD_EVENTS_MEMBER_2_IP=" \
\"172.172.0.5\" \
" && \
export ETCD_EVENTS_MEMBER_2_HOSTNAMES="etcd-even02\
" && \
cat << EOF | tee /apps/work/k8s/cfssl/etcd/"${ETCD_EVENTS_MEMBER_2_HOSTNAMES}".json
{
"CN": "etcd",
"hosts": [
"127.0.0.1",
${ETCD_EVENTS_MEMBER_2_IP},
"${ETCD_EVENTS_MEMBER_2_HOSTNAMES}"
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "GuangDong",
"L": "GuangZhou",
"O": "niuke",
"OU": "niuke"
}
]
}
EOF

生成 ETCD EVENTS Member 2 证书和私钥

cfssl gencert \
-ca=/apps/work/k8s/cfssl/pki/etcd/etcd-ca.pem \
-ca-key=/apps/work/k8s/cfssl/pki/etcd/etcd-ca-key.pem \
-config=/apps/work/k8s/cfssl/ca-config.json \
-profile=kubernetes \
/apps/work/k8s/cfssl/etcd/${ETCD_EVENTS_MEMBER_2_HOSTNAMES}.json | \
cfssljson -bare /apps/work/k8s/cfssl/pki/etcd/etcdevents${ETCD_EVENTS_MEMBER_2_HOSTNAMES}

3.5.4 创建 ETCD EVENTS Member 3 配置文件

export ETCD_EVENTS_MEMBER_3_IP=" \
\"172.172.0.6\" \
" && \
export ETCD_EVENTS_MEMBER_3_HOSTNAMES="etcd-even03\
" && \
cat << EOF | tee /apps/work/k8s/cfssl/etcd/${ETCD_EVENTS_MEMBER_3_HOSTNAMES}.json
{
"CN": "etcd",
"hosts": [
"127.0.0.1",
${ETCD_EVENTS_MEMBER_3_IP},
"${ETCD_EVENTS_MEMBER_3_HOSTNAMES}"
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "GuangDong",
"L": "GuangZhou",
"O": "niuke",
"OU": "niuke"
}
]
}
EOF

生成 ETCD EVENTS Member 3 证书和私钥

cfssl gencert \
-ca=/apps/work/k8s/cfssl/pki/etcd/etcd-ca.pem \
-ca-key=/apps/work/k8s/cfssl/pki/etcd/etcd-ca-key.pem \
-config=/apps/work/k8s/cfssl/ca-config.json \
-profile=kubernetes \
/apps/work/k8s/cfssl/etcd/${ETCD_EVENTS_MEMBER_3_HOSTNAMES}.json | \
cfssljson -bare /apps/work/k8s/cfssl/pki/etcd/etcdevents${ETCD_EVENTS_MEMBER_3_HOSTNAMES}

3.6 创建 ETCD Client 配置文件

cat << EOF | tee /apps/work/k8s/cfssl/etcd/etcd_client.json
{
"CN": "client",
"hosts": [""],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "GuangDong",
"L": "GuangZhou",
"O": "niuke",
"OU": "niuke"
}
]
}
EOF

生成 ETCD Client 证书和私钥

cfssl gencert \
-ca=/apps/work/k8s/cfssl/pki/etcd/etcd-ca.pem \
-ca-key=/apps/work/k8s/cfssl/pki/etcd/etcd-ca-key.pem \
-config=/apps/work/k8s/cfssl/ca-config.json \
-profile=kubernetes \
/apps/work/k8s/cfssl/etcd/etcd_client.json | \
cfssljson -bare /apps/work/k8s/cfssl/pki/etcd/etcd_client

3.7 cp 证书到分发目录 /apps/work/k8s/etcd/ssl

cp -pdr /apps/work/k8s/cfssl/pki/etcd/ /apps/work/k8s/etcd/ssl

4. 创建 etcd 启动配置文件只写一个例子其它节点参考

 cd /apps/work/k8s/etcd/conf
 vim etcd
 ETCD_OPTS="--name=etcd01        --data-dir=/apps/etcd/data/default.etcd        --listen-peer-urls=https://172.172.0.1:2380        --listen-client-urls=https://172.172.0.1:2379,https://127.0.0.1:2379        --advertise-client-urls=https://172.172.0.1:2379        --initial-advertise-peer-urls=https://172.172.0.1:2380        --initial-cluster=etcd01=https://172.172.0.1:2380,etcd02=https://172.172.0.2:2380,etcd03=https://172.172.0.3:2380        --initial-cluster-token=etcd01=https://172.172.0.1:2380,etcd02=https://172.172.0.2:2380,etcd03=https:/172.172.0.3:2380        --initial-cluster-state=new        --heartbeat-interval=6000        --election-timeout=30000        --snapshot-count=5000        --auto-compaction-retention=1        --max-request-bytes=33554432        --quota-backend-bytes=17179869184        --trusted-ca-file=/apps/etcd/ssl/etcd-ca.pem        --cert-file=/apps/etcd/ssl/etcd_server.pem        --key-file=/apps/etcd/ssl/etcd_server-key.pem        --peer-cert-file=/apps/etcd/ssl/etcd_member_etcd01.pem        --peer-key-file=/apps/etcd/ssl/etcd_member_etcd01-key.pem        --peer-client-cert-auth        --peer-trusted-ca-file=/apps/etcd/ssl/etcd-ca.pem"

5. 创建etcd.service

vim etcd.service

 [Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
LimitNOFILE=1024000
LimitNPROC=1024000
LimitCORE=infinity
LimitMEMLOCK=infinity
User=etcd
Group=etcd

EnvironmentFile=-/apps/etcd/conf/etcd
ExecStart=/apps/etcd/bin/etcd $ETCD_OPTS
Restart=on-failure

[Install]
WantedBy=multi-user.target

6. ansible 分发ETCD 到etcd 集群

6.1 创建远程服务器etcd 账号 host 文件写etcd 集群ip

ansible -i host etcd -m shell -a "useradd etcd -s /sbin/nologin -M"

6.2 分发etcd 文件到etcd 集群

ansible -i host etcd -m copy -a "src=etcd/ dest=/apps/etcd"

6.3 分发etcd etcd.service

ansible -i host etcd -m copy -a "src=etcd.service dest=/usr/lib/systemd/system/etcd.service"

6.3 修改远程服务器etcd 目录权限

ansible -i host etcd -m shell -a "chown -R etcd:etcd /apps/etcd"

6.4 设置etcd 可执行权限

ansible -i host etcd -m shell -a "chmod +x /apps/etcd/bin/*"

6.5 分发 /apps/work/k8s/conf/etcd 每台服务器配置不一样可以配置成etcd-hostname 然后分发

ansible -i 192.168.2.247, all -m copy -a "src=etcd/etcd-etcd01 dest=/apps/etcd/conf/etcd"

6.6 启动etcd 集群

ansible -i host etcd -m shell -a "systemctl daemon-reload"
ansible -i host etcd -m shell -a "systemctl start etcd"
ansible -i host etcd -m shell -a "systemctl enable etcd"

6.7 检查etcd 是否启动成功

ansible -i host etcd -m shell -a "systemctl status etcd"

6.8 2个etcd 集群任意服务检查etcd 集群是否成功

设置环境变量使etcd 集群支持v3版本
vim /etc/profile
export ETCDCTL_API=3
export ENDPOINTS=https://172.172.0.1:2379,https://172.172.0.2:2379,https://172.172.0.3:2379
vim ~/.bashrc
alias etcdctl=‘/apps/etcd/bin/etcdctl --endpoints=${ENDPOINTS} --cacert=/apps/etcd/ssl/etcd-ca.pem‘
etcdctl endpoint health
[root@etcd01 ~]# etcdctl endpoint health
https://172.172.0.1:2379 is healthy: successfully committed proposal: took = 1.879627ms
https://172.172.0.2:2379 is healthy: successfully committed proposal: took = 2.504475ms
https://172.172.0.3:2379 is healthy: successfully committed proposal: took = 2.389547ms
etcdctl endpoint status
[root@etcd01 ~]# etcdctl endpoint status
https://172.172.0.1:2379, 7b98f2ed4d780753, 3.3.12, 290 MB, true, 37627, 22396898
https://172.172.0.2:2379, 47fa5d2eb78a7751, 3.3.12, 289 MB, false, 37627, 22396898
https://172.172.0.3:2379, 76c6cd81499cf7ba, 3.3.12, 289 MB, false, 37627, 22396898
第二个集群用同样的方法检查查看是否正常

7. 个人etcd 集群playbook

cd /apps/work/k8s
mkdir -p roles/etcd/{defaults,files,handlers,meta,tasks,templates,vars}
host 配置
[etcd]
192.168.2.249
192.168.2.248
192.168.2.247
[events]
192.168.2.250
192.168.2.251
192.168.2.252
[etcd:vars]
initial_cluster="etcd01=https://172.172.0.1:2380,etcd02=https://172.172.0.2:2380,etcd03=https:/172.172.0.3:2380"
cert_file=etcd_server
ca=etcd-ca
ETCD_PATH=/apps
[events:vars]
initial_cluster="etcd-even01=https://172.172.0.4:2380,etcd-even02=https://172.172.0.5:2380,etcd-even03=https:/172.172.0.6:2380"
cert_file=etcd_events
ca=etcd-ca
ETCD_PATH=/apps

目录结构

cd /apps/work/k8s/roles/etcd

[root@jenkins etcd]# tree
.
├── defaults
├── files
│?? ├── bin
│?? │?? ├── etcd
│?? │?? └── etcdctl
│?? └── ssl
│?? ├── etcd-ca.csr
│?? ├── etcd-ca-key.pem
│?? ├── etcd-ca.pem
│?? ├── etcd_client.csr
│?? ├── etcd_client-key.pem
│?? ├── etcd_client.pem
│?? ├── etcd_events_ceph-2-57.csr
│?? ├── etcd_events_ceph-2-57-key.pem
│?? ├── etcd_events_ceph-2-57.pem
│?? ├── etcd_events_ceph-2-91.csr
│?? ├── etcd_events_ceph-2-91-key.pem
│?? ├── etcd_events_ceph-2-91.pem
│?? ├── etcd_events_ceph-2-92.csr
│?? ├── etcd_events_ceph-2-92-key.pem
│?? ├── etcd_events_ceph-2-92.pem
│?? ├── etcd_events.csr
│?? ├── etcd_events-key.pem
│?? ├── etcd_events_member01.csr
│?? ├── etcd_events_member01-key.pem
│?? ├── etcd_events_member01.pem
│?? ├── etcd_events_member02.csr
│?? ├── etcd_events_member02-key.pem
│?? ├── etcd_events_member02.pem
│?? ├── etcd_events_member03.csr
│?? ├── etcd_events_member03-key.pem
│?? ├── etcd_events_member03.pem
│?? ├── etcd_events.pem
│?? ├── etcd_member01.csr
│?? ├── etcd_member01-key.pem
│?? ├── etcd_member01.pem
│?? ├── etcd_member02.csr
│?? ├── etcd_member02-key.pem
│?? ├── etcd_member02.pem
│?? ├── etcd_member03.csr
│?? ├── etcd_member03-key.pem
│?? ├── etcd_member03.pem
│?? ├── etcd_server.csr
│?? ├── etcd_server-key.pem
│?? └── etcd_server.pem
├── handlers
├── meta
├── tasks
│?? └── main.yml
├── templates
│?? ├── etcd
│?? └── etcd.service
└── vars

tasks/main.yml

- name: create groupadd etcd
  group: name=etcd
- name: create name etcd
  user: name=etcd shell="/sbin/nologin etcd" group=etcd
- name: mkdir {{ ETCD_PATH }}
  raw: mkdir -p {{ ETCD_PATH }}/etcd/{conf,ssl,bin} && mkdir -p {{ ETCD_PATH }}/etcd/data/default.etcd
- name: copy etcd
  copy: src=bin dest={{ ETCD_PATH }}/etcd/ owner=root group=root mode=755
- name: copy etcd ssl
  copy: src=ssl dest={{ ETCD_PATH }}/etcd/
- name: src=etcd dest={{ ETCD_PATH }}/etcd/conf
  template: src=etcd dest={{ ETCD_PATH }}/etcd/conf
- name: copy etcd.service
  template: src=etcd.service  dest=/usr/lib/systemd/system/
- name: chown -R etcd:etcd {{ ETCD_PATH }}/etcd/
  shell: chown -R etcd:etcd {{ ETCD_PATH }}/etcd/
- name: systemctl daemon-reload
  shell: systemctl daemon-reload
- name: systemctl enable etcd && systemctl start etcd
  shell: systemctl enable etcd && systemctl start etcd

templates/etcd

说明 ansible_default_ipv4.address 根据情况进行修改成服务器所在的网卡ip地址可以使用ansible setup 模块获取

ETCD_OPTS="--name={{ ansible_hostname }}            --data-dir={{ ETCD_PATH }}/etcd/data/default.etcd            --listen-peer-urls=https://{{ ansible_default_ipv4.address }}:2380            --listen-client-urls=https://{{ ansible_default_ipv4.address }}:2379,https://127.0.0.1:2379            --advertise-client-urls=https://{{ ansible_default_ipv4.address }}:2379            --initial-advertise-peer-urls=https://{{ ansible_default_ipv4.address }}:2380            --initial-cluster={{ initial_cluster }}            --initial-cluster-token={{ initial_cluster }}            --initial-cluster-state=new            --heartbeat-interval=6000            --election-timeout=30000            --snapshot-count=5000            --auto-compaction-retention=1            --max-request-bytes=33554432            --quota-backend-bytes=17179869184            --trusted-ca-file={{ ETCD_PATH }}/etcd/ssl/{{ ca }}.pem            --cert-file={{ ETCD_PATH }}/etcd/ssl/{{ cert_file }}.pem            --key-file={{ ETCD_PATH }}/etcd/ssl/{{ cert_file }}-key.pem            --peer-cert-file={{ ETCD_PATH }}/etcd/ssl/etcd_events_{{ ansible_hostname }}.pem            --peer-key-file={{ ETCD_PATH }}/etcd/ssl/etcd_events_{{ ansible_hostname }}-key.pem            --peer-client-cert-auth            --peer-trusted-ca-file={{ ETCD_PATH }}/etcd/ssl/{{ ca }}.pem"

templates/etcd.service

[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
LimitNOFILE=1024000
LimitNPROC=1024000
LimitCORE=infinity
LimitMEMLOCK=infinity
User=etcd
Group=etcd

EnvironmentFile=-{{ ETCD_PATH }}/etcd/conf/etcd
ExecStart={{ ETCD_PATH }}/etcd/bin/etcd $ETCD_OPTS
Restart=on-failure

[Install]
WantedBy=multi-user.target

Kubernetes 生产环境安装部署基于 Kubernetes v1.14.0 之 kube-apiserver集群部署

Kubernetes 生产环境安装部署基于 Kubernetes v1.14.0 之 etcd集群

原文：https://blog.51cto.com/juestnow/2403075

踩

(0)

评论一句话评论（0）

分享档案

更多>

2021年09月23日 (328)
2021年09月24日 (313)
2021年09月17日 (191)
2021年09月15日 (369)
2021年09月16日 (411)
2021年09月13日 (439)
2021年09月11日 (398)
2021年09月12日 (393)
2021年09月10日 (160)
2021年09月08日 (222)

Kubernetes 生产环境安装部署 基于 Kubernetes v1.14.0 之 etcd集群

k8s-operation 目录规划，工作目录/apps/work/k8s 目录进行操作 集群所用域名 niuke.local

etcd 集群准备

1、etcd 服务器配置

etcd 中心集群

etcd 事件集群

2、etcd 二进制准备

3、etcd 证书准备

3.1 创建证书配置文件

3.2 创建etcd ca证书配置

生成 ETCD CA 证书和私钥

3.3 创建 ETCD Server 证书 中心集群

3.3.1 创建 ETCD Server 配置文件

生成 ETCD Server 证书和私钥

3.4 创建 ETCD Member 证书 中心集群

3.4.1 创建 ETCD Member 1 配置文件

生成 ETCD Member 1 证书和私钥

3.4.2 创建 ETCD Member 2 配置文件

生成 ETCD Member 2 证书和私钥

3.4.3 创建 ETCD Member 3 配置文件

生成 ETCD Member 3 证书和私钥

3.5 创建 ETCD EVENTS Server 证书

3.5.1 创建 ETCD EVENTS Server 配置文件

生成 ETCD EVENTS 证书和私钥

3.5.2 创建 ETCD EVENTS Member 1 配置文件

生成 ETCD EVENTS Member 1 证书和私钥

3.5.3 创建 ETCD EVENTS Member 2 配置文件

生成 ETCD EVENTS Member 2 证书和私钥

3.5.4 创建 ETCD EVENTS Member 3 配置文件

生成 ETCD EVENTS Member 3 证书和私钥

3.6 创建 ETCD Client 配置文件

生成 ETCD Client 证书和私钥

3.7 cp 证书到分发目录 /apps/work/k8s/etcd/ssl

4. 创建 etcd 启动配置文件 只写一个例子 其它节点参考

5. 创建etcd.service

6. ansible 分发ETCD 到etcd 集群

6.1 创建远程服务器etcd 账号 host 文件写etcd 集群ip

6.2 分发etcd 文件到etcd 集群

6.3 分发etcd etcd.service

6.3 修改远程服务器etcd 目录权限

6.4 设置etcd 可执行权限

6.5 分发 /apps/work/k8s/conf/etcd 每台服务器配置不一样 可以配置成etcd-hostname 然后分发

6.6 启动etcd 集群

6.7 检查etcd 是否启动成功

6.8 2个etcd 集群任意服务检查etcd 集群是否成功

7. 个人etcd 集群playbook

目录结构

tasks/main.yml

templates/etcd

templates/etcd.service

Kubernetes 生产环境安装部署基于 Kubernetes v1.14.0 之 etcd集群

k8s-operation 目录规划，工作目录/apps/work/k8s 目录进行操作集群所用域名 niuke.local

3.3 创建 ETCD Server 证书中心集群

3.4 创建 ETCD Member 证书中心集群

4. 创建 etcd 启动配置文件只写一个例子其它节点参考

6.5 分发 /apps/work/k8s/conf/etcd 每台服务器配置不一样可以配置成etcd-hostname 然后分发