OpenShift 部署 CSR requests from inventory nodes 问题处理 | 进阶技巧

【作者】泊涯,公司分部测试经理,集团公司技术专家成员之一,目前主要在客户现场做银行系统的性能诊断分析优化和测试…

【作者】泊涯,公司分部测试经理,集团公司技术专家成员之一,目前主要在客户现场做银行系统的性能诊断分析优化和测试管理工作。
因客户业务发展迅速,原有系统对业务响应的能力就显得更加捉襟见肘,因此公司跟客户沟通后对原有系统进行IT架构转型重构,使用微服务架构方式,服务器资源虚拟化容器云服务PaaS平台,我负责基础环境的构建,包括对原有硬件进行资源虚拟化构建、阿里云服务APP端服务的PAAS平台构建部署配置,这次我们对内部管理系统和对C端客户的APP服务统一使用Openshift的容器云平台进行构建部署,实现混合云服务构建,如果根据以往方式部署,在某个tomcat出现内存泄漏等现象会导致该台服务器资源使用紧张现象,因此我们采用KVM工具对服务硬件资源进行虚拟化成多个服务,以便资源熔断隔离,然后再使用openshift来进行PAAS平台管理部署等,主要涉及的技术点多规划配置资源部署也是头等头疼事情,还好原有postgres数据库主从等还是部署在实体机上暂时不改造,最多也是数据库版本升级、操作系统版本升级、把报表服务独立出来,变成一主3从,然后原有tomcat应用、redis、MQ、NFS、DNS等部署都需要在新的容器进行资源合理规划配置部署,这也是我们公司首次打造了面向DevOps的PaaS平台的产品,这是具有开创性的 ,很多技术层面也是需要摸索,使用开源工具在配置安装部署上确实存在不少问题,很多坑需要耐心、细心进行分析排查处理。
碰到问题最多的应该是openshift配置部署, master独立一个节点、ETCD独立一个节点、若干个node子节点,子节点的硬件资源要求一模一样,因此我装完KVM 虚拟化系统后直接使用克隆方式来配置多个nodes提高效率,但是因为某个节点的IP的配置疏忽,导致报错,排查很久,而开源信息的报错信息有时跟实际发生问题提示信息不是很吻合,导致排查了很长时间,类似头痛医脚。具体问题如下:
INSTALLER STATUS *******************************************************************************************************************
Initialization : Complete (0:00:44)
Health Check : Complete (0:00:52)
Node Bootstrap Preparation : Complete (0:02:41)
在线
etcd Install : Complete (0:00:39)
Master Install : Complete (0:03:39)
Master Additional Install : Complete (0:00:43)
Node Join : In Progress (0:03:17)
This phase can be restarted by running: playbooks/openshift-node/join.yml
Failure summary:
Hosts: paas-node3.mylike.okd
Play: Disable excluders
Task: Install docker excluder – yum
Message: http://mirrors.nwsuaf.edu.cn/centos/7.6.1810/os/x86_64/repodata/repomd.xml: [Errno 14] curl#6 – “Could not resolve host: mirrors.nwsuaf.edu.cn; Unknown error”
Trying other mirror.
http://mirrors.cn99.com/centos/7.6.1810/os/x86_64/repodata/repomd.xml: [Errno 14] curl#6 – “Could not resolve host: mirrors.cn99.com; Unknown error”
Trying other mirror.
http://ap.stykers.moe/centos/7.6.1810/os/x86_64/repodata/repomd.xml: [Errno 14] curl#6 – “Could not resolve host: ap.stykers.moe; Unknown error”
Trying other mirror.
http://mirrors.tuna.tsinghua.edu.cn/centos/7.6.1810/os/x86_64/repodata/repomd.xml: [Errno 14] curl#6 – “Could not resolve host: mirrors.tuna.tsinghua.edu.cn; Unknown error”
Trying other mirror.
http://mirrors.huaweicloud.com/centos/7.6.1810/os/x86_64/repodata/repomd.xml: [Errno 14] curl#6 – “Could not resolve host: mirrors.huaweicloud.com; Unknown error”
Trying other mirror.
http://mirrors.zju.edu.cn/centos/7.6.1810/os/x86_64/repodata/repomd.xml: [Errno 14] curl#6 – “Could not resolve host: mirrors.zju.edu.cn; Unknown error”
Trying other mirror.
http://mirrors.163.com/centos/7.6.1810/os/x86_64/repodata/repomd.xml: [Errno 14] curl#6 – “Could not resolve host: mirrors.163.com; Unknown error”
Trying other mirror.
http://mirrors.nju.edu.cn/centos/7.6.1810/os/x86_64/repodata/repomd.xml: [Errno 14] curl#6 – “Could not resolve host: mirrors.nju.edu.cn; Unknown error”
Trying other mirror.
http://mirrors.aliyun.com/centos/7.6.1810/os/x86_64/repodata/repomd.xml: [Errno 14] curl#6 – “Could not resolve host: mirrors.aliyun.com; Unknown error”
Trying other mirror.
http://ftp.sjtu.edu.cn/centos/7.6.1810/os/x86_64/repodata/repomd.xml: [Errno 14] curl#6 – “Could not resolve host: ftp.sjtu.edu.cn; Unknown error”
Trying other mirror.
http://mirror.centos.org/centos/7/paas/x86_64/openshift-origin311/repodata/repomd.xml: [Errno 14] curl#6 – “Could not resolve host: mirror.centos.org; Unknown error”
Trying other mirror.
http://mirror.lzu.edu.cn/centos/7.6.1810/extras/x86_64/repodata/repomd.xml: [Errno 14] curl#6 – “Could not resolve host: mirror.lzu.edu.cn; Unknown error”
Trying other mirror.
http://centos.ustc.edu.cn/centos/7.6.1810/extras/x86_64/repodata/repomd.xml: [Errno 14] curl#6 – “Could not resolve host: centos.ustc.edu.cn; Unknown error”
Trying other mirror.
http://mirrors.njupt.edu.cn/centos/7.6.1810/extras/x86_64/repodata/repomd.xml: [Errno 14] curl#6 – “Could not resolve host: mirrors.njupt.edu.cn; Unknown error”
Trying other mirror.
http://mirrors.nwsuaf.edu.cn/centos/7.6.1810/extras/x86_64/repodata/repomd.xml: [Errno 14] curl#6 – “Could not resolve host: mirrors.nwsuaf.edu.cn; Unknown error”
Trying other mirror.
http://mirrors.cn99.com/centos/7.6.1810/extras/x86_64/repodata/repomd.xml: [Errno 14] curl#6 – “Could not resolve host: mirrors.cn99.com; Unknown error”
Trying other mirror.
http://ap.stykers.moe/centos/7.6.1810/extras/x86_64/repodata/repomd.xml: [Errno 14] curl#6 – “Could not resolve host: ap.stykers.moe; Unknown error”
Trying other mirror.
http://mirrors.tuna.tsinghua.edu.cn/centos/7.6.1810/extras/x86_64/repodata/repomd.xml: [Errno 14] curl#6 – “Could not resolve host: mirrors.tuna.tsinghua.edu.cn; Unknown error”
Trying other mirror.
http://mirrors.huaweicloud.com/centos/7.6.1810/extras/x86_64/repodata/repomd.xml: [Errno 14] curl#6 – “Could not resolve host: mirrors.huaweicloud.com; Unknown error”
Trying other mirror.
http://mirrors.aliyun.com/centos/7.6.1810/extras/x86_64/repodata/repomd.xml: [Errno 14] curl#6 – “Could not resolve host: mirrors.aliyun.com; Unknown error”
Trying other mirror.
http://ftp.sjtu.edu.cn/centos/7.6.1810/extras/x86_64/repodata/repomd.xml: [Errno 14] curl#6 – “Could not resolve host: ftp.sjtu.edu.cn; Unknown error”
Trying other mirror.
http://mirrors.neusoft.edu.cn/centos/7.6.1810/updates/x86_64/repodata/repomd.xml: [Errno 14] curl#6 – “Could not resolve host: mirrors.neusoft.edu.cn; Unknown error”
Trying other mirror.
http://mirrors.163.com/centos/7.6.1810/updates/x86_64/repodata/repomd.xml: [Errno 14] curl#6 – “Could not resolve host: mirrors.163.com; Unknown error”
Trying other mirror.
http://mirrors.nju.edu.cn/centos/7.6.1810/updates/x86_64/repodata/repomd.xml: [Errno 14] curl#6 – “Could not resolve host: mirrors.nju.edu.cn; Unknown error”
Trying other mirror.
http://mirror.lzu.edu.cn/centos/7.6.1810/updates/x86_64/repodata/repomd.xml: [Errno 14] curl#6 – “Could not resolve host: mirror.lzu.edu.cn; Unknown error”
Trying other mirror.
http://centos.ustc.edu.cn/centos/7.6.1810/updates/x86_64/repodata/repomd.xml: [Errno 14] curl#6 – “Could not resolve host: centos.ustc.edu.cn; Unknown error”
Trying other mirror.
http://mirrors.njupt.edu.cn/centos/7.6.1810/updates/x86_64/repodata/repomd.xml: [Errno 14] curl#6 – “Could not resolve host: mirrors.njupt.edu.cn; Unknown error”
Trying other mirror.
http://mirrors.nwsuaf.edu.cn/centos/7.6.1810/updates/x86_64/repodata/repomd.xml: [Errno 14] curl#6 – “Could not resolve host: mirrors.nwsuaf.edu.cn; Unknown error”
Trying other mirror.
http://mirrors.cn99.com/centos/7.6.1810/updates/x86_64/repodata/repomd.xml: [Errno 14] curl#6 – “Could not resolve host: mirrors.cn99.com; Unknown error”
Trying other mirror.
http://ap.stykers.moe/centos/7.6.1810/updates/x86_64/repodata/repomd.xml: [Errno 14] curl#6 – “Could not resolve host: ap.stykers.moe; Unknown error”
Trying other mirror.
http://mirrors.tuna.tsinghua.edu.cn/centos/7.6.1810/updates/x86_64/repodata/repomd.xml: [Errno 14] curl#6 – “Could not resolve host: mirrors.tuna.tsinghua.edu.cn; Unknown error”
Trying other mirror.
http://mirror.centos.org/centos/7/paas/x86_64/openshift-origin311/origin-docker-excluder-3.11.0-1.el7.git.0.62803d0.noarch.rpm: [Errno 14] curl#6 – “Could not resolve host: mirror.centos.org; Unknown error”
Trying other mirror.
Error downloading packages:
origin-docker-excluder-3.11.0-1.el7.git.0.62803d0.noarch: [Errno 256] No more mirrors to try.
Hosts: paas-master.mylike.okd
Play: Approve any pending CSR requests from inventory nodes
Task: Approve node certificates when bootstrapping
Message: Could not find csr for nodes: paas-node3.mylike.okd
刚开始碰到问题时,以为是因为openshift在部署时要通过master节点然后通过网络下载拉取各个不同的docker镜像进行安装部署,并再各个node同样部署,网络问题导致失效,重复多次后发现问题都一样,然后怀疑是master或者node3的yum.conf 配置有问题,修改对比多次后发现还是同样问题。最终在node3服务器上想通过yum makecache 发现也有问题,然后就ping www.baidu.com才发现这台服务无法访问外网,于是删除了网卡重新配置后,能访问外网,在重新安装openshift,问题解决,安装成功。
如有任何问题,可点击文末阅读原文到社区原文下评论交流
资料/文章推荐:
Redhat openshift okd 3.11 all in one 环境安装
http://www.talkwithtrend.com/Article/243141
开源容器云OpenShift 构建基于Kubernetes的企业应用云平台
http://www.talkwithtrend.com/Document/detail/tid/418219
欢迎关注社区“OpenShift”技术主题,将会不断更新优质资料、文章。地址:
http://www.talkwithtrend.com/Topic/91779
下载 twt 社区客户端 APP
与更多同行在一起
高手随时解答你的疑难问题
轻松订阅各领域技术主题
浏览下载最新文章资料
长按识别二维码即可下载
或到应用商店搜索“twt”
长按二维码关注公众号
*本公众号所发布内容仅代表作者观点,不代表社区立场

为您推荐

发表评论

电子邮件地址不会被公开。 必填项已用*标注