证券行业首个混合金融云面世,海通证券”信息化”探索之路再领先一步

“集团化、国际化、信息化”是海通证券近年来的发展方向。科技战略作为海通的“四根支柱”之一,打造自主可控的金融云成为公司建设智慧海通、实现科技战略的重要一环。历经三年的探索与实践,海通证券金融云从单一的VMware资源池到开源OpenStack+公有云混合资源池、从单一的集中式存储池到开源分布式存储池再到容量/性能混合存储池、从传统网络架构到新一代SDN云网架构、从研发测试环境试点到生产环境大范围推广,进而打造了中国证券行业首个自主可控的统一纳管、编排私有云资源池、行业云资源池和公有云资源池的混合基础设施服务平台。海通证券在混合金融云方案上的积极探索与实践使得业务价值进一步释放,不但引领了证券行业金融云发展,更是整个证券行业数字化转型过程的探索和创新先行者。

作为国内成立最早、实力最强、规模最大的综合性券商之一,海通证券从1988年创立至今,一直处在行业内的领军阵营,不断发展壮大,迄今公司总资产近6000亿元、净资产近1200亿元,位居国内证券行业前列,经营网点遍及全球14个国家和地区,境内拥有近340家证券及期货经纪营业网点,境内外拥有近1300万零售及机构和高净值客户。

海通证券积极拥抱金融科技的发展浪潮

伴随着互联网金融的快速发展和普惠金融的深入落地,金融科技能力成为传统金融企业支持未来业务发展、实现企业数字化转型的核心能力。在此背景下,IT逐渐从后台走向前台,IT服务能力和管理水平直接影响着企业核心竞争力的打造。作为中国综合性证券机构的龙头企业,海通证券积极拥抱金融科技的发展浪潮,坚持“统一管理、自主可控、融合业务、引领发展”的科技指导思想,并率先在证券行业启动金融云的建设规划。

稳步建设海通证券金融云:2016年,海通证券发布了《海通证券2016-2020年科技发展规划》,明确了要把“稳步建设海通金融云”作为今后5年科技发展规划的战略任务之一,并采用开源虚拟化和软件定义技术,建立计算、网络、存储资源池,实现基础设施资源共享和自动化管理,为生产、测试、研发环境提供按需服务、按需计量能力;采用容器相关技术,实现生产、测试、研发环境应用构件的快速部署,提高系统可用性和部署效率。

明确定位海通证券金融云:在符合合规性、安全性、可用性的前提下,通过使用开源OpenStack框架,打造具有一定服务能力和金融架构特征的从IaaS到PaaS到SaaS的金融云平台。在此平台基础上,立足海通证券的经纪业务、投资业务、资管业务,发挥海通证券在证券行业龙头地位的优势,在利用技术平台进行业务能力输出的同时加强海通证券金融生态能力的建设,进一步构建上下游业务、资金、风险、信用等信息的分享和交换平台。

大胆描绘海通证券金融云:围绕公司战略发展规划和业务驱动,结合行业发展趋势,海通证券基于开源技术架构,通过开放合作和自主研发相结合的规划目标,制定了“五年五朵云”的金融云蓝图,目前办公云、研发测试云、生产云均已顺利上线并大范围使用。未来将有序开展托管云和行业云的建设工作,从而更好的为子公司、分公司提供高效便捷的服务,进一步释放集团化业务能力。

行业领先的混合金融云架构助力数字化转型

基础设施建设是数字化转型的基础。海通证券率先使用云管理平台(CMP)统一服务入口、开源OpenStack核心技术自主可控、国产硬件SDN网络智能运维、多后端存储资源池按需交付的金融云架构,充分展现了敏捷自动化、稳定高可用、可弹性扩展、可智能运维等一系列技术优势,引领证券行业金融云发展,同时也在多方面进行了创新。

异构纳管、统一编排:为了更好的支撑业务创新,海通证券基础设施资源池一直不断延展,既有大量的物理机资源池,又有基于X86服务器和超融合一体机的VMWare虚拟化资源池;既有多套基于X86服务器和开源分布式架构的OpenStack私有云资源池,又有基于上证信息和深证通的行业云资源池和基于阿里云、腾讯云和华为云等厂商的公有云资源池。海通证券混合金融云平台不但实现了对多种异构资源池的统一纳管,也形成了异构资源池的业务流程编排、IT资源成本分析与管理体系,这不仅仅是混合IT资源纳管的范畴和能力的提升,更是标志着海通混合金融云进入了一个自助交付、智能运营的新阶段。

基于开源、自主可控:为了全面贯彻习总书记提出的“关键核心技术自主可控,把创新主动权、发展主动权牢牢掌握在自己手中”的指导思想,海通证券从一开始就坚持要走开源、开放的技术路线。海通证券基于开源OpenStack的金融云平台不但满足金融云自主可控、稳定安全的诉求,同时其健全庞大的生态体系、分享开放的社区模式也有利于持续赋能技术团队,打造业务和技术兼备的综合型人才。在架构设计上海通证券采用了Multi-Region物理隔离和多可用区域(AZ)逻辑分区相结合的高可用方案,为后续云平台的大规模扩展、“两地三中心”的应用交付打下坚实的基础。

云网联动、融合业务:数字化转型离不开信息技术架构的创新,海通证券金融云新一代云网整体架构引入了国产硬件SDN方案,实现了云网联动,使得网络和安全策略可以随业务自动化下发与配置。SDN网络实现了面向应用的网络编排,基于不同业务组的定义,实现不同业务组间的策略编排。当计算资源发生变更时,网络策略自动迁移,无需人工参与,减少了网络人员变更端口配置的重复劳动,网络人员更可聚焦网络优化和自动化运维的工作。Fabric网络采用Spine-Leaf架构,通过VxLAN技术构建分布式组网架构,可以支持业务灵活扩展,极大增强了网络可靠性和扩展性,实现应用业务间高性能互访、虚拟机灵活迁移、网络资源自动适配。

性能容量、各取所长:随着在金融云环境上的应用场景不断丰富、数据量持续增长,数据来源呈现多样性,业务应用对数据可用性、性能的要求也不尽相同。在海通证券金融云架构中,根据业务需求不同,按资源类型分别提供了开源分布式存储资源池和集中式存储资源池;按资源性能分别提供了“超大容量存储资源池”“混合存储资源池”和“超高性能存储资源池”。通过对不同性能的存储资源进行池化封装、定义标签,并建立放置策略,从而有效的满足了敏态和稳态并存的业务应用,进一步确保了海通金融云的高效运营与服务交付,为用户提供极佳的使用体验。

行稳致远持续创新

海通证券生产云的落地,标志着海通证券拥有足够的能力驾驭云计算相关技术,标志着海通证券金融云进入了一个更具挑战、需承担更多责任的新阶段。着眼未来,海通证券将继续发挥综合性龙头券商的优势,积极探索和实践容器相关技术、应用微服务化架构、SDN网络智能分析与运维等多项创新应用,为实现从“科技支撑业务”到“科技引领业务”的数字化转型奠定基础,从而进一步引领云计算技术在证券行业的应用和发展。

Oracle Cloud03-Oracle Compute Cloud Service快速实践

这是Oracle Cloud系列的第三篇文章,按照第一篇文章中的介绍申请试用账户,就从这一篇开始使用试用账户进行相关的操作和学习,这篇主要讲了:
1.如何创建Oracle Cloud实例;
2.如何进行网络配置;
3.如何进行实例磁盘扩容;
4.如何进行CPU/内存资源扩容;

Oracle Cloud 03-Oracle Compute Cloud Service快速实践

How to build an in-house Docker environment step by step?

主要内容:
1.环境准备;
2.安装并启动Docker;
3.部署私有Registry服务器;
4.安装并配置Rancher;
5.创建一个多容器的应用;
1.环境准备:
1.1 软件版本介绍:
    1.OS:CentOS7.2/3.10.0-327.el7.x86_64(软件);
    2.docker:1.11.2(软件);
    3.rancher/server:v1.1.2(容器);
    4.rancher/agent:v1.0.2(容器);
    5.rancher/agent-instance:v0.8.3(容器);
    6.registry:2.5.0(容器);
    7.mysql:5.7.13(容器);
1.2 服务器准备:
    1.Rancher服务器:192.168.10.160/rancher.htsec.com;用来安装Docker服务,Register服务,Rancher服务,Rancher代理和MySQL服务;
    2.Container1服务器:192.168.10.161/container1.htsec.com;用来安装Docker服务,Rancher代理和创建容器;
    3.Container2服务器:192.168.10.162/container2.htsec.com;用来安装Docker服务,Rancher代理和创建容器;
1.3 服务器配置:
# 分别设置hostname:
hostnamectl set-hostname rancher.htsec.com
hostnamectl set-hostname container1.htsec.com
hostnamectl set-hostname container2.htsec.com
# 分别关闭防火墙和网络管理服务:
systemctl stop firewalld.service
systemctl disable firewalld.service
systemctl stop NetworkManager.service
systemctl disable NetworkManager.service
# 分别配置/etc/hosts:
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.10.160  rancher.htsec.com       rancher
192.168.10.160  docker.htsec.com        docker
192.168.10.160  registry.htsec.com      registry
192.168.10.160  mysql.htsec.com         mysql
192.168.10.161  container1.htsec.com    container1
192.168.10.162  container2.htsec.com    container2
1.4 下载Docker相关包(在可以连公网的机器下载,上传到内网服务器):
wget -S -c -r -np -L https://yum.dockerproject.org/repo/main/centos/7/;
mkdir -p /tools/docker/centos7
mv yum.dockerproject.org/repo/main/centos/7/* /tools/docker/centos7/
1.5 添加/etc/yum.repos.d/docker.repo文件,配置yum源;
[dockerrepo]
name=Docker Repository
baseurl=file:///tools/docker/centos7
enabled=1
gpgcheck=0
1.6 挂载操作系统光盘,添加/etc/yum.repos.d/centos.repo文件,配置yum源:
mkdir -p /tools/centos72
mount -o loop /dev/sr0 /tools/centos72
[centosrepo]
name=CentOS7 Repository
baseurl=file:///tools/centos72/
enabled=1
gpgcheck=0
1.7 生成YUM缓存:
yum clean all
yum makecache
2.分别安装并启动Docker:
2.1 必要条件:Docker必须安装在64位操作系统上,而且内核版本必须大于3.10;(CentOS7的内核版本最小是3.10,使用uname -r命令查看)
2.2 使用yum安装:
yum update -y
yum install -y docker-engine
2.3 如果直接使用rpm安装的话,需要先解决包依赖;
yum install -y libcgroup libtool-ltdl policycoreutils-python
rpm -ivh docker-engine-selinux-1.11.2-1.el7.centos.noarch.rpm
rpm -ivh docker-engine-1.11.2-1.el7.centos.x86_64.rpm
2.4 启动docker;
service docker start 或者 systemctl start docker.service
2.5 配置开机启动;
chkconfig docker on 或者 systemctl enable docker.service
3.部署私有Registry服务器:
3.1 加载registry容器(所以需要的容器都需要在其它环境下载,然后加载):
[root@rancher ~]# docker load < /tools/images/registry2.5.tar
[root@rancher ~]# docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
registry            latest              c6c14b3960bd        5 days ago          33.28 MB
3.2 启动一个Registry,并把数据映射到本地卷:
[root@rancher ~]# mkdir -p /var/lib/registry
[root@rancher ~]# docker run -d -p 5000:5000 –restart=always –name registry -v /var/lib/registry:/var/lib/registry registry:latest
3.3 加载其它镜像:
[root@rancher ~]# docker load < /tools/images/rancherserver_v1.1.2.tar
[root@rancher ~]# docker load < /tools/images/rancheragent_v1.0.2.tar
[root@rancher ~]# docker load < /tools/images/rancheragent-instance_v0.8.3.tar
[root@rancher ~]# docker load < /tools/images/mysql5.7.13.tar
[root@rancher ~]# docker load < /tools/images/wordpress.tar
[root@rancher ~]# docker load < /tools/images/oraclelinux6.8.tar
[root@rancher ~]# docker load < /tools/images/hello-world.tar
[root@rancher ~]# docker images
REPOSITORY               TAG                 IMAGE ID            CREATED             SIZE
wordpress                latest              106a375e769a        46 hours ago        420.5 MB
registry                 latest              c6c14b3960bd        5 days ago          33.28 MB
rancher/server           latest              ffe9c46b500a        11 days ago         842 MB
oraclelinux              6.8                 175adfa05e40        13 days ago         223.1 MB
hello-world              latest              c54a2cc56cbb        4 weeks ago         1.848 kB
rancher/agent-instance   v0.8.3              b6b013f2aa85        7 weeks ago         330.9 MB
rancher/agent            v1.0.2              860ed2b2e8e3        7 weeks ago         454.3 MB
mysql                    latest              1195b21c3a45        7 weeks ago         380.2 MB
3.4 对镜像打标签,指向私有Registry:
[root@rancher ~]# docker tag registry:latest registry.htsec.com:5000/registry:2.5
[root@rancher ~]# docker tag rancher/server:latest registry.htsec.com:5000/rancher/server:v1.1.2
[root@rancher ~]# docker tag rancher/agent:v1.0.2 registry.htsec.com:5000/rancher/agent:v1.0.2
[root@rancher ~]# docker tag rancher/agent-instance:v0.8.3 registry.htsec.com:5000/rancher/agent-instance:v0.8.3
[root@rancher ~]# docker tag mysql:latest registry.htsec.com:5000/mysql:5.7.13
[root@rancher ~]# docker tag wordpress:latest registry.htsec.com:5000/wordpress:4.5.3
[root@rancher ~]# docker tag oraclelinux:6.8 registry.htsec.com:5000/oraclelinux:6.8
[root@rancher ~]# docker tag hello-world:latest registry.htsec.com:5000/hello-world:latest
3.5 把镜像推到私有Registry中:
[root@rancher ~]# docker push registry.htsec.com:5000/registry:2.5
[root@rancher ~]# docker push registry.htsec.com:5000/rancher/server:v1.1.2
[root@rancher ~]# docker push registry.htsec.com:5000/rancher/agent:v1.0.2
[root@rancher ~]# docker push registry.htsec.com:5000/rancher/agent-instance:v0.8.3
[root@rancher ~]# docker push registry.htsec.com:5000/mysql:5.7.13
[root@rancher ~]# docker push registry.htsec.com:5000/wordpress:4.5.3
[root@rancher ~]# docker push registry.htsec.com:5000/oraclelinux:6.8
[root@rancher ~]# docker push registry.htsec.com:5000/hello-world:latest
3.6 从私有Registry中拉取镜像:
[root@rancher ~]# docker pull registry.htsec.com:5000/registry:2.5
[root@rancher ~]# docker pull registry.htsec.com:5000/rancher/server:v1.1.2
[root@rancher ~]# docker pull registry.htsec.com:5000/rancher/agent:v1.0.2
[root@rancher ~]# docker pull registry.htsec.com:5000/rancher/agent-instance:v0.8.3
[root@rancher ~]# docker pull registry.htsec.com:5000/mysql:5.7.13
[root@rancher ~]# docker pull registry.htsec.com:5000/wordpress:4.5.3
[root@rancher ~]# docker pull registry.htsec.com:5000/oraclelinux:6.8
[root@rancher ~]# docker pull registry.htsec.com:5000/hello-world:latest
3.7 将本地的镜像删掉(untag):
[root@rancher ~]# docker rmi registry:latest
[root@rancher ~]# docker rmi rancher/server:latest
[root@rancher ~]# docker rmi rancher/agent:v1.0.2
[root@rancher ~]# docker rmi rancher/agent-instance:v0.8.3
[root@rancher ~]# docker rmi mysql:latest
[root@rancher ~]# docker rmi wordpress:latest
[root@rancher ~]# docker rmi oraclelinux:6.8
[root@rancher ~]# docker rmi hello-world:latest
[root@rancher ~]# docker images
REPOSITORY                                       TAG                 IMAGE ID            CREATED             SIZE
registry.htsec.com:5000/wordpress                4.5.3               106a375e769a        47 hours ago        420.5 MB
registry.htsec.com:5000/registry                 2.5                 c6c14b3960bd        5 days ago          33.28 MB
registry.htsec.com:5000/rancher/server           v1.1.2              ffe9c46b500a        11 days ago         842 MB
registry.htsec.com:5000/oraclelinux              6.8                 175adfa05e40        2 weeks ago         223.1 MB
registry.htsec.com:5000/hello-world              latest              c54a2cc56cbb        4 weeks ago         1.848 kB
registry.htsec.com:5000/rancher/agent-instance   v0.8.3              b6b013f2aa85        7 weeks ago         330.9 MB
registry.htsec.com:5000/rancher/agent            v1.0.2              860ed2b2e8e3        7 weeks ago         454.3 MB
registry.htsec.com:5000/mysql                    5.7.13              1195b21c3a45        7 weeks ago         380.2 MB
3.8 在私有Registry中检索镜像,不能使用docker search命令,需要通过使用v2版本的API:
[root@rancher ~]# docker search registry.htsec.com:5000/rancher/server
Error response from daemon: Unexpected status code 404
[root@rancher ~]# curl http://registry.htsec.com:5000/v2/_catalog
{“repositories”:[“hello-world”,”mysql”,”oraclelinux”,”rancher/agent”,”rancher/agent-instance”,”rancher/server”,”registry”,”wordpress”]}
[root@rancher ~]# curl http://registry.htsec.com:5000/v2/rancher/server/tags/list
{“name”:”rancher/server”,”tags”:[“v1.1.2”]}
4.安装并配置Rancher:
4.1 启动并配置MySQL容器:
# 启动MySQL容器;
[root@rancher ~]# mkdir -p /var/lib/mysql
[root@rancher ~]# docker run -d –restart=always –name mysql -v /var/lib/mysql:/var/lib/mysql -e MYSQL_ROOT_PASSWORD=pwd registry.htsec.com:5000/mysql:5.7.13
# 配置MySQL容器;
[root@rancher ~]# docker exec -it mysql /bin/bash
root@c94ee0fabd33:/# mysql -uroot -p
mysql> CREATE DATABASE IF NOT EXISTS cattle COLLATE = ‘utf8_general_ci’ CHARACTER SET = ‘utf8’;
mysql> GRANT ALL ON cattle.* TO ‘cattle’@’%’ IDENTIFIED BY ‘cattle’;
mysql> GRANT ALL ON cattle.* TO ‘cattle’@’localhost’ IDENTIFIED BY ‘cattle’;
mysql> flush privileges;
mysql> exit
root@c94ee0fabd33:/# exit
4.2 启动rancher/server容器(Rancher的三个组件[rancher/server,rancher/agent,rancher/agent-instance]的版本是有关联的):
[root@rancher ~]# docker run -d –restart=always -p 8080:8080 –name rancher-server –link mysql:mysql \
    -e CATTLE_BOOTSTRAP_REQUIRED_IMAGE=registry.htsec.com:5000/rancher/agent:v1.0.2 \
    -e CATTLE_AGENT_INSTANCE_IMAGE=registry.htsec.com:5000/rancher/agent-instance:v0.8.3 \
    registry.htsec.com:5000/rancher/server:v1.1.2
# 打开浏览器测试;
http://192.168.10.160:8080
4.3 添加hosts:
# 在管理服务器上运行;
[root@rancher ~]# sudo docker run -e CATTLE_AGENT_IP=”192.168.10.160″  -e CATTLE_HOST_LABELS=’role=administrator&id=010505′  -d –privileged -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/rancher:/var/lib/rancher registry.htsec.com:5000/rancher/agent:v1.0.2 http://192.168.10.160:8080/v1/scripts/2D061F6830FEC5A215F3:1470247200000:kz7K8gpchOMXD8poxh3kglKEBsc
# 在容器服务器上运行;
[root@container1 ~]# sudo docker run -e CATTLE_AGENT_IP=”192.168.10.161″  -e CATTLE_HOST_LABELS=’role=owner’  -d –privileged -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/rancher:/var/lib/rancher registry.htsec.com:5000/rancher/agent:v1.0.2 http://192.168.10.160:8080/v1/scripts/2D061F6830FEC5A215F3:1470247200000:kz7K8gpchOMXD8poxh3kglKEBsc
[root@container2 ~]# sudo docker run -e CATTLE_AGENT_IP=”192.168.10.162″  -e CATTLE_HOST_LABELS=’role=owner’  -d –privileged -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/rancher:/var/lib/rancher registry.htsec.com:5000/rancher/agent:v1.0.2 http://192.168.10.160:8080/v1/scripts/2D061F6830FEC5A215F3:1470247200000:kz7K8gpchOMXD8poxh3kglKEBsc
4.4 查看拓扑图:
5.创建一个多容器的应用:
5.1 创建一个Stacks:名叫WordPress;
5.2 添加一个Service(wp-mysql),由一个[registry.htsec.com:5000/mysql:5.7.13]镜像创建的容器组成,需要做端口转换(因为客户端可能在其它的主机上),并指定MYSQL_ROOT_PASSWORD环境变量;
5.2 添加一个Service(wp-app),由两个[registry.htsec.com:5000/wordpress:4.5.3]镜像创建的容器组成,链接到wp-mysql服务,并指定服务别名;
5.3 添加一个Service(wp-lb),由一个负载均衡器容器组成,映射80端口到8000端口,并链接到wp-app服务上;
5.4 访问负载均衡器的8000端口:http://192.168.10.162:8000/;
问题:[root@rancher ~]# docker push registry.htsec.com:5000/registry:2.5
The push refers to a repository [registry.htsec.com:5000/registry]
Get https://registry.htsec.com:5000/v1/_ping: tls: oversized record received with length 20527
解决办法:
1.添加文件:
[root@rancher ~]# vi /etc/sysconfig/docker
DOCKER_OPTS=”–insecure-registry registry.htsec.com:5000″
2.修改文件:
[root@rancher ~]# vi /lib/systemd/system/docker.service
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network.target docker.socket
Requires=docker.socket
[Service]
Type=notify
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
EnvironmentFile=-/etc/sysconfig/docker
ExecStart=/usr/bin/docker daemon $DOCKER_OPTS -H fd://
MountFlags=slave
LimitNOFILE=1048576
LimitNPROC=1048576
LimitCORE=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
[Install]
WantedBy=multi-user.target
3.重启Docker:
[root@rancher ~]# systemctl daemon-reload
[root@rancher ~]# systemctl stop docker.service
[root@rancher ~]# systemctl start docker.service
问题:WARNING: IPv4 forwarding is disabled. Networking will not work.
解决办法:
1.在/usr/lib/sysctl.d/00-system.conf配置文件中添加代码;
net.ipv4.ip_forward=1
2.重启network服务;
systemctl restart network
3.查看是否修改成功;
sysctl net.ipv4.ip_forward

Docker环境安装及配置

1.实验环境:
1.1 虚拟机:VirtualBox 4.3;
1.2 操作系统:CentOS7.2/3.10.0-327.el7.x86_64;
1.3 服务器名:docker.htsec.com;
1.4 IP:192.168.10.100(内网IP)/10.0.2.15(外网IP);
2.下载docker的rpm包,并配置YUM源:
2.1 下载docker的yum包:wget -S -c -r -np -L https://yum.dockerproject.org/repo/main/centos/7/;
2.2 修改目录:
mkdir -p /tools/docker/centos7
mv yum.dockerproject.org/repo/main/centos/7/* /tools/docker/centos7/
2.3 添加/etc/yum.repos.d/docker.repo文件,配置yum源;
[dockerrepo]
name=Docker Repository
baseurl=file:///tools/docker/centos7
enabled=1
gpgcheck=0
3.添加CentOS操作系统的YUM源:
3.1 挂载光盘并把内容拷贝至/tools/centos72/目录;
3.2 添加/etc/yum.repos.d/centos.repo文件,配置yum源;
[centosrepo]
name=CentOS7 Repository
baseurl=file:///tools/centos72/
enabled=1
gpgcheck=0
3.3 创建yum缓存:
yum clean all
yum makecache
4.安装并启动docker:
4.1 必要条件:Docker必须安装在64位操作系统上,而且内核版本必须大于3.10;(CentOS7的内核版本最小是3.10,使用uname -r命令查看)
4.2 使用yum安装:如果服务器可以访问外网,则可以直接配置指向官方的YUM源;n在企业内部一般会构建私有的YUM源;
yum update -y
yum install -y docker-engine
4.3 如果直接使用rpm安装的话,需要先解决包依赖;
yum -y install libcgroup libtool-ltdl policycoreutils-python
rpm -ivh docker-engine-selinux-1.11.2-1.el7.centos.noarch.rpm
rpm -ivh docker-engine-1.11.2-1.el7.centos.x86_64.rpm
4.4 启动docker;
service docker start 或者 /bin/systemctl start  docker.service
4.5 配置开机启动;
chkconfig docker on 或者 systemctl enable docker.service
5.验证docker是否安装成功;
[root@docker ~]# docker run hello-world
Unable to find image ‘hello-world:latest’ locally
latest: Pulling from library/hello-world
c04b14da8d14: Pull complete
Digest: sha256:0256e8a36e2070f7bf2d0b0763dbabdd67798512411de4cdcf9431a1feb60fd9
Status: Downloaded newer image for hello-world:latest
WARNING: IPv4 forwarding is disabled. Networking will not work.
Hello from Docker!
This message shows that your installation appears to be working correctly.
To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the “hello-world” image from the Docker Hub.
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.
To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash
Share images, automate workflows, and more with a free Docker Hub account:
 https://hub.docker.com
For more examples and ideas, visit:
问题:WARNING: IPv4 forwarding is disabled. Networking will not work.
解决办法:
1.在/usr/lib/sysctl.d/00-system.conf配置文件中添加代码;
net.ipv4.ip_forward=1
2.重启network服务;
systemctl restart network
3.查看是否修改成功;
sysctl net.ipv4.ip_forward

构建openstack YUM源(openstack-kilo on centos7)

构建openstack YUM源(openstack-kilo on centos7)
1.对于企业的openstack私有云,出于安全和某些因素的考虑,有些服务器无法访问公网,导致服务器无法更新某些RPM包,同时内部常有openstack新特性开发需求,版本的维护与升级,因此非常有必要构建企业私有的openstack YUM源;
1.1构建openstack YUM源有两个步骤:
    1.同步(下载)官方的源至企业yum服务器中;
    2.重新创建repo并通过nginx(apache)发布;
1.2设置用户名:
hostnamectl set-hostname cloud.htsec.com;
1.3修改/etc/sysconfig/network-scripts/ifcfg-enp0s3文件来设置静态IP:
    TYPE=Ethernet               #网络类型
    DEVICE=enp0s3               #网卡地址代号/子代号
    ONBOOT=yes                  #开机运行
    BOOTPROTO=static            #使用协议(静态)
    IPADDR=192.168.10.200       #IP地址
    NETMASK=255.255.255.0       #子网掩码
    GATEWAY=192.168.10.1        #网关
    HWADDR=00:16:3E:89:85:38    #MAC地址
    BROADCAST=216.18.194.199    #广播地址
    DNS1=8.8.8.8                #域名解析服务器1
    DNS2=8.8.4.4                #域名解析服务器2
1.4手动修改/etc/hosts文件:
192.168.10.200  cloud.htsec.com         cloud
1.5禁用selinux:
setenforce 0
1.6关闭防火墙:
systemctl stop firewalld.service
systemctl disable firewalld.service
systemctl status firewalld.service
1.7关闭网络管理:
service NetManager stop
1.8配置每个节点的公钥访问;
1.9配置NTP服务器;
2.YUM源下载:
2.1安装openstack需要用到8个repo,分两类:
    1.Centos源:CentOS-Base.repo,CentOS-Debuginfo.repo,CentOS-Media.repo,CentOS-Vault.repo;
    2.openstack源及相关依赖源(epel,foreman,puppet):epel.repo,foreman.repo,puppetlabs.repo,rdo-release.repo;
2.2openstack源地址:
    1.所有版本的openstack源:https://repos.fedorapeople.org/repos/openstack/
    2.openstack-kilo版本源:https://repos.fedorapeople.org/repos/openstack/openstack-kilo/
2.3下载openstack-kilo源:
wget -S -c -r -np -L https://repos.fedorapeople.org/repos/openstack/openstack-kilo/
wget -S -c -r -np -L https://repos.fedorapeople.org/repos/openstack/openstack-juno/
2.4下载epel源:
wget -S -c -r -np -L http://mirrors.yun-idc.com/epel/7/
2.5下载puppetlabs源:
wget -S -c -r -np -L https://yum.puppetlabs.com/el/7/
2.6下载foreman源:
wget -S -c -r -np -L http://yum.theforeman.org/plugins/1.10/el7/
wget -S -c -r -np -L http://yum.theforeman.org/releases/1.10/el7/
2.7下载nginx:
wget -S -c -r -np -L http://nginx.org/packages/centos/7/
2.8CentOS的源可以通过国内的镜像下载:
wget -S -c -r -np -L http://mirrors.sohu.com/centos/7
2.9调整目录结构:
2.9.1创建新的目录:
mkdir -p openstack-kilo openstack-juno epel puppetlabs foreman centos nginx;
2.9.2移动文件:
mv repos.fedorapeople.org/repos/openstack/openstack-kilo/* openstack-kilo/;
mv repos.fedorapeople.org/repos/openstack/openstack-juno/* openstack-juno/;
mv mirrors.yun-idc.com/epel/* epel/;
mv yum.puppetlabs.com/el/* puppetlabs/;
mv yum.theforeman.org/plugins foreman/;
mv yum.theforeman.org/releases foreman/;
mv mirrors.sohu.com/centos/* centos/
mv nginx.org/packages/centos/ nginx/
2.9.3删除不需要的软件包和文件:
find ./ -name index.html* -exec rm -rf {} \;
find ./ -name fedora-* -exec rm -rf {} \;
rm -rf repos.fedorapeople.org/
rm -rf mirrors.yun-idc.com/
rm -rf yum.puppetlabs.com/
rm -rf yum.theforeman.org/
rm -rf mirrors.sohu.com/
rm -rf nginx.org/
3.Nginx配置:
3.1安装nginx rpm包:
rpm -ivh /openstack/nginx/centos/7/noarch/RPMS/nginx-release-centos-7-0.el7.ngx.noarch.rpm
3.2修改/etc/yum.repos.d/nginx.repo文件,把yum源指向本地;
[nginx]
name=nginx repo
baseurl=file:///openstack/nginx/centos/7/x86_64
gpgcheck=0
enabled=1
3.3安装nginx:yum install -y nginx;
3.4修改/etc/nginx/conf.d/default.conf文件配置nginx:
server {
    listen       80;
    server_name  cloud.htsec.com;
    #charset koi8-r;
    #access_log  /var/log/nginx/log/host.access.log  main;
    location / {
        #root   /usr/share/nginx/html;
        root   /openstack;
        autoindex on;
        index  index.html index.htm;
    }
    #error_page  404              /404.html;
    # redirect server error pages to the static page /50x.html
    #
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }
    # proxy the PHP scripts to Apache listening on 127.0.0.1:80
    #
    #location ~ \.php$ {
    #    proxy_pass   http://127.0.0.1;
    #}
    # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
    #
    #location ~ \.php$ {
    #    root           html;
    #    fastcgi_pass   127.0.0.1:9000;
    #    fastcgi_index  index.php;
    #    fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;
    #    include        fastcgi_params;
    #}
    # deny access to .htaccess files, if Apache’s document root
    # concurs with nginx’s one
    #
    #location ~ /\.ht {
    #    deny  all;
    #}
}
3.5服务重启:
systemctl enable nginx
systemctl start nginx
systemctl check nginx
4.制作rdo-release.rpm:
4.1下载rdo-release-kilo-1.src.rpm源码包:
[root@cloud ~]# wget http://cloud.htsec.com/openstack-kilo/rdo-release-kilo-1.src.rpm
4.2创建alan用户和组:
useradd alan;
4.3解压rpm包,会在~目录生成一个rpmbuild目录(其中SOURCE目录下存放repo文件,SPECS目录下存放spec文件):
rpm -i rdo-release-kilo-1.src.rpm;
4.4修改并添加repo文件:
vi rdo-release.repo
[openstack-kilo]
name=OpenStack Kilo Repository
baseurl=http://cloud.htsec.com/openstack-kilo/el7/
skip_if_unavailable=0
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-RDO-kilo
vi rdo-testing.repo
[openstack-kilo-testing]
name=OpenStack Kilo Testing
baseurl=http://cloud.htsec.com/openstack-kilo/testing/el7
skip_if_unavailable=0
gpgcheck=0
enabled=0
vi epel.repo
[epel]
name=EPEL Repository
baseurl=http://cloud.htsec.com/epel/7/x86_64
skip_if_unavailable=0
enabled=1
gpgcheck=0
vi foreman.repo
[foreman-releases]
name=Foreman Releases Repository
baseurl=http://cloud.htsec.com/foreman/releases/1.10/el7/x86_64
enabled=1
gpgcheck=0
[foreman-releases-source]
name=Foreman Releases Source Repository
baseurl=http://cloud.htsec.com/foreman/releases/1.10/el7/source
enabled=1
gpgcheck=0
[foreman-plugins]
name=Foreman Plugins Repository
baseurl=http://cloud.htsec.com/foreman/plugins/1.10/el7/x86_64
enabled=1
gpgcheck=0
[foreman-plugins-source]
name=Foreman Plugins Source Repository
baseurl=http://cloud.htsec.com/foreman/plugins/1.10/el7/source
enabled=1
gpgcheck=0
vi puppetlabs.repo
[puppetlabs-products]
name=Puppet Labs Products
baseurl=http://cloud.htsec.com/puppetlabs/7/products/x86_64
enabled=1
gpgcheck=0
[puppetlabs-deps]
name=Puppet Labs Dependencies
baseurl=http://cloud.htsec.com/puppetlabs/7/dependencies/x86_64
enabled=1
gpgcheck=0
[puppetlabs-devel]
name=Puppet Labs Devel
baseurl=http://cloud.htsec.com/puppetlabs/7/devel/x86_64
enabled=1
gpgcheck=0
vi centos.repo
[base]
name=CentOS7 Base Repository
baseurl=http://cloud.htsec.com/centos/7/os/x86_64
enabled=1
gpgcheck=0
[updates]
name=CentOS7 Updates Repository
baseurl=http://cloud.htsec.com/centos/7/updates/x86_64
enabled=1
gpgcheck=0
[extras]
name=CentOS7 Extras Repository
baseurl=http://cloud.htsec.com/centos/7/extras/x86_64/
enabled=1
gpgcheck=0
[centosplus]
name=CentOS7 Plus Repository
baseurl=http://cloud.htsec.com/centos/7/centosplus/x86_64/
enabled=1
gpgcheck=0
4.5修改.spec文件内容:
vi rdo-release.spec
Name:           rdo-release
Version:        kilo
Release:        1
Summary:        RDO repository configuration
Group:          System Environment/Base
License:        Apache2
URL:            https://github.com/redhat-openstack/rdo-release
Source0:        rdo-release.repo
Source2:        rdo-testing.repo
Source1:        RPM-GPG-KEY-RDO-kilo
Source3:        epel.repo
Source4:        foreman.repo
Source5:        puppetlabs.repo
Source6:        centos.repo
Source7:        RPM-GPG-KEY-CentOS-7
BuildArch:      noarch
%description
This package contains the RDO repository
%install
install -p -d %{buildroot}%{_sysconfdir}/yum.repos.d
install -p -m 644 %{SOURCE0} %{buildroot}%{_sysconfdir}/yum.repos.d
install -p -m 644 %{SOURCE2} %{buildroot}%{_sysconfdir}/yum.repos.d
install -p -m 644 %{SOURCE3} %{buildroot}%{_sysconfdir}/yum.repos.d
install -p -m 644 %{SOURCE4} %{buildroot}%{_sysconfdir}/yum.repos.d
install -p -m 644 %{SOURCE5} %{buildroot}%{_sysconfdir}/yum.repos.d
install -p -m 644 %{SOURCE6} %{buildroot}%{_sysconfdir}/yum.repos.d
#GPG Keys
install -p -d %{buildroot}%{_sysconfdir}/pki/rpm-gpg
install -Dpm 644 %{SOURCE1} %{buildroot}%{_sysconfdir}/pki/rpm-gpg
install -Dpm 644 %{SOURCE7} %{buildroot}%{_sysconfdir}/pki/rpm-gpg
%files
%{_sysconfdir}/yum.repos.d/*.repo
%{_sysconfdir}/pki/rpm-gpg/RPM-GPG-KEY-*
%post
# Adjust repos as per dist and version
source /etc/os-release
DIST=$ID
RELEASEVER=$VERSION_ID
if [ “$DIST” != ‘fedora’ ]; then
  DIST=el
  FDIST=el
  # $releasever doesn’t seem to be a reliable way to get the major version on RHEL
  # e.g. if distroverpkg isn’t present in yum.conf mine was set to 6Server
  # because this was the version of the package redhat-release-server-6Server
  RELEASEVER=$(sed -e ‘s/.*release \([0-9]\+\).*/\1/’ /etc/system-release)
else
  FDIST=f
fi
for repo in rdo-release rdo-testing epel foreman puppetlabs centos ; do
  for var in DIST FDIST RELEASEVER; do
    sed -i -e “s/%$var%/$(eval echo \$$var)/g” %{_sysconfdir}/yum.repos.d/$repo.repo
  done
done
4.6重新打包rpm:
[root@cloud ~]# rpmbuild -ba rpmbuild/SPECS/rdo-release.spec
4.7把生成的rpm放到指定的目录:
[root@cloud ~]# mv /root/rpmbuild/RPMS/noarch/rdo-release-kilo-1.noarch.rpm /openstack/openstack-kilo/rdo-release-kilo-1-ht.noarch.rpm
[root@cloud ~]# mv /root/rpmbuild/SRPMS/rdo-release-kilo-1.src.rpm /openstack/openstack-kilo/rdo-release-kilo-1-ht.src.rpm
5.安装openstack-kilo:
wget http://cloud.htsec.com/openstack-kilo/rdo-release-kilo-1-ht.noarch.rpm
rpm -ivh rdo-release-kilo-1-ht.noarch.rpm –replacefiles
yum update -y
yum install -y openstack-packstack
packstack –allinone