Kolla-Ansible 部署 Openstack

Posted by Sunday on 2019-06-24

环境准备

主机名 IP 配置 系统
部署机 192.168.10.20 4G内存 40G 网卡一张 CentOS7
openstack主机 192.168.10.81 8G内存 40G磁盘 网卡两张 CentOS7

openstack-kolla

部署机准备

配置pypi 国内源加速

1
2
3
4
5
mkdir -p ~/.config/pip/
tee ~/.config/pip/pip.conf << 'EOF'
[global]
index-url = https://pypi.doubanio.com/simple
EOF

安装依赖

1
yum install -y vim net-tools git python-devel libffi-devel gcc openssl-devel libselinux-python

安装 pip & ansible

1
2
3
4
5
yum install epel-release
sed -i 's+download.fedoraproject.org/pub+mirrors.ustc.edu.cn+' /etc/yum.repos.d/epel.repo
yum install -y python-pip ansible
pip install --upgrade pip
pip install -U ansible

ansible 配置

1
2
3
4
5
6
vim /etc/ansible/ansible.cfg 

[defaults]
host_key_checking=False
pipelining=True
forks=100

docker 安装

1
2
3
cd /etc/yum.repos.d/
wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
yum install docker-ce -y

开启 Docker 的共享挂载功能

1
2
3
4
5
mkdir /etc/systemd/system/docker.service.d
tee /etc/systemd/system/docker.service.d/kolla.conf << 'EOF'
[Service]
MountFlags=shared
EOF

使用阿里的加速器
登陆阿里云–>控制台–>产品与服务–>容器镜像服务–>镜像加速器 获取加速器地址

1
2
3
4
5
6
mkdir -p /etc/docker
tee /etc/docker/daemon.json <<-'EOF'
{
  "registry-mirrors": ["https://xxxx.mirror.aliyuncs.com"]
}
EOF
1
2
systemctl daemon-reload
systemctl restart docker

Kolla-Ansible

1
2
3
4
5
6
7
8
9
10
git clone https://github.com/openstack/kolla
git clone https://github.com/openstack/kolla-ansible

pip install -r kolla/requirements.txt
pip install -r kolla-ansible/requirements.txt

pip install --ignore-installed requests
sudo pip install --ignore-installed PyYAML

cd kolla-ansible && python setup.py install

将globals.yml和passwords.yml复制到/etc/kolla目录

1
2
mkdir -p /etc/kolla
cp -r kolla-ansible/etc/kolla/* /etc/kolla

将multinode和multinode库存文件复制到当前目录

1
cp kolla-ansible/ansible/inventory/* .

部署中使用的密码存储在/etc/kolla/passwords.yml文件中。
修改keystone_admin_password密码（OpenStack登录密码）

1
2
3
4
kolla-genpwd

vim /etc/kolla/passwords.yml
keystone_admin_password: sunday

kolla配置

1
2
3
4
5
6
7
8
9
10
11
12
vim /etc/kolla/globals.yml

kolla_base_distro: "centos"
kolla_install_type: "source"  
openstack_release: "stein"
network_interface: "eth0" # OpenStack使用的网络接口
neutron_external_interface: "eth1"
#eth1桥接模式，ip其实是dhcp分配，这个其实是让neutron的br-ex 绑定使用，虚拟机是通过这块网卡访问外网
# 第二个接口专用于Neutron外部（或公共）网络，可以是vlan或flat，取决于网络的创建方式。
# 这个接口应该是活动的，没有IP地址。否则，实例将无法访问外部网络
kolla_internal_vip_address: "10.1.1.10"  # VIP 与network_interface同网段且未被使用的IP
docker_new_yum_url: "https://mirrors.aliyun.com/docker-ce/linux/{{ ansible_distribution | lower }}"

拉取镜像

1
kolla-ansible pull -vvv

这里可以将镜像pull下来后上传至内网，然后修改vim /etc/kolla/globals.yml 中docker_registry: “192.168.10.20:4000” 方便扩展

将镜像上传至内网

启动容器

1
2
mkdir -p /var/www/html/registry
docker run -d -p 4000:5000 -v /var/www/html/registry:/var/lib/registry --restart=always --name registry registry:2.6.2

修改docker配置,192.168.10.20是eth0的ip

1
2
3
4
5
vim /etc/docker/daemon.json
{
  "registry-mirrors": ["https://mb8n4btz.mirror.aliyuncs.com"],
  "insecure-registries":["192.168.10.20:4000"]
}

修改镜像tag

1
for i in `docker images|grep -v registry|grep -v R|awk '{print $1}'`;do docker image tag $i:stein 192.168.10.20:4000/$i:stein;done

上传镜像到私有镜像仓库

1
for i in `docker images|grep 192.168.10.20|awk '{print $1}'`;do docker push $i:stein;done

查看镜像是否上传成功

1
curl -XGET http://192.168.10.20:4000/v2/_catalog

备份镜像文件

1
tar -zcvf kolla-openstack-stein-registry.tar.gz /var/www/html/registry

Openstack 部署

修改multinode

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
vim multinode

192.168.10.81 ansible_user=root ansible_password=sunday ansible_become=true
192.168.10.82 ansible_user=root ansible_password=sunday ansible_become=true

# The above can also be specified as follows:
#control[01:03]     ansible_user=kolla

# The network nodes are where your l3-agent and loadbalancers will run
# This can be the same as a host in the control group
[network]
192.168.10.81 ansible_user=root ansible_password=sunday ansible_become=true
192.168.10.82 ansible_user=root ansible_password=sunday ansible_become=true

[compute]
192.168.10.83 ansible_user=root ansible_password=sunday ansible_become=true

[monitoring]
192.168.10.83 ansible_user=root ansible_password=sunday ansible_become=true

# When compute nodes and control nodes use different interfaces,
# you need to comment out "api_interface" and other interfaces from the globals.yml
# and specify like below:
#compute01 neutron_external_interface=eth0 api_interface=em1 storage_interface=em1 tunnel_interface=em1

[storage]
192.168.10.83 ansible_user=root ansible_password=sunday ansible_become=true

[deployment]
localhost       ansible_connection=local
...
#下面还有很多

对主机安装依赖，如docker等

1
2
# -vvv 可以打印出最详细的信息
kolla-ansible -i ./multinode bootstrap-servers

对主机执行预部署检查

1
kolla-ansible -i ./multinode prechecks

执行OpenStack部署

1
kolla-ansible -i ./multinode deploy

OpenStack 使用

OpenStack需要一个openrc文件，其中设置了admin用户的凭证。
要生成这个文件运行以下命令

1
2
kolla-ansible post-deploy
. /etc/kolla/admin-openrc.sh

安装基本的OpenStack CLI客户端

1
pip install --ignore-installed python-openstackclient python-glanceclient python-neutronclient

运行脚本创建示例网络，图像等

1
. /usr/share/kolla-ansible/init-runonce

浏览器访问openstack dashboard

1
2
3
IP 192.168.8.88
用户名 admin
密码 sunday

Kolla 构建镜像
上面的镜像到docker hub pull下来。这里是build

1
2
3
4
5
6
cd kolla
git checkout stable/stein
git pull
pip install ./
pip show kolla
kolla-build -b centos -t binary -p default

报错解决

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
如果出现requests 2.20.0 has requirement idna<2.8,>=2.5, but you'll have idna 2.4 which is 
incompatible.错误，则强制更新requets库
pip install --ignore-installed requests

同样，出现Cannot uninstall 'PyYAML'. It is a distutils installed project and thus we cannot 
accurately determine which files belong to it which would lead to only a partial uninstall.错误，强制更新
sudo pip install --ignore-installed PyYAML

Cannot uninstall 'ipaddress'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
此类问题强制升级此安装包即可。(已发现PyYAML,ipaddress)
pip install --ignore-installed ipaddress

ImportError: cannot import name decorate
pip install -U decorator

问题：kolla "Waiting for virtual IP to appear"
/etc/kolla/globals.yml
keepalived_virtual_router_id: "250"

网络连接
iptables -A INPUT -i br-ex -j ACCEPT #注意内核转发问题（INPUT都需要允许才行）
iptables -t nat -A POSTROUTING -s 10.0.2.0/255.255.255.0  -j SNAT --to-source 10.8.250.57
ifconfig br-ex 10.0.2.1/24

由于错误的出现，可能需要多次的部署，而有些错误重新部署是不会进行修正的，所以需要将整个环境进行清理:
    /usr/share/kolla-ansible/tools/cleanup-containers                #可用于从系统中移除部署的容器
    /usr/share/kolla-ansible/tools/cleanup-host                      #可用于移除由于残余网络变化引发的docker启动的neutron-agents主机
    /usr/share/kolla-ansible/tools/cleanup-images                    #可用于从本地缓存中移除所有的doc

https://www.jianshu.com/p/5d58f6f9e2c7
https://www.jianshu.com/p/c549a512c224
https://docs.openstack.org/project-deploy-guide/kolla-ansible/ocata/quickstart.html
https://xiexianbin.cn/openstack/kolla/2016-10-23-use-kolla-to-deploy-openstack-multinode-env/
云计算底层技术-使用openvswitch

CATALOG
    FEATURED TAGS
    FRIENDS