Aug 8, 2018

How to configure Contrail/TungstenFabric to control EVPN/VXLAN on QFX51x0

Contrail/TungstenFabric has supported EVPN/VXLAN since 4.1.
**OVSDB/VXLAN had been deprecated on 5.0 due to scaling and stability issue.

Automated configuration for EVPN/VXLAN is only supported Juniper QFX51x0 series on 5.0.
**5.1 will support multi vender configuration.

How to Configure

in this article, Juniper QFX5100 is used.

1. Register Leaf Switch

Configure > Physical Devices > Physical Routers
Select "Netconf Managed Physical Router" from "+".
Name: Host name of Leaf Switch (HVTEP)
Vender: Juniper
Model: qfx5100
Management IP: IP address of Netconf/SSH connection.
Netconf Username: User name to accept configuration by Netconf.
Netconf Password: User password to accept configuration by Netconf.
Role: Leaf
EVPN Peered TOR: Enable check
JUNOS Service Port: Listen port of Netconf on Leaf Switch

2. Configure BGP Router

Configure > Infrastructure > BGP Routers
Router Type: BGP Router
Name: Hostname of Leaf Switch (HVTEP)
Vender ID: Juniper
IP Address: Source IP address of BGP as same as Loopback.
Router ID: Router ID of BGP. Same as IP Address
Autonomous System: AS number of Leaf Switch
BGP Router ASN: AS number which is used BGP connection of BGP Peer by Leaf Switch. Same as Autonomous System.
Address Families: Configure only inet-vpn, route-target, e-vpn
Advanced Options > Physical Router: Select Leaf Switch which is configured at "Physical Device"
Assosiate Peer(s) > Peer: Select Control node

3. Configure VTEP

Configure > Physical Devices > Physical Routers
Configure VTEP information on configured Physical Router. Click right side "Gear" icon.
VTEP Address: Configure Loopback IP address
Loopback IP: Configure Loopback IP address


4. Confirm Leaf Switch

Configuration is seen under "group __contrail__" which is configured by Contrail/TungstenFabric.
"show configuration groups __contrail__" shows loopback, BGP configuration and so on.
interfaces {
    /* Interfaces Configuration */
    lo0 {
        /* Router Loopback Interface */
        unit 0 {
            family inet {
                address 10.84.54.2/32 {
                    primary;
                    preferred;
                }
            }
        }
    }
}
routing-options {
    /* Global Routing Options */
    router-id 10.84.54.2;
    route-distinguisher-id 10.84.54.2;
    autonomous-system 64519;
    resolution {
        rib bgp.rtarget.0 {
            resolution-ribs inet.0;
        }
    }
}
protocols {
    /* Protocols Configuration */
    bgp {
        /* BGP Router: qfx5100-48t-1, UUID: 5e96224d-f708-4207-9ac3-b15a2699494e */
        group _contrail_asn-64519 {
            type internal;
            local-address 10.84.54.2;
            hold-time 90;
            family evpn {
                signaling;
            }
            family route-target;
        }
        /* BGP Router: qfx5100-48t-1, UUID: 5e96224d-f708-4207-9ac3-b15a2699494e */
        group _contrail_asn-64519-external {
            type external;
            multihop;
            local-address 10.84.54.2;
            hold-time 90;
            family evpn {
                signaling;
            }
            family route-target;
            /* BGP Router: overcloud-contrailcontroller-0, UUID: 72950843-4008-4e6f-8019-0d875e06dcd6 */
            neighbor 10.84.50.91 {
                peer-as 65534;
            }
        }
    }
}
policy-options {
    community _contrail_switch_policy_ members target:64519:1;
}
switch-options {
    vtep-source-interface lo0.0;
}

5. Configure Virtual network on Leaf Switch

Configure > Physical Devices > Interfaces > {Leaf Swtich}
Name: Configure Interface. Set Physical interface with sub-interface like "xe-0/0/1.0".
Logical Interface Properties > Logical Interface Type: Server
Logical Interface Properties > VLAN ID: Configure VLAN nmber. In case of Untag, set "0"
Logical Interface Properties > Virtual Network: Select Virtual network
Logical Interface Properties > Server MAC: Configure MAC address to asign IP address to Bare metal server by DHCP. Dummy MAC address is acceptable.
** TSN is required to assign IP address by DHCP.
Logical Interface Properties > IP address: IP address of DHCP. Blank is acceptable.

6. Confirm Leaf Switch

After configuration, interface is automatically configured. Necessary configuration is also configured.
__contrail__ {
    interfaces {
        /* Interfaces Configuration */
        lo0 {
            /* Router Loopback Interface */
            unit 0 {
                family inet {
                    address 10.84.54.2/32 {
                        primary;
                        preferred;
                    }
                }
            }
        }
        xe-0/0/1 {
            flexible-vlan-tagging;
            native-vlan-id 4094;
            encapsulation extended-vlan-bridge;
            /* L2 EVPN Untagged Interface, Virtual Network: vxlan-vn1, UUID: 1e1009e1-8113-4e62-b06f-24a3f5467b3f */
            unit 0 {
                vlan-id 4094;
            }
        }
    }
    routing-options {
        /* Global Routing Options */
        router-id 10.84.54.2;
        route-distinguisher-id 10.84.54.2;
        autonomous-system 64519;
        resolution {
            rib bgp.rtarget.0 {
                resolution-ribs inet.0;
            }
        }
    }
    protocols {
        /* Protocols Configuration */
        bgp {
            /* BGP Router: qfx5100-48t-1, UUID: 5e96224d-f708-4207-9ac3-b15a2699494e */
            group _contrail_asn-64519 {
                type internal;
                local-address 10.84.54.2;
                hold-time 90;
                family evpn {
                    signaling;
                }
                family route-target;
            }
            /* BGP Router: qfx5100-48t-1, UUID: 5e96224d-f708-4207-9ac3-b15a2699494e */
            group _contrail_asn-64519-external {
                type external;
                multihop;
                local-address 10.84.54.2;
                hold-time 90;
                family evpn {
                    signaling;
                }
                family route-target;
                /* BGP Router: overcloud-contrailcontroller-0, UUID: 72950843-4008-4e6f-8019-0d875e06dcd6 */
                neighbor 10.84.50.91 {
                    peer-as 65534;
                }
            }
        }
        evpn {
            vni-options {
                vni 5 {
                    vrf-target target:64512:100000;
                }
            }
            encapsulation vxlan;
            multicast-mode ingress-replication;
            extended-vni-list all;
        }
    }
    policy-options {
        /* Policy Options */
        /* Virtual Network: vxlan-vn1, UUID: 1e1009e1-8113-4e62-b06f-24a3f5467b3f, Route Targets Type: Import */
        policy-statement _contrail_vxlan-vn1-l2-5-import {
            term _contrail_switch_policy_ {
                from community _contrail_switch_policy_;
                then accept;
            }
            term t1 {
                from community [ _contrail_target_65534_8000003 _contrail_target_64512_100000 ];
                then accept;
            }
        }
        /* L2 Switch Global Export Policy */
        policy-statement _contrail_switch_export_policy_ {
            term t1 {
                then {
                    community add _contrail_switch_export_community_;
                }
            }
        }
        community _contrail_switch_export_community_ members [ target:65534:8000003 target:64512:100000 ];
        community _contrail_target_65534_8000003 members target:65534:8000003;
        community _contrail_target_64512_100000 members target:64512:100000;
        community _contrail_switch_policy_ members target:64519:1;
    }
    switch-options {
        vtep-source-interface lo0.0;
        route-distinguisher 10.84.54.2:1;
        vrf-import _contrail_vxlan-vn1-l2-5-import;
        vrf-export _contrail_switch_export_policy_;
        vrf-target {
            target:64519:1;
            auto;
        }
    }
    vlans {
        contrail_vxlan-vn1-l2-5 {
            interface xe-0/0/1.0;
            vxlan {
                vni 5;
            }
        }
    }
}

7. Confirm table on Leaf Switch

QFX shows MAC address and Remote VTEP infomation which are advertised by either Controller or other BGP Peer.
show ethernet-switching table 
MAC flags (S - static MAC, D - dynamic MAC, L - locally learned, P - Persistent static
           SE - statistics enabled, NM - non configured MAC, R - remote PE MAC, O - ovsdb MAC)


Ethernet switching table : 5 entries, 5 learned
Routing instance : default-switch
   Vlan                MAC                 MAC      Logical                Active
   name                address             flags    interface              source
   contrail_vxlan-vn1-l2-5 00:00:5e:00:01:01 DR     esi.1736               05:00:00:fc:00:00:00:00:05:00
   contrail_vxlan-vn1-l2-5 02:ed:68:05:27:94 D      vtep.32769             10.84.50.94
   contrail_vxlan-vn1-l2-5 08:81:f4:89:72:e0 D      vtep.32771             10.84.54.1
   contrail_vxlan-vn1-l2-5 10:0e:7e:dd:8f:43 D      vtep.32770             10.84.54.4
   contrail_vxlan-vn1-l2-5 10:0e:7e:dd:8f:44 D      xe-0/0/1.0
show route table bgp.evpn.0
   bgp.evpn.0: 12 destinations, 12 routes (12 active, 0 holddown, 0 hidden)
   + = Active Route, - = Last Active, * = Both

   1:10.84.54.1:0::050000fc000000000500::FFFF:FFFF/192 AD/ESI
                      *[BGP/170] 00:03:34, localpref 100, from 10.84.50.91
                         AS path: 65534 64512 I, validation-state: unverified
                       > to 10.84.53.13 via xe-0/0/46.0
   2:10.84.50.94:20::5::02:ed:68:05:27:94/304 MAC/IP
                      *[BGP/170] 00:03:34, MED 100, localpref 100, from 10.84.50.91
                         AS path: 65534 ?, validation-state: unverified
                       > to 10.84.53.13 via xe-0/0/46.0
   2:10.84.54.1:7::5::00:00:5e:00:01:01/304 MAC/IP
                      *[BGP/170] 00:03:34, localpref 100, from 10.84.50.91
                         AS path: 65534 64512 I, validation-state: unverified
                       > to 10.84.53.13 via xe-0/0/46.0
   2:10.84.54.1:7::5::08:81:f4:89:72:e0/304 MAC/IP
                      *[BGP/170] 00:03:34, localpref 100, from 10.84.50.91
                         AS path: 65534 64512 I, validation-state: unverified
                       > to 10.84.53.13 via xe-0/0/46.0
   2:10.84.54.4:1::5::10:0e:7e:dd:8f:43/304 MAC/IP
                      *[BGP/170] 00:03:34, localpref 100, from 10.84.50.91
                         AS path: 65534 64520 I, validation-state: unverified
                       > to 10.84.53.13 via xe-0/0/46.0
   2:10.84.50.94:20::5::02:ed:68:05:27:94::10.0.0.6/304 MAC/IP
                      *[BGP/170] 00:03:34, MED 100, localpref 100, from 10.84.50.91
                         AS path: 65534 ?, validation-state: unverified
                       > to 10.84.53.13 via xe-0/0/46.0
   2:10.84.54.1:7::5::00:00:5e:00:01:01::10.0.0.1/304 MAC/IP
                      *[BGP/170] 00:03:34, localpref 100, from 10.84.50.91
                         AS path: 65534 64512 I, validation-state: unverified
                       > to 10.84.53.13 via xe-0/0/46.0
   2:10.84.54.1:7::5::08:81:f4:89:72:e0::10.0.0.14/304 MAC/IP
                      *[BGP/170] 00:03:34, localpref 100, from 10.84.50.91
                         AS path: 65534 64512 I, validation-state: unverified
                       > to 10.84.53.13 via xe-0/0/46.0
   3:10.84.50.94:20::5::10.84.50.94/248 IM
                      *[BGP/170] 00:03:34, MED 200, localpref 100, from 10.84.50.91
                         AS path: 65534 ?, validation-state: unverified
                       > to 10.84.53.13 via xe-0/0/46.0
   3:10.84.54.1:7::5::10.84.54.1/248 IM
                      *[BGP/170] 00:03:34, localpref 100, from 10.84.50.91
                         AS path: 65534 64512 I, validation-state: unverified
                       > to 10.84.53.13 via xe-0/0/46.0
   3:10.84.54.2:1::5::10.84.54.2/248 IM
                      *[EVPN/170] 00:03:33
                         Indirect
   3:10.84.54.4:1::5::10.84.54.4/248 IM
                      *[BGP/170] 00:03:34, localpref 100, from 10.84.50.91
                         AS path: 65534 64520 I, validation-state: unverified
                       > to 10.84.53.13 via xe-0/0/46.0



Nov 15, 2017

How to Connect existing Network without BGP gateway by Simple Gateway

Previous post described how to deploy Openstack Ocata with OpenContrail 4.0. At that setup, VM can connect other VM, but VM cannot connect existing network such as Internet.
Basically, OpenContrail requires Gateway router which can speak MPBGP and MPLSoGRE to communicate other networks. (i.e. Juniper MX router or Cisco ASR router)
If you don't have such router, need to use "Simple Gateway" that uses Contrail vRouter as a gateway.

Simple Gateway works like a Network Node of Neutron.
Simple Gateway is created in Compute node and it works together with Contrail vRouter. It takes over overlay Tunnel and Physical NIC.


In case of VM wants to connect Internet, Simple Gateway terminates VM packet, then Simple Gateway routes its pakcets to Compute node kernel. After that, the packets look up routing table on Compute node. Eventually the packets reach to Internet via GW.

How to setup Simple Gateway

  1. Create Virtual-network for Public (For External network connection)
  2. Create Simple Gateway
  3. Create Virtual-network for Private (For Virtual-machine)
  4. Spawn Virtual-machine
  5. set Floating IP to Virtual-machine
  6. Verify connectivity

1. Create Virtual-network for Public (For External network connection)

Possible to create either Openstack or OpenContrail.

In case of Openstack

Create
source openstackrc
openstack network create public --external
openstack subnet create --network public --subnet-range 203.0.113.0/24 public_subnet
Verify
openstack network list --external
+--------------------------------------+--------+--------------------------------------+
| ID                                   | Name   | Subnets                              |
+--------------------------------------+--------+--------------------------------------+
| 08960915-ef83-4980-ae3d-08aea937d4fb | public | bea3e1c9-4658-4b17-bfb6-9ee8047d04c6 |
+--------------------------------------+--------+--------------------------------------+

In case of Contrail

Open Configure > Networking >Networks, Click "+" then set below items and Save
Name: Name of Virtual-network (i.e. public)
Subnet:Network address in "CIDR" (i.e.203.0.113.0/24)
Adbanced Options: Check "External"

2.Create Simple Gateway

Login target node to create SimpleGateway due to Simple Gateway is created on Compute node.
Execute command below after login.
docker exec agent python /opt/contrail/utils/provision_vgw_interface.py --oper create --interface vgw1 --subnets 203.0.113.0/24 --routes 0.0.0.0/0 --vrf default-domain:admin:public:public
Arguments

  • --oper: "create" or "delete"
  • --interface: Name of Simple Gateway. In case of multiple simple gateways are created, set unique name to each gateway.
  • --subnets: Define subnet address of Simple Gateway using. Define subnet address what you want to expose to external.
  • --routes: Define destination network that Simple Gateway allows to connect. Define 0.0.0.0/0, in case of unspecified network such as Internet. Define particular network, such ad 10.0.0.0/24, in case of specified network. Multiple networks are allowed.
  • --vrf: Define name of VRF that Contrail uses. Syntax is "default-domain:<project name>:<virtual network name>:<virtual-network name>
Network will be seen by "route -n" after creation.
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.84.50.100    0.0.0.0         UG    0      0        0 vhost0
10.84.50.0      0.0.0.0         255.255.255.0   U     0      0        0 vhost0
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0
203.0.113.0     0.0.0.0         255.255.255.0   U     0      0        0 vgw1
If GW or other nodes need to reach 203.0.113.0/24, add routing table that nexthop address is vhost0 of Compute node.
i.e.)kvm host needs to reach 203.0.113.0/24, below configuration is required.
also, KVM Host needs to connect external (internet), Configure NAT if necessary.
route add -net 203.0.113.0 netmask 255.255.255.0 gw 10.84.50.119
iptables -A POSTROUTING -t nat -o em1 -s 203.0.113.0/24 -j MASQUERADE

3.Create Virtual-network for Private (For Virtual-machine)

Possible to create either Openstack or OpenContrail.

In case of Openstack

Create
source openstackrc
openstack network create user-VN1
openstack subnet create --network user-VN1 --subnet-range 192.168.0.0/24 user-VN1_subnet
Verify
openstack network list --internal
+--------------------------------------+----------+--------------------------------------+
| ID                                   | Name     | Subnets                              |
+--------------------------------------+----------+--------------------------------------+
| 7f26e637-e4d2-4ce1-9afc-572837a096d4 | user-VN1 | 019c0a3c-b4a3-492a-8832-633ceffc07a3 |
+--------------------------------------+----------+--------------------------------------+

In case of Contrail

Open "Configure > Networking >Networks" Click "+", set blow items and Save
Name: Name of Virtual-network
Subnet:Network address in "CIDR"  (i.e.192.168.0.0/24)

4.Spawn Virtual-machine

Create Virtual-machine on "user-VN1" by either Horizon or command
source openstackrc
nova boot --flavor m1.tiny --image cirros --nic net-id=7f26e637-e4d2-4ce1-9afc-572837a096d4 VM1 

5.set Floating IP to Virtual-machine

Possible to create either Openstack or OpenContrail.

In case of Openstack

Create
source openstackrc
openstack floating ip create public
openstack server add floating ip VM1 203.0.113.3
Verify
openstack floating ip list
+--------------------------------------+---------------------+------------------+--------------------------------------+--------------------------------------+----------------------------------+
| ID                                   | Floating IP Address | Fixed IP Address | Port                                 | Floating Network                     | Project                          |
+--------------------------------------+---------------------+------------------+--------------------------------------+--------------------------------------+----------------------------------+
| ef192165-151f-42f9-b58b-6215e6f92ba1 | 203.0.113.3         | 192.168.0.3      | a3b9d5a8-f33a-48be-92a3-d3b3508f9503 | 08960915-ef83-4980-ae3d-08aea937d4fb | 31ecaa5de0ea4ba783b8e267e6249d79 |
+--------------------------------------+---------------------+------------------+--------------------------------------+--------------------------------------+----------------------------------+

In case of Contrail

Open "Configure > Networking >Floating IPs", Click "+", set below items and Save
Floating IP Pool: admin:public:default (203.0.113.0/24)
Click right side Icon and Select "Associate Port" after creating Floating IP. Select target IP address of VM and Save

6.Verify connectivity

Verify connectivity from Compute node
if failed, confirm below
  • Subnet, route configuration of Simple gateway
  • Permit External connection by Security group, if from External Connection if failed.
  • Routing teble of KVM host and GW
  • NAT is required.
To isolate issues, "tcpdump" on vhost0 of Compute node or KVM host then confirm where the packets are missing.
Fro more help, visit Slack channel of OpenContrail.

Oct 16, 2017

How to install Kolla Openstack Ocata and Open Contrail4.0

This is English version Japanese is here.

This article describe how to install Openstack ocata and OpenContrail 4.0 which are uploaded at Dockerhub.

There are 4 Ubuntu 16.04 Virtual machines spawned KVM and each roles are below. Spawned Virtual machines must have more than 2 NICs. (Openstack kolla requirements)
All of Installation and provisioning are don by kolla1.
** In case of KVM, must enable "nested" on KVM host and VMX attribute on Virtual machine.
Servers:
kolla1(10.84.50.117): Openstack Controller/Ansible
kolla2(10.84.50.118): Contrail Controller
kolla3(10.84.50.119): Compute node
kolla4(10.84.50.120): Compute node
Procedure:
  1. Preparation
  2. Install/Download Ansible and Playbook
  3. Install
  4. Configuration after Installation
  5. Operation check

1.Preparation

Create CA and register target servers because of Kolla1 connects to all of nodes as Root.
Execute Kolla1
ssh-keygen -t rsa 
Execute command below to all of nodes.
ssh-copy-id -i ~/.ssh/id_rsa.pub lab@10.84.50.117
ssh -t lab@10.84.50.117 'sudo mkdir /root/.ssh'
ssh -t lab@10.84.50.117 'sudo cp /home/lab/.ssh/authorized_keys /root/.ssh/authorized_keys' 

Preparation to use Ansible
Execute Kolla1
apt update
apt install python-pip sshpass
pip install -U pip
pip install -U ansible
pip install  pyOpenSSL==16.2.0
Execute command below to all of nodes.
ssh -t root@10.84.50.117 'apt-get -y install python-simplejson'

2. Download and Configure Ansible Playbook

Download install file from github.
git clone https://github.com/gokulpch/OpenContrail-Kolla.git
Two directories are available, contrail-ansible for OpenContrial and kolla-ansible for Opentack.

Modify Configuration file of Openstack

Modify OpenContrail-Kolla/kolla-ansible/etc/kolla/globals.yml

Modifiy 4 items below;
network_interface: "ens3"
kolla_internal_vip_address: "10.84.50.117"
contrail_api_interface_address: "10.84.50.118"
neutron_external_interface: "ens4"
  • network_interface: Configure VM NIC which connects Underlay network
  • kolla_internal_vip_address: Configure IP address of Openstack Controller
  • contrail_api_interface_address: Configure IP address of OpenContrail Config node
  • neutron_external_interface: Configure NIC as Fake NIC

Modify OpenContrail-Kolla/kolla-ansible/etc/kolla/passwords.yml

The file is configured password of each components.(default is contrail1)
If you need to modify password, use this file.

Modify OpenContrail-Kolla/kolla-ansible/ansible/inventory/multinode

Modify [control] [network] [compute] [monitoring] [storage] items. An example below uses hostname. In case of using host name, must able to resolve IP address from hostname by /etc/hosts and so on.
[control]
kolla1
[network]
kolla1
[compute]
kolla3
kolla4
[monitoring]
kolla1
[storage]
kolla1

Configure configuration file of OpenContrail

Modify OpenContrail-Kolla/contrail-ansible/playbooks/inventory/my-inventory/hosts

Modify [contrail-controllers] [contrail-analytics] [contrail-compute] [openstack-controllers] items. Must use IP address instead of host name otherwise OpenContrail doesn't work correctly.
[contrail-controllers]
10.84.50.118
[contrail-analyticsdb]
10.84.50.118
[contrail-analytics]
10.84.50.118
[contrail-compute]
10.84.50.119
10.84.50.120
[openstack-controllers]
10.84.50.117

Modify OpenContrail-Kolla/contrail-ansible/playbooks/inventory/my-inventory/group_vars/all.yml 

Modify 5 items below;
ansible_ssh_private_key_file: ~/.ssh/id_rsa
global_config: { external_rabbitmq_servers: 10.84.50.117 }
rabbitmq_config: { user: openstack, password: contrail1 }
keystone_config: {ip: 10.84.50.117, admin_password: contrail1, auth_protocol: http}
vrouter_physical_interface: ens3
  • ansible_ssh_private_key_file: Configure CA for login (uncomment this item. remove "#")
  • global_config: Configure IP address of RabbitMQ
  • rabbitmq_config: Configure password of RabbitMQ. Must modify if the password is modified at passwords.yml.
  • keystone_config: Configure IP address and password of Keysone. Must modify if the password is modified at passwords.yml.
  • vrouter_physical_interface: Configure NIC of vRouter of OpenContrailの. If NIC is different of each Compute nodes, configure NIC at hosts file.

3.Install

Preparation for installation of Openstack

Install dependency file and export variable to ignore ssh key verification.
ssh -t root@10.84.50.117 'apt-get -y install python-oslo-config'
export ANSIBLE_HOST_KEY_CHECKING=False

Install Openstack

cd OpenContrail-Kolla/kolla-ansible/ansible/
ansible-playbook -i inventory/multinode -e @../etc/kolla/globals.yml -e @../etc/kolla/passwords.yml -e action=bootstrap-servers kolla-host.yml
ansible-playbook -i inventory/multinode -e @../etc/kolla/globals.yml -e @../etc/kolla/passwords.yml -e action=deploy site.yml

Preparation for installation of OpenContrail

Install Docker
ssh -t root@10.84.50.118 'curl -sSL https://get.docker.io | bash'

Install OpenContrail

cd OpenContrail-Kolla/contrail-ansible/playbooks/
ansible -i inventory/my-inventory -m shell -a 'apt-get install -y ntp' all
ansible-playbook  -i inventory/my-inventory site.yml

Reboot compute nodes

ssh -t root@10.84.50.119 reboot
ssh -t root@10.84.50.120 reboot

4.Configuration after installation

Fix a bug of horizon

docker exec -i horizon sudo sed -i -e 's:/usr/share/openstack-dashboard/static:/var/lib/openstack-dashboard/static:g' /etc/apache2/conf-enabled/000-default.conf
docker exec -i horizon /usr/share/openstack-dashboard/manage.py collectstatic --noinput
docker exec -i horizon /usr/share/openstack-dashboard/manage.py compress
docker exec -i horizon sudo service apache2 reload

Create openstackrc

echo 'export OS_USERNAME=admin
export OS_PASSWORD=contrail1
export OS_PROJECT_NAME=admin
export OS_USER_DOMAIN_NAME=Default
export OS_PROJECT_DOMAIN_NAME=Default
export OS_AUTH_URL=http://10.84.50.117:35357/v3
export OS_IDENTITY_API_VERSION=3
export OS_IMAGE_API_VERSION=2' > openstackrc

Install Openstack client

pip install python-openstackclient

Create flavor

source openstackrc
openstack flavor create --id 1 --disk 1 --ram 512 --public m1.tiny
openstack flavor create --id 2 --disk 20 --ram 2048 --public m1.small
openstack flavor create --id 3 --disk 40 --ram 4096 --public m1.medium
openstack flavor create --id 4 --disk 80 --ram 8192 --public m1.large
openstack flavor create --id 5 --disk 160 --ram 16384 --public m1.xlarge

5.Operation Check

Execute openstack network list. If 3 network date is returned, finish to install/provision Openstack and OpenContrail.
+--------------------------------------+-------------------------+--------------------------------------+
| ID                                   | Name                    | Subnets                              |
+--------------------------------------+-------------------------+--------------------------------------+
| a5432e6b-0155-4b13-abe8-e6290f6dc9fe | __link_local__          |                                      |
| 5a58e2d9-88b1-4ff2-9e0f-e0ed5e4e72ce | default-virtual-network |                                      |
| 65549c94-5ffc-4b30-9025-00a54813a695 | ip-fabric               |                                      |
+--------------------------------------+-------------------------+--------------------------------------+

Next article describes how to configure Simple Gateway to connect outside of OpenContrail.

Refference
https://gitlab.com/gokulpch/OpenContrail-Kolla/blob/master/README.md

Apr 19, 2016

Contrail R3.0 Manual installation on ubuntu(12)

Operation check

Check Contrail status.

1.Check Controller
Execute "contrail-status", then confirm whether Contrail runs well.
It's good work if Status shows either "active" or "backup".
** contrail-device-manager,contrail-schema,contrail-svc-monitor show "Active" only one of three server.
== Contrail Control ==
supervisor-control:           active
contrail-control              active              
contrail-control-nodemgr      active              
contrail-dns                  active              
contrail-named                active              

== Contrail Analytics ==
supervisor-analytics:         active
contrail-alarm-gen            active              
contrail-analytics-api        active              
contrail-analytics-nodemgr    active              
contrail-collector            active              
contrail-query-engine         active              
contrail-snmp-collector       active              
contrail-topology             active              

== Contrail Config ==
supervisor-config:            active
contrail-api:0                active              
contrail-config-nodemgr       active              
contrail-device-manager       backup              
contrail-discovery:0          active              
contrail-schema               active              
contrail-svc-monitor          backup              
ifmap                         active              

== Contrail Web UI ==
supervisor-webui:             active
contrail-webui                active              
contrail-webui-middleware     active              

== Contrail Database ==
contrail-database:            active
supervisor-database:          active
contrail-database-nodemgr     active              
kafka                         active              

== Contrail Support Services ==
supervisor-support-service:   active
rabbitmq-server               active    

2.Check vRouter
Execute "contrail-status" as well, Confirm it.
== Contrail vRouter ==
supervisor-vrouter:           active
contrail-vrouter-agent        active              
contrail-vrouter-nodemgr      active  

3.Check WebUI
Open "http://Contrail1:8080/" by Web browser, then login by admin user.
Dashboard shows number of nodes. If there is no alarm or error, Contrail works well.
Please make sure actual traffic to create Virtual-machine and Virtual-network.


Contrail R3.0 Manual installation on ubuntu(11)

Install Compute Node

A configuration of Nova Compute is many options, but it is done simple configuration. Please modify as your environment. Further more, if you want to use Nova-compute packages other than Local repository, I recommend you to install Nova compute before installing Contrail packages.
**Target Server: Compute1

1.Set variables
HOST=10.84.50.135  ## Set IP address of an Installation server
OPENSTACK=10.84.50.134  ## Set IP address of Openstack server
IVIP=10.84.50.139  ## Set Virtual IP address of Control/Data
DEV=p514p2  ## A name of Physical NIC for Control/Data
MAC=90:e2:ba:a1:aa:e1  ## MAC address of Physical NIC for Control/Data
DGW=10.84.50.252  ## Default Gateway of Control/Data. It is not used by Routing of Linux kernel.

2.Install Nova Compute
apt-get update
apt-get install -y nova-compute sysfsutils

3.Modify /etc/nova.conf
echo "[DEFAULT]
dhcpbridge_flagfile=/etc/nova/nova.conf
dhcpbridge=/usr/bin/nova-dhcpbridge
logdir=/var/log/nova
state_path=/var/lib/nova
lock_path=/var/lock/nova
force_dhcp_release=True
libvirt_use_virtio_for_bridges=True
verbose=True
ec2_private_dns_show_ip=False
api_paste_config=/etc/nova/api-paste.ini
enabled_apis=ec2,osapi_compute,metadata
rabbit_host = $OPENSTACK
security_group_api = neutron
service_neutron_metadata_proxy = True
compute_driver = libvirt.LibvirtDriver
firewall_driver = nova.virt.firewall.NoopFirewallDriver
novncproxy_port = 5999
novncproxy_host = 0.0.0.0
auth_strategy = keystone
network_api_class = nova.network.neutronv2.api.API
novncproxy_base_url = http://$OPENSTACK:6080/vnc_auto.html
vncserver_enabled = true
vncserver_listen = $HOST
vncserver_proxyclient_address = $HOST

[keystone_authtoken]
admin_tenant_name = service
admin_user = nova
admin_password = password
auth_protocol = http
auth_host = $OPENSTACK
auth_port = 35357
signing_dir = /tmp/keystone-signing-nova

[neutron]
admin_auth_url = http://$OPENSTACK:35357/v2.0/
admin_username = neutron
admin_password = password
admin_tenant_name = service
url = http://$IVIP:9696/
url_timeout = 300
service_metadata_proxy = True

[compute]
compute_driver = libvirt.LibvirtDriver

[glance]
host = $OPENSTACK" > /etc/nova.conf

4.Modify /etc/libvirt/qemu.conf
echo 'cgroup_device_acl = [
    "/dev/null", "/dev/full", "/dev/zero",
    "/dev/random", "/dev/urandom",
    "/dev/ptmx", "/dev/kvm", "/dev/kqemu",
    "/dev/rtc", "/dev/hpet","/dev/net/tun",
]' >> /etc/libvirt/qemu.conf

5.Install Contrail vRouter
In case of Linux Kernel version is other than 3.13.0-40.
apt-get -y install contrail-openstack-vrouter contrail-vrouter-dkms contrail-vrouter-common contrail-nova-vif
In case of Linux Kernel version is 3.13.0-40.
apt-get -y install contrail-openstack-vrouter contrail-vrouter-3.13.0-40-generic contrail-vrouter-common contrail-nova-vif

6.Modify /etc/contrail/vrouter_nodemgr_param
echo "DISCOVERY=$IVIP" > /etc/contrail/vrouter_nodemgr_param

7.Modify /etc/contrail/agent_param
echo "LOG=/var/log/contrail.log
CONFIG=/etc/contrail/contrail-vrouter-agent.conf
prog=/usr/bin/contrail-vrouter-agent
kmod=vrouter
pname=contrail-vrouter-agent
LIBDIR=/usr/lib64
DEVICE=vhost0
dev=$DEV
LOGFILE=--log-file=/var/log/contrail/vrouter.log" > /etc/contrail/agent_param

8.Modify /etc/contrail/contrail-vrouter-agent.conf
sed -i -e "/^\[DISCOVERY\]/i platform=default" /etc/contrail/contrail-vrouter-agent.conf
sed -i -e "/^\[DISCOVERY\]/i physical_interface_mac=$MAC" /etc/contrail/contrail-vrouter-agent.conf
sed -i "s/# server=127.0.0.1/server=$IVIP/g" /etc/contrail/contrail-vrouter-agent.conf
sed -i "s/# max_control_nodes=1/max_control_nodes=2/g" /etc/contrail/contrail-vrouter-agent.conf
sed -i "s/# type=kvm/type=kvm/g" /etc/contrail/contrail-vrouter-agent.conf
sed -i "s/# control_network_ip=/control_network_ip=$HOST/g" /etc/contrail/contrail-vrouter-agent.conf
sed -i "s/# name=vhost0/name=vhost0/g" /etc/contrail/contrail-vrouter-agent.conf
sed -i "s/# ip=10.1.1.1/24/ip=$HOST/24/g" /etc/contrail/contrail-vrouter-agent.conf
sed -i "s/# gateway=10.1.1.254/gateway=$DGW/g" /etc/contrail/contrail-vrouter-agent.conf
sed -i "s/# physical_interface=vnet0/physical_interface=$DEV/g" /etc/contrail/contrail-vrouter-agent.conf
sed -i "s/# thread_count = 4/thread_count = 4/g" /etc/contrail/contrail-vrouter-agent.conf

9.Modify /etc/network/interfaces
vRouter binds Physical NIC for Control/Data to vhost0. Hence, Physical interface doesn't have IP address, instead of that vhost0 has IP address.
Execute a command below, then copy and paste it to /etc/network/interface.
echo "auto $DEV
iface $DEV inet manual
    pre-up ifconfig $IDEV up
    post-down ifconfig $IDEV down


auto vhost0
iface vhost0 inet static
    pre-up /opt/contrail/bin/if-vhost0
    netmask 255.255.255.0
    network_name application
    address $HOST
    gateway $DGW
    dns-nameservers 8.8.8.8"

10.Modify /etc/contrail/contrail-vrouter-nodemgr.conf
echo "[DISCOVERY]
server=$IVIP
port=5998" >> /etc/contrail/contrail-vrouter-nodemgr.conf

11.Reboot ComputeNode
Reboot Compute Node

12.Add Compute node to Contrail
Open WebUI then Open "Configure"->"Infrastructure"->"Virtual Routers", Click"+"
The dialog below is opened. Set host name of Compute node to "Name" and IP address of vhost0 to "IP Address".

The all of installation and configuration is down.
Next topic is "Operation check".

Contrail R3.0 Manual installation on ubuntu(10)

Set initial configuration

Set initial Configuration
**It's enough to be done by only one target server.

1.Set variables
HOST=10.84.50.131  ## Set IP address of an Installation server
OPENSTACK=10.84.50.134  ## Set IP address of Openstack server
IVIP=10.84.50.139  ## Set Virtual IP address of Control/Data
host1=10.84.50.131  ## Physical IP address for Control/Data of Contrail1
host2=10.84.50.132  ## Physical IP address for Control/Data of Contrail2
host3=10.84.50.133  ## Physical IP address for Control/Data of Contrail3
host_name1=contrail1  ## Host name of Contrail1
host_name2=contrail2  ## Host name of Contrail2
host_name3=contrail3  ## Host name of Contrail3

2.Set Config Node
python /opt/contrail/utils/provision_config_node.py --api_server_ip $IVIP --admin_user admin --admin_password password --admin_tenant_name admin --oper add --host_name $host_name1 --host_ip $host1
python /opt/contrail/utils/provision_config_node.py --api_server_ip $IVIP --admin_user admin --admin_password password --admin_tenant_name admin --oper add --host_name $host_name2 --host_ip $host2
python /opt/contrail/utils/provision_config_node.py --api_server_ip $IVIP --admin_user admin --admin_password password --admin_tenant_name admin --oper add --host_name $host_name3 --host_ip $host3

3.Set Database Node
python /opt/contrail/utils/provision_database_node.py --api_server_ip $IVIP --admin_user admin --admin_password password --admin_tenant_name admin --oper add --host_name $host_name1 --host_ip $host1
python /opt/contrail/utils/provision_database_node.py --api_server_ip $IVIP --admin_user admin --admin_password password --admin_tenant_name admin --oper add --host_name $host_name2 --host_ip $host2
python /opt/contrail/utils/provision_database_node.py --api_server_ip $IVIP --admin_user admin --admin_password password --admin_tenant_name admin --oper add --host_name $host_name3 --host_ip $host3

4.Set Analytics Node
python /opt/contrail/utils/provision_analytics_node.py --api_server_ip $IVIP --admin_user admin --admin_password password --admin_tenant_name admin --oper add --host_name $host_name1 --host_ip $host1
python /opt/contrail/utils/provision_analytics_node.py --api_server_ip $IVIP --admin_user admin --admin_password password --admin_tenant_name admin --oper add --host_name $host_name2 --host_ip $host2
python /opt/contrail/utils/provision_analytics_node.py --api_server_ip $IVIP --admin_user admin --admin_password password --admin_tenant_name admin --oper add --host_name $host_name3 --host_ip $host3

5.Configure Contrail
python /opt/contrail/utils/provision_control.py --api_server_ip $IVIP --api_server_port 8082 --admin_user admin --admin_password password --admin_tenant_name admin --oper add --host_name $host_name1 --host_ip $host1 --router_asn 64512
python /opt/contrail/utils/provision_encap.py --api_server_ip $IVIP --admin_user admin --admin_password password --oper add --encap_priority MPLSoUDP,MPLSoGRE,VXLAN
python /opt/contrail/utils/provision_linklocal.py --api_server_ip $IVIP --admin_user admin --admin_password password --admin_tenant_name admin --oper add --ipfabric_service_ip $OPENSTACK --ipfabric_service_port 8775 --linklocal_service_name metadata --linklocal_service_ip 169.254.169.254 --linklocal_service_port 80

Next topic is "Install Compute Node".

Contrail R3.0 Manual installation on ubuntu(9)

Install Contrail WebUI


WebUI consists of contrail-webui,contrail-webui-middleware.
**Target Server: Contrail1-3

1.Set variables
HOST=10.84.50.131  ## Set IP address of an Installation server
OPENSTACK=10.84.50.134  ## Set IP address of Openstack server
IVIP=10.84.50.139  ## Set Virtual IP address of Control/Data
host1=10.84.50.131  ## Physical IP address for Control/Data of Contrail1
host2=10.84.50.132  ## Physical IP address for Control/Data of Contrail2
host3=10.84.50.133  ## Physical IP address for Control/Data of Contrail3

2.Install package
apt-get -y install contrail-openstack-webui

3.Modify /etc/contrail/config.global.js
sed -i "s/config.networkManager.ip =.*/config.networkManager.ip = '$IVIP';/g" /etc/contrail/config.global.js
sed -i "s/config.imageManager.ip = .*/config.imageManager.ip = '$OPENSTACK'/g" /etc/contrail/config.global.js
sed -i "s/config.computeManager.ip = .*/config.computeManager.ip = '$OPENSTACK'/g" /etc/contrail/config.global.js
sed -i "s/config.identityManager.ip = .*/config.identityManager.ip = '$OPENSTACK'/g" /etc/contrail/config.global.js
sed -i "s/config.storageManager.ip = .*/config.storageManager.ip = '$OPENSTACK'/g" /etc/contrail/config.global.js
sed -i "s/config.cnfg.server_ip = /config.cnfg.server_ip = '$IVIP'/g" /etc/contrail/config.global.js
sed -i "s/config.analytics.server_ip = /config.analytics.server_ip = '$IVIP'/g" /etc/contrail/config.global.js
sed -i "s/config.cassandra.server_ips = .*/config.cassandra.server_ips = ['$host1', '$host2', '$host3']/g" /etc/contrail/config.global.js

4.Modify contrail-webui-userauth.js
set -i "s/contrail123/password/g" /etc/contrail/contrail-webui-userauth.js

5.Create symbolic link
ln -s /usr/bin/node /usr/bin/nodejs

6.Restart process
service supervisor-webui restart

Next topic is "Set initial configuration".