Apr 19, 2016

Contrail R3.0 Manual installation on ubuntu(12)

Operation check

Check Contrail status.

1.Check Controller
Execute "contrail-status", then confirm whether Contrail runs well.
It's good work if Status shows either "active" or "backup".
** contrail-device-manager,contrail-schema,contrail-svc-monitor show "Active" only one of three server.
== Contrail Control ==
supervisor-control:           active
contrail-control              active              
contrail-control-nodemgr      active              
contrail-dns                  active              
contrail-named                active              

== Contrail Analytics ==
supervisor-analytics:         active
contrail-alarm-gen            active              
contrail-analytics-api        active              
contrail-analytics-nodemgr    active              
contrail-collector            active              
contrail-query-engine         active              
contrail-snmp-collector       active              
contrail-topology             active              

== Contrail Config ==
supervisor-config:            active
contrail-api:0                active              
contrail-config-nodemgr       active              
contrail-device-manager       backup              
contrail-discovery:0          active              
contrail-schema               active              
contrail-svc-monitor          backup              
ifmap                         active              

== Contrail Web UI ==
supervisor-webui:             active
contrail-webui                active              
contrail-webui-middleware     active              

== Contrail Database ==
contrail-database:            active
supervisor-database:          active
contrail-database-nodemgr     active              
kafka                         active              

== Contrail Support Services ==
supervisor-support-service:   active
rabbitmq-server               active    

2.Check vRouter
Execute "contrail-status" as well, Confirm it.
== Contrail vRouter ==
supervisor-vrouter:           active
contrail-vrouter-agent        active              
contrail-vrouter-nodemgr      active  

3.Check WebUI
Open "http://Contrail1:8080/" by Web browser, then login by admin user.
Dashboard shows number of nodes. If there is no alarm or error, Contrail works well.
Please make sure actual traffic to create Virtual-machine and Virtual-network.


Contrail R3.0 Manual installation on ubuntu(11)

Install Compute Node

A configuration of Nova Compute is many options, but it is done simple configuration. Please modify as your environment. Further more, if you want to use Nova-compute packages other than Local repository, I recommend you to install Nova compute before installing Contrail packages.
**Target Server: Compute1

1.Set variables
HOST=10.84.50.135  ## Set IP address of an Installation server
OPENSTACK=10.84.50.134  ## Set IP address of Openstack server
IVIP=10.84.50.139  ## Set Virtual IP address of Control/Data
DEV=p514p2  ## A name of Physical NIC for Control/Data
MAC=90:e2:ba:a1:aa:e1  ## MAC address of Physical NIC for Control/Data
DGW=10.84.50.252  ## Default Gateway of Control/Data. It is not used by Routing of Linux kernel.

2.Install Nova Compute
apt-get update
apt-get install -y nova-compute sysfsutils

3.Modify /etc/nova.conf
echo "[DEFAULT]
dhcpbridge_flagfile=/etc/nova/nova.conf
dhcpbridge=/usr/bin/nova-dhcpbridge
logdir=/var/log/nova
state_path=/var/lib/nova
lock_path=/var/lock/nova
force_dhcp_release=True
libvirt_use_virtio_for_bridges=True
verbose=True
ec2_private_dns_show_ip=False
api_paste_config=/etc/nova/api-paste.ini
enabled_apis=ec2,osapi_compute,metadata
rabbit_host = $OPENSTACK
security_group_api = neutron
service_neutron_metadata_proxy = True
compute_driver = libvirt.LibvirtDriver
firewall_driver = nova.virt.firewall.NoopFirewallDriver
novncproxy_port = 5999
novncproxy_host = 0.0.0.0
auth_strategy = keystone
network_api_class = nova.network.neutronv2.api.API
novncproxy_base_url = http://$OPENSTACK:6080/vnc_auto.html
vncserver_enabled = true
vncserver_listen = $HOST
vncserver_proxyclient_address = $HOST

[keystone_authtoken]
admin_tenant_name = service
admin_user = nova
admin_password = password
auth_protocol = http
auth_host = $OPENSTACK
auth_port = 35357
signing_dir = /tmp/keystone-signing-nova

[neutron]
admin_auth_url = http://$OPENSTACK:35357/v2.0/
admin_username = neutron
admin_password = password
admin_tenant_name = service
url = http://$IVIP:9696/
url_timeout = 300
service_metadata_proxy = True

[compute]
compute_driver = libvirt.LibvirtDriver

[glance]
host = $OPENSTACK" > /etc/nova.conf

4.Modify /etc/libvirt/qemu.conf
echo 'cgroup_device_acl = [
    "/dev/null", "/dev/full", "/dev/zero",
    "/dev/random", "/dev/urandom",
    "/dev/ptmx", "/dev/kvm", "/dev/kqemu",
    "/dev/rtc", "/dev/hpet","/dev/net/tun",
]' >> /etc/libvirt/qemu.conf

5.Install Contrail vRouter
In case of Linux Kernel version is other than 3.13.0-40.
apt-get -y install contrail-openstack-vrouter contrail-vrouter-dkms contrail-vrouter-common contrail-nova-vif
In case of Linux Kernel version is 3.13.0-40.
apt-get -y install contrail-openstack-vrouter contrail-vrouter-3.13.0-40-generic contrail-vrouter-common contrail-nova-vif

6.Modify /etc/contrail/vrouter_nodemgr_param
echo "DISCOVERY=$IVIP" > /etc/contrail/vrouter_nodemgr_param

7.Modify /etc/contrail/agent_param
echo "LOG=/var/log/contrail.log
CONFIG=/etc/contrail/contrail-vrouter-agent.conf
prog=/usr/bin/contrail-vrouter-agent
kmod=vrouter
pname=contrail-vrouter-agent
LIBDIR=/usr/lib64
DEVICE=vhost0
dev=$DEV
LOGFILE=--log-file=/var/log/contrail/vrouter.log" > /etc/contrail/agent_param

8.Modify /etc/contrail/contrail-vrouter-agent.conf
sed -i -e "/^\[DISCOVERY\]/i platform=default" /etc/contrail/contrail-vrouter-agent.conf
sed -i -e "/^\[DISCOVERY\]/i physical_interface_mac=$MAC" /etc/contrail/contrail-vrouter-agent.conf
sed -i "s/# server=127.0.0.1/server=$IVIP/g" /etc/contrail/contrail-vrouter-agent.conf
sed -i "s/# max_control_nodes=1/max_control_nodes=2/g" /etc/contrail/contrail-vrouter-agent.conf
sed -i "s/# type=kvm/type=kvm/g" /etc/contrail/contrail-vrouter-agent.conf
sed -i "s/# control_network_ip=/control_network_ip=$HOST/g" /etc/contrail/contrail-vrouter-agent.conf
sed -i "s/# name=vhost0/name=vhost0/g" /etc/contrail/contrail-vrouter-agent.conf
sed -i "s/# ip=10.1.1.1/24/ip=$HOST/24/g" /etc/contrail/contrail-vrouter-agent.conf
sed -i "s/# gateway=10.1.1.254/gateway=$DGW/g" /etc/contrail/contrail-vrouter-agent.conf
sed -i "s/# physical_interface=vnet0/physical_interface=$DEV/g" /etc/contrail/contrail-vrouter-agent.conf
sed -i "s/# thread_count = 4/thread_count = 4/g" /etc/contrail/contrail-vrouter-agent.conf

9.Modify /etc/network/interfaces
vRouter binds Physical NIC for Control/Data to vhost0. Hence, Physical interface doesn't have IP address, instead of that vhost0 has IP address.
Execute a command below, then copy and paste it to /etc/network/interface.
echo "auto $DEV
iface $DEV inet manual
    pre-up ifconfig $IDEV up
    post-down ifconfig $IDEV down


auto vhost0
iface vhost0 inet static
    pre-up /opt/contrail/bin/if-vhost0
    netmask 255.255.255.0
    network_name application
    address $HOST
    gateway $DGW
    dns-nameservers 8.8.8.8"

10.Modify /etc/contrail/contrail-vrouter-nodemgr.conf
echo "[DISCOVERY]
server=$IVIP
port=5998" >> /etc/contrail/contrail-vrouter-nodemgr.conf

11.Reboot ComputeNode
Reboot Compute Node

12.Add Compute node to Contrail
Open WebUI then Open "Configure"->"Infrastructure"->"Virtual Routers", Click"+"
The dialog below is opened. Set host name of Compute node to "Name" and IP address of vhost0 to "IP Address".

The all of installation and configuration is down.
Next topic is "Operation check".

Contrail R3.0 Manual installation on ubuntu(10)

Set initial configuration

Set initial Configuration
**It's enough to be done by only one target server.

1.Set variables
HOST=10.84.50.131  ## Set IP address of an Installation server
OPENSTACK=10.84.50.134  ## Set IP address of Openstack server
IVIP=10.84.50.139  ## Set Virtual IP address of Control/Data
host1=10.84.50.131  ## Physical IP address for Control/Data of Contrail1
host2=10.84.50.132  ## Physical IP address for Control/Data of Contrail2
host3=10.84.50.133  ## Physical IP address for Control/Data of Contrail3
host_name1=contrail1  ## Host name of Contrail1
host_name2=contrail2  ## Host name of Contrail2
host_name3=contrail3  ## Host name of Contrail3

2.Set Config Node
python /opt/contrail/utils/provision_config_node.py --api_server_ip $IVIP --admin_user admin --admin_password password --admin_tenant_name admin --oper add --host_name $host_name1 --host_ip $host1
python /opt/contrail/utils/provision_config_node.py --api_server_ip $IVIP --admin_user admin --admin_password password --admin_tenant_name admin --oper add --host_name $host_name2 --host_ip $host2
python /opt/contrail/utils/provision_config_node.py --api_server_ip $IVIP --admin_user admin --admin_password password --admin_tenant_name admin --oper add --host_name $host_name3 --host_ip $host3

3.Set Database Node
python /opt/contrail/utils/provision_database_node.py --api_server_ip $IVIP --admin_user admin --admin_password password --admin_tenant_name admin --oper add --host_name $host_name1 --host_ip $host1
python /opt/contrail/utils/provision_database_node.py --api_server_ip $IVIP --admin_user admin --admin_password password --admin_tenant_name admin --oper add --host_name $host_name2 --host_ip $host2
python /opt/contrail/utils/provision_database_node.py --api_server_ip $IVIP --admin_user admin --admin_password password --admin_tenant_name admin --oper add --host_name $host_name3 --host_ip $host3

4.Set Analytics Node
python /opt/contrail/utils/provision_analytics_node.py --api_server_ip $IVIP --admin_user admin --admin_password password --admin_tenant_name admin --oper add --host_name $host_name1 --host_ip $host1
python /opt/contrail/utils/provision_analytics_node.py --api_server_ip $IVIP --admin_user admin --admin_password password --admin_tenant_name admin --oper add --host_name $host_name2 --host_ip $host2
python /opt/contrail/utils/provision_analytics_node.py --api_server_ip $IVIP --admin_user admin --admin_password password --admin_tenant_name admin --oper add --host_name $host_name3 --host_ip $host3

5.Configure Contrail
python /opt/contrail/utils/provision_control.py --api_server_ip $IVIP --api_server_port 8082 --admin_user admin --admin_password password --admin_tenant_name admin --oper add --host_name $host_name1 --host_ip $host1 --router_asn 64512
python /opt/contrail/utils/provision_encap.py --api_server_ip $IVIP --admin_user admin --admin_password password --oper add --encap_priority MPLSoUDP,MPLSoGRE,VXLAN
python /opt/contrail/utils/provision_linklocal.py --api_server_ip $IVIP --admin_user admin --admin_password password --admin_tenant_name admin --oper add --ipfabric_service_ip $OPENSTACK --ipfabric_service_port 8775 --linklocal_service_name metadata --linklocal_service_ip 169.254.169.254 --linklocal_service_port 80

Next topic is "Install Compute Node".

Contrail R3.0 Manual installation on ubuntu(9)

Install Contrail WebUI


WebUI consists of contrail-webui,contrail-webui-middleware.
**Target Server: Contrail1-3

1.Set variables
HOST=10.84.50.131  ## Set IP address of an Installation server
OPENSTACK=10.84.50.134  ## Set IP address of Openstack server
IVIP=10.84.50.139  ## Set Virtual IP address of Control/Data
host1=10.84.50.131  ## Physical IP address for Control/Data of Contrail1
host2=10.84.50.132  ## Physical IP address for Control/Data of Contrail2
host3=10.84.50.133  ## Physical IP address for Control/Data of Contrail3

2.Install package
apt-get -y install contrail-openstack-webui

3.Modify /etc/contrail/config.global.js
sed -i "s/config.networkManager.ip =.*/config.networkManager.ip = '$IVIP';/g" /etc/contrail/config.global.js
sed -i "s/config.imageManager.ip = .*/config.imageManager.ip = '$OPENSTACK'/g" /etc/contrail/config.global.js
sed -i "s/config.computeManager.ip = .*/config.computeManager.ip = '$OPENSTACK'/g" /etc/contrail/config.global.js
sed -i "s/config.identityManager.ip = .*/config.identityManager.ip = '$OPENSTACK'/g" /etc/contrail/config.global.js
sed -i "s/config.storageManager.ip = .*/config.storageManager.ip = '$OPENSTACK'/g" /etc/contrail/config.global.js
sed -i "s/config.cnfg.server_ip = /config.cnfg.server_ip = '$IVIP'/g" /etc/contrail/config.global.js
sed -i "s/config.analytics.server_ip = /config.analytics.server_ip = '$IVIP'/g" /etc/contrail/config.global.js
sed -i "s/config.cassandra.server_ips = .*/config.cassandra.server_ips = ['$host1', '$host2', '$host3']/g" /etc/contrail/config.global.js

4.Modify contrail-webui-userauth.js
set -i "s/contrail123/password/g" /etc/contrail/contrail-webui-userauth.js

5.Create symbolic link
ln -s /usr/bin/node /usr/bin/nodejs

6.Restart process
service supervisor-webui restart

Next topic is "Set initial configuration".

Contrail R3.0 Manual installation on ubuntu(8)

Analytics Nodeの設定


Analytics Node consists of contrail-alarm-gen,contrail-analytics-api,contrail-collector,contrail-query-engine,contrail-snmp-collector,contrail-topology.
**Target Server: Contrail1-3

1.Set variables
HOST=10.84.50.131  ## Set IP address of an Installation server
OPENSTACK=10.84.50.134  ## Set IP address of Openstack server
IVIP=10.84.50.139  ## Set Virtual IP address of Control/Data
host1=10.84.50.131  ## Physical IP address for Control/Data of Contrail1
host2=10.84.50.132  ## Physical IP address for Control/Data of Contrail2
host3=10.84.50.133  ## Physical IP address for Control/Data of Contrail3

2.Install package
apt-get -y install contrail-openstack-analytics

3.Modify /etc/redis/redis.conf
sed -i "s/^bind/#bind/g" /etc/redis/redis.conf
sed -i "s/^save/#save/g" /etc/redis/redis.conf
sed -i "s/^dbfilename/#dbfilename/g" /etc/redis/redis.conf
sed -i "s/^lua-time-limit 5000/lua-time-limit 15000/g" /etc/redis/redis.conf

4.Modify /etc/contrail/contrail-collector.conf
sed -i "s/^# cassandra_server_list=.*/cassandra_server_list=$host1:9160 $host2:9160 $host3:9160/g" /etc/contrail/contrail-collector.conf
sed -i "s/^# kafka_broker_list=.*/kafka_broker_list=$host1:9092 $host2:9092 $host3:9092/g" /etc/contrail/contrail-collector.conf
sed -i "s/^# hostip=/hostip=$HOST/g" /etc/contrail/contrail-collector.conf
sed -i "s/^# http_server_port=8089/http_server_port=8089/g" /etc/contrail/contrail-collector.conf
sed -i "s/^# syslog_port=514/syslog_port=-1/g" /etc/contrail/contrail-collector.conf
sed -i "s/^# port=8086/port=8086/g" /etc/contrail/contrail-collector.conf
sed -i -e "/^# IP address of discovery server/a server=$IVIP" /etc/contrail/contrail-collector.conf
sed -i -e "/^# IP address of redis-server/a server=127.0.0.1" /etc/contrail/contrail-collector.conf
sed -i -e "/^cassandra_server_list/a zookeeper_server_list=$host1:2181,$host2:2181,$host3:2181" /etc/contrail/contrail-collector.conf

5.Modify /etc/contrail/contrail-query-engine.conf
sed -i "s/^# Collector configuration options/# Query-Engine configuration options/g" /etc/contrail/contrail-query-engine.conf
sed -i "s/^# cassandra_server_list=127.0.0.1:9160/cassandra_server_list=$host1:9160 $host2:9160 $host3:9160/g" /etc/contrail/contrail-query-engine.conf
sed -i "s/^# hostip=.*/hostip=$HOST/g" /etc/contrail/contrail-query-engine.conf
sed -i "s/^# http_server_port=8089/http_server_port=8091/g" /etc/contrail/contrail-query-engine.conf
sed -i "s/^# server=127.0.0.1 # discovery_server IP address/server=$IVIP/g" /etc/contrail/contrail-query-engine.conf
sed -i "s/^# port=6379/port=6379/g" /etc/contrail/contrail-query-engine.conf
sed -i "s/^# server=127.0.0.1/server=127.0.0.1/g" /etc/contrail/contrail-query-engine.conf

6.Modify /etc/contrail/contrail-analytics-api.conf
sed -i "s/^#host_ip = 127.0.0.1/host_ip = $HOST/g" /etc/contrail/contrail-analytics-api.conf
sed -i "s/^#cassandra_server_list=127.0.0.1:9160/cassandra_server_list=$host1:9160 $host2:9160 $host3:9160/g" /etc/contrail/contrail-analytics-api.conf
sed -i "s/^#http_server_port = 8090/http_server_port = 8090/g" /etc/contrail/contrail-analytics-api.conf
sed -i "s/^#rest_api_port = 8081/rest_api_port = 9081/g" /etc/contrail/contrail-analytics-api.conf
sed -i "s/^#rest_api_ip = 0.0.0.0/rest_api_ip = 0.0.0.0/g" /etc/contrail/contrail-analytics-api.conf
sed -i "s/^#log_category = /log_category = /g" /etc/contrail/contrail-analytics-api.conf
sed -i "s/^#disc_server_ip = 127.0.0.1/disc_server_ip = $IVIP/g" /etc/contrail/contrail-analytics-api.conf
sed -i "s/^#disc_server_port = 5998/disc_server_port = 5998/g" /etc/contrail/contrail-analytics-api.conf
sed -i "s/^#redis_server_port=6379/redis_server_port=6379/g" /etc/contrail/contrail-analytics-api.conf
sed -i "s/^#redis_query_port=6379/redis_query_port=6379/g" /etc/contrail/contrail-analytics-api.conf

7.Modify /etc/contrail/contrail-analytics-nodemgr.conf
sed -i "s/server=.*/server=$IVIP/g" /etc/contrail/contrail-analytics-nodemgr.conf

8.Modify /etc/contrail/contrail-snmp-collector.conf
sed -i "s/^#zookeeper=.*/zookeeper=$host1:2181,$host2:2181,$host3:2181/g" /etc/contrail/contrail-snmp-collector.conf
sed -i "s/^#disc_server_ip=.*/disc_server_ip=$IVIP/g" /etc/contrail/contrail-snmp-collector.conf
sed -i "s/^#disc_server_port=5998/disc_server_port=5998/g" /etc/contrail/contrail-snmp-collector.conf

9.Modify /etc/contrail/supervisord_analytics_files/contrail-snmp-collector.ini
sed -i "s/command=.*/command=\/usr\/bin\/contrail-snmp-collector --conf_file \/etc\/contrail\/contrail-snmp-collector.conf --conf_file \/etc\/contrail\/contrail-keystone-auth.conf/g" /etc/contrail/supervisord_analytics_files/contrail-snmp-collector.ini

10.Modify /etc/contrail/contrail-topology.conf
sed -i "s/^#zookeeper=.*/zookeeper=$host1:2181,$host2:2181,$host3:2181/g" /etc/contrail/contrail-topology.conf

echo "[DISCOVERY]
disc_server_ip = $IVIP
disc_server_port = 5998" >> /etc/contrail/contrail-topology.conf

11.Modify /etc/contrail/supervisord_analytics_files/contrail-topology.ini
sed -i "s/command=.*/command=\/usr\/bin\/contrail-topology --conf_file \/etc\/contrail\/contrail-topology.conf --conf_file \/etc\/contrail\/contrail-keystone-auth.conf/g" /etc/contrail/supervisord_analytics_files/contrail-topology.ini

12.Modify /etc/contrail/contrail-alarm-gen.conf
sed -i "s/^#host_ip =.*/host_ip = $HOST/g" /etc/contrail/contrail-alarm-gen.conf
sed -i "s/^#kafka_broker_list =.*/kafka_broker_list=$host1:9092 $host2:9092 $host3:9092/g" /etc/contrail/contrail-alarm-gen.conf
sed -i "s/^#zk_list =.*/zk_list = $host1:2181,$host2:2181,$host3:2181/g" /etc/contrail/contrail-alarm-gen.conf
sed -i "s/^#disc_server_ip =.*/disc_server_ip=$IVIP/g" /etc/contrail/contrail-alarm-gen.conf
sed -i "s/^#disc_server_port =.*/disc_server_port=5998/g" /etc/contrail/contrail-alarm-gen.conf

13.Restart process
service supervisor-analytics restart

Next topic is "Install Contrail WebUI".

Contrail R3.0 Manual installation on ubuntu(7)

Install Contrail Control Node

Control Node consists of contrail-control,contrail-dns,contrail-named.**Target Server: Contrail1-3

1.Set variables
HOST=10.84.50.131  ## Set IP address of an Installation server
IVIP=10.84.50.139  ## Set Virtual IP address of Control/Data

2.Modify /etc/contrail/contrail-control.conf
sed -i "s/# hostip=/hostip=$HOST/g" /etc/contrail/contrail-control.conf
sed -i "s/# server=127.0.0.1/server=$IVIP/g" /etc/contrail/contrail-control.conf
sed -i "s/# password=control-node/password=$HOST/g" /etc/contrail/contrail-control.conf
sed -i "s/# user=control-node/user=$HOST/g" /etc/contrail/contrail-control.conf

3.Modify /etc/contrail/contrail-dns.conf
cp /etc/contrail/contrail-control.conf /etc/contrail/contrail-dns.conf
sed -i "s/^# Control-node/# DNS/g" /etc/contrail/contrail-dns.conf
sed -i "s/^password=.*/password=$HOST.dns/g" /etc/contrail/contrail-dns.conf
sed -i "s/^user=.*/user=$HOST.dns/g" /etc/contrail/contrail-dns.conf

4.Restart process
service supervisor-control restart

Next topic is "Install Contrail Analytics Node".

Contrail R3.0 Manual installation on ubuntu(6)

Install Contrail Config Node

Contrail Node consists of contrail-api, contrail-device-manager, contrail-discovery, contrail-schema, contrail-svc-monitor, ifmap.
**Target Server: Contrail1-3

1.Set variables
HOST=10.84.50.131  ## Set IP address of an Installation server
OPENSTACK=10.84.50.134  ## Set IP address of Openstack server
IVIP=10.84.50.139  ## Set Virtual IP address of Control/Data
host1=10.84.50.131  ## Physical IP address for Control/Data of Contrail1
host2=10.84.50.132  ## Physical IP address for Control/Data of Contrail2
host3=10.84.50.133  ## Physical IP address for Control/Data of Contrail3
2.Modify /etc/security/limits.conf
echo 'root soft nofile 65535' >> /etc/security/limits.conf 
echo 'root hard nofile 65535' >> /etc/security/limits.conf 
echo '* hard nofile 65535' >> /etc/security/limits.conf 
echo '* soft nofile 65535' >> /etc/security/limits.conf 
echo '* hard nproc 65535' >> /etc/security/limits.conf 

3.Modify /etc/contrail/contrail-keystone-auth.conf
echo "
[KEYSTONE]
auth_host=$OPENSTACK
auth_protocol=http
auth_port=35357
admin_user=admin
admin_password=password
admin_tenant_name=admin
insecure=False
memcache_servers=127.0.0.1:11211" > /etc/contrail/contrail-keystone-auth.conf

4.Modify /etc/ifmap-server/log4j.properties
sed -i 's/log4j.rootLogger=INFO/log4j.rootLogger=TRACE, CONSOLE/g' /etc/ifmap-server/log4j.properties
sed -i 's/log4j.logger.de.fhhannover.inform.irond.proc=INFO, A1/log4j.logger.de.fhhannover.inform.irond.proc=TRACE, A1, A2/g' /etc/ifmap-server/log4j.properties
sed -i 's/log4j.appender.A1=org.apache.log4j.FileAppender/log4j.appender.A1=org.apache.log4j.ConsoleAppender/g' /etc/ifmap-server/log4j.properties
sed -i -e '/log4j.appender.A1.File=\/var\/log\/contrail\/ifmap-server.log/d'  /etc/ifmap-server/log4j.properties

echo '
log4j.appender.A2=org.apache.log4j.FileAppender
log4j.appender.A2.File=/var/log/contrail/ifmap-server.log
log4j.appender.A2.layout=org.apache.log4j.PatternLayout
log4j.appender.A2.layout.ConversionPattern=%d [%t] %-5p %x - %m%n

log4j.logger.de.fhhannover.inform.irond.rawrequests=TRACE, A3
log4j.additivity.de.fhhannover.inform.irond.rawrequests=false
log4j.appender.A3=org.apache.log4j.FileAppender
log4j.appender.A3.file=irond_raw.log
log4j.appender.A3.layout=org.apache.log4j.PatternLayout
log4j.appender.A3.layout.ConversionPattern=%d %-5p %x - %m%n

log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender
log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout
log4j.appender.CONSOLE.layout.ConversionPattern=%-8r [%t] %-5p %C{1} %x - %m%n' >> /etc/ifmap-server/log4j.properties

5.Modify /etc/ifmap-server/basicauthusers.properties
User name might be changed due to bug. https://bugs.launchpad.net/juniperopenstack/+bug/1499206
echo "
api-server:api-server
schema-transformer:schema-transformer
svc-monitor:svc-monitor

# This is a read-only MAPC
reader:reader
$host1:$host1
$host1.dns:$host1.dns
$host2:$host2
$host2.dns:$host2.dns
$host3:$host3
$host3.dns:$host3.dns" > /etc/ifmap-server/basicauthusers.properties

echo 'api-server=api-server-1--0000000001-1' > /etc/ifmap-server/publisher.properties

6.Modify /etc/contrail/contrail-api.conf
sed -i -e '/^\#/d' /etc/contrail/contrail-api.conf 
sed -i -e '/^$/d' /etc/contrail/contrail-api.conf 
sed -i "s/cassandra_server_list = .*/cassandra_server_list=$host1:9160 $host2:9160 $host3:9160/g" /etc/contrail/contrail-api.conf
sed -i "s/disc_server_ip = .*/disc_server_ip=$IVIP/g" /etc/contrail/contrail-api.conf
sed -i 's/ = /=/g' /etc/contrail/contrail-api.conf

echo "ifmap_server_ip=$HOST
ifmap_server_port=8443
listen_ip_addr=0.0.0.0
listen_port=9100
disc_server_ip=$IVIP
disc_server_port=5998
zk_server_ip=$host1:2181,$host2:2181,$host3:2181
rabbit_server=$IVIP:5673
list_optimization_enabled=True
auth = keystone
multi_tenancy=True

[SECURITY]
use_certs=False
keyfile=/etc/contrail/ssl/private_keys/apiserver_key.pem
certfile=/etc/contrail/ssl/certs/apiserver.pem
ca_certs=/etc/contrail/ssl/certs/ca.pem" >> /etc/contrail/contrail-api.conf

7.Modify /etc/init.d/contrail-api
echo '#!/bin/sh

# chkconfig: 2345 99 01
# description: Juniper Network Virtualization API

supervisorctl -s unix:///tmp/supervisord_config.sock ${1} `basename ${0}:0`' > /etc/init.d/contrail-api

8.Modify /etc/contrail/contrail-schema.conf
sed -i "s/disc_server_ip = .*/disc_server_ip=$IVIP/g" /etc/contrail/contrail-schema.conf
sed -i 's/ = /=/g' /etc/contrail/contrail-schema.conf

echo "ifmap_server_ip=$HOST
ifmap_server_port=8443
ifmap_username=schema-transformer
ifmap_password=schema-transformer
api_server_ip=$IVIP
api_server_port=8082
api_server_use_ssl=False
zk_server_ip=$host1:2181,$host2:2181,$host3:2181
cassandra_server_list=$host1:9160 $host2:9160 $host3:9160
rabbit_server=$IVIP:5673

[SECURITY]
use_certs=False
keyfile=/etc/contrail/ssl/private_keys/schema_xfer_key.pem
certfile=/etc/contrail/ssl/certs/schema_xfer.pem
ca_certs=/etc/contrail/ssl/certs/ca.pem" >> /etc/contrail/contrail-schema.conf

9.Modify /etc/contrail/contrail-device-manager.conf
sed -i "s/disc_server_ip = .*/disc_server_ip=$IVIP/g" /etc/contrail/contrail-device-manager.conf
sed -i 's/ = /=/g' /etc/contrail/contrail-device-manager.conf

echo "api_server_ip=$IVIP
api_server_port=8082
api_server_use_ssl=False
zk_server_ip=$host1:2181,$host2:2181,$host3:2181
cassandra_server_list=$host1:9160 $host2:9160 $host3:9160
rabbit_server=$IVIP:5673" >> /etc/contrail/contrail-device-manager.conf

10.Modify /etc/contrail/contrail-svc-monitor.conf
echo "[DEFAULTS]
ifmap_server_ip=$HOST
ifmap_server_port=8443
ifmap_username=svc-monitor
ifmap_password=svc-monitor
api_server_ip=$IVIP
api_server_port=8082
api_server_use_ssl=False
zk_server_ip=$host1:2181,$host2:2181,$host3:2181
log_file=/var/log/contrail/contrail-svc-monitor.log
cassandra_server_list=$host1:9160 $host2:9160 $host3:9160
disc_server_ip=$IVIP
disc_server_port=5998
region_name=RegionOne
log_local=1
log_level=SYS_NOTICE
rabbit_server=$IVIP:5673

[SECURITY]
use_certs=False
keyfile=/etc/contrail/ssl/private_keys/svc_monitor_key.pem
certfile=/etc/contrail/ssl/certs/svc_monitor.pem
ca_certs=/etc/contrail/ssl/certs/ca.pem

[SCHEDULER]
analytics_server_ip=$IVIP
analytics_server_port=8081" > /etc/contrail/contrail-svc-monitor.conf

11.Modify /etc/contrail/contrail-discovery.conf
sed -i "s/zk_server_ip=.*/zk_server_ip=$host1,$host2,$host3/g" /etc/contrail/contrail-discovery.conf
sed -i 's/listen_port=.*/listen_port=9110/g' /etc/contrail/contrail-discovery.conf
sed -i "s/cassandra_server_list.*/cassandra_server_list=$host1:9160 $host2:9160 $host3:9160/g" /etc/contrail/contrail-discovery.conf


echo '[DNS-SERVER]
policy=fixed ' >> /etc/contrail/contrail-discovery.conf

#"modify contrail discovery init script"
echo '#!/bin/sh

# chkconfig: 2345 99 01
# description: Juniper Network Virtualization API

supervisorctl -s unix:///tmp/supervisord_config.sock ${1} `basename ${0}:0`' >  /etc/init.d/contrail-discovery

12.Modify /etc/contrail/contrail-discovery.conf
sed -i 's/;AUTHN_TYPE =.*/AUTHN_TYPE=keystone/g' /etc/contrail/vnc_api_lib.ini
sed -i 's/;AUTHN_PROTOCOL =.*/AUTHN_PROTOCOL=http/g' /etc/contrail/vnc_api_lib.ini
sed -i "s/;AUTHN_SERVER =.*/AUTHN_SERVER=$OPENSTACK/g" /etc/contrail/vnc_api_lib.ini
sed -i 's/;AUTHN_PORT =.*/AUTHN_PORT=35357/g' /etc/contrail/vnc_api_lib.ini
sed -i 's/;AUTHN_URL =.*/AUTHN_URL=\/v2.0\/tokens/g' /etc/contrail/vnc_api_lib.ini

13.Modify /etc/contrail/contrail-config-nodemgr.conf
sed -i "s/server=.*/server=$IVIP/g" /etc/contrail/contrail-config-nodemgr.conf

14.Modify /etc/sudoers.d/contrail_sudoers
echo 'Defaults:contrail !requiretty

Cmnd_Alias CONFIGRESTART = /usr/sbin/service supervisor-config restart

contrail ALL = (root) NOPASSWD:CONFIGRESTART' > /etc/sudoers.d/contrail_sudoers

15.Modify /etc/neutron/plugins/opencontrail/ContrailPlugin.ini
echo "[APISERVER]
api_server_ip = $IVIP
api_server_port = 8082
multi_tenancy = True
contrail_extensions = ipam:neutron_plugin_contrail.plugins.opencontrail.contrail_plugin_ipam.NeutronPluginContrailIpam,policy:neutron_plugin_contrail.plugins.opencontrail.contrail_plugin_policy.NeutronPluginContrailPolicy,route-table:neutron_plugin_contrail.plugins.opencontrail.contrail_plugin_vpc.NeutronPluginContrailVpc,contrail:None

[COLLECTOR]
analytics_api_ip = $IVIP
analytics_api_port = 8081

[KEYSTONE]
auth_url = http://$OPENSTACK:35357/v2.0
admin_user=admin
admin_password=password
admin_tenant_name=admin" > /etc/neutron/plugins/opencontrail/ContrailPlugin.ini

16.Modify /etc/default/neutron-server
sed -i 's/NEUTRON_PLUGIN_CONFIG=.*/NEUTRON_PLUGIN_CONFIG="\/etc\/neutron\/plugins\/opencontrail\/ContrailPlugin.ini"/g' /etc/default/neutron-server

set values to neutron config
sed -i 's/^core_plugin = .*/core_plugin = neutron_plugin_contrail.plugins.opencontrail.contrail_plugin.NeutronPluginContrailCoreV2/g' /etc/neutron/neutron.conf
sed -i -e '/^core_plugin/a log_format = %(asctime)s.%(msecs)d %(levelname)8s [%(name)s] %(message)s/' /etc/neutron/neutron.conf
sed -i -e '/^core_plugin/a service_plugins = neutron_plugin_contrail.plugins.opencontrail.loadbalancer.plugin.LoadBalancerPlugin' /etc/neutron/neutron.conf
sed -i -e "/^core_plugin/a rabbit_hosts = $IVIP:5673" /etc/neutron/neutron.conf
sed -i -e '/^core_plugin/a api_extensions_path = extensions:\/usr\/lib\/python2.7\/dist-packages\/neutron_plugin_contrail\/extensions' /etc/neutron/neutron.conf
sed -i -e '/^core_plugin/a allow_overlapping_ips = True' /etc/neutron/neutron.conf
sed -i -e '/^core_plugin/a auth_strategy = keystone' /etc/neutron/neutron.conf
sed -i -e '/^core_plugin/a bind_port = 9697' /etc/neutron/neutron.conf

sed -i -e "/^\[quotas\]/a quota_driver = neutron_plugin_contrail.plugins.opencontrail.quota.driver.QuotaDriver" /etc/neutron/neutron.conf

sed -i "s/auth_uri = .*/auth_uri = http:\/\/$OPENSTACK:35357\/v2.0\//g" /etc/neutron/neutron.conf
sed -i "s/identity_uri = .*/identity_uri = http:\/\/$IVIP:5000/g" /etc/neutron/neutron.conf
sed -i "s/admin_tenant_name = .*/admin_tenant_name = service/g" /etc/neutron/neutron.conf
sed -i "s/admin_user = .*/admin_user = neutron/g" /etc/neutron/neutron.conf
sed -i "s/admin_password = .*/admin_password = password/g"  /etc/neutron/neutron.conf
sed -i -e '/^admin_password/a auth_protocol = http' /etc/neutron/neutron.conf
sed -i -e "/^admin_password/a auth_host = $OPENSTACK" /etc/neutron/neutron.conf

echo '[QUOTAS]
quota_network = -1
quota_subnet = -1
quota_port = -1

[service_providers]
service_provider = LOADBALANCER:Opencontrail:neutron_plugin_contrail.plugins.opencontrail.loadbalancer.driver.OpencontrailLoadbalancerDriver:default' >> /etc/neutron/neutron.conf

17.Restart process
service neutron-server restart
service supervisor-config restart

Next topic is "Install Contrail Control Node".

Contrail R3.0 Manual installation on ubuntu(5)

Install Contrail Support Service

Configure RabbitMQ.
**Target Server: Contrail1-3

1.Set variables
HOST=10.84.50.131  ## Set IP address of an Installation server
HOSTNAME=contrail1  ## Set hostname of a target Server
COOKIES=AANIJMOJRKEOIUDWHAVA  ## Set Cookies of RabbitMQ. Use same Cookies in all of Servers.

2.Stop unused proces and delete directory
pkill -9 beam
pkill -9 epmd
rm -rf /var/lib/rabbitmq/mnesia

3.Modify /etc/rabbitmq/rabbitmq-env.conf
echo "NODE_IP_ADDRESS=$HOST
NODENAME=rabbit@$HOSTNAME-ctrl" >/etc/rabbitmq/rabbitmq-env.conf

4.Modify /etc/rabbitmq/rabbitmq-env.conf
echo "[
   {rabbit, [ {tcp_listeners, [{\"$HOST\", 5672}]}, {cluster_partition_handling, autoheal},{loopback_users, []},
              {cluster_nodes, {['rabbit@contrail1-ctrl', 'rabbit@contrail2-ctrl', 'rabbit@contrail3-ctrl'], disc}},
              {vm_memory_high_watermark, 0.4},
              {disk_free_limit,50000000},
              {log_levels,[{connection, info},{mirroring, info}]},
              {heartbeat,10},
              {delegate_count,20},
              {channel_max,5000},
              {tcp_listen_options,
                        [binary,
                          {packet, raw},
                          {reuseaddr, true},
                          {backlog, 128},
                          {nodelay, true},
                          {exit_on_close, false},
                          {keepalive, true}
                         ]
              },
              {collect_statistics_interval, 60000}
            ]
   },
   {rabbitmq_management_agent, [ {force_fine_statistics, true} ] },
   {kernel, [{net_ticktime,  30}]}
]." > /etc/rabbitmq/rabbitmq.config

5.Modify /var/lib/rabbitmq/.erlang.cookie
echo $COOKIES > /var/lib/rabbitmq/.erlang.cookie

6.Start process
service supervisor-support-service restart

7.Check RabbitMQ status
Execute "rabbitmqctl cluster_status" and confirm cluster.
rabbitmqctl cluster_status
Cluster status of node 'rabbit@contrail1-ctrl' ...
[{nodes,[{disc,['rabbit@contrail1-ctrl','rabbit@contrail2-ctrl',
                'rabbit@contrail3-ctrl']}]},
 {running_nodes,['rabbit@contrail3-ctrl','rabbit@contrail2-ctrl','rabbit@contrail1-ctrl']},
 {cluster_name,<<"rabbit@contrail1">>},
 {partitions,[]}]

Next topic is "Install Contrail Config Node".

Contrail R3.0 Manual installation on ubuntu(4)

Install Contrail Database Node

Install Database Node.
Database Node consists of Cassandra,Zookeeper,Kafka.
**Target Server: Contrail1-3

1.Set variables
HOST=10.84.50.131  ## Set IP address of an Installation server
IVIP=10.84.50.139  ## Set Virtual IP address of Control/Data
ID=1  ## ID number for Zookeeper. Set unique ID number to each servers. For instance, set 1,2,3 to each servers.
host1=10.84.50.131  ## Physical IP address for Control/Data of Contrail1
host2=10.84.50.132  ## Physical IP address for Control/Data of Contrail2
host3=10.84.50.133  ## Physical IP address for Control/Data of Contrail3
2.Install package
apt-get -y install contrail-openstack-database
If error is happened like below, execute same command again.
Errors were encountered while processing:
 cassandra
 contrail-openstack-database
E: Sub-process /usr/bin/dpkg returned an error code (1)

3.Create directory for Analytics
mkdir -p /var/lib/cassandra/data/config_db_uuid 
mkdir -p /var/lib/cassandra/data/ContrailAnalytics
mkdir -p /var/lib/cassandra/data
sudo chown -R cassandra /var/lib/cassandra
sudo chgrp -R cassandra /var/lib/cassandra

4.Modify /etc/cassandra/cassandra.yaml
sed -i "s/cluster_name: 'Test Cluster'/cluster_name: 'Contrail'/g" /etc/cassandra/cassandra.yaml
sed -i "s/- seeds: \"127.0.0.1\"/- seeds: \"$host2,$host3\"/g" /etc/cassandra/cassandra.yaml
sed -i "s/listen_address: localhost/listen_address: $HOST/g" /etc/cassandra/cassandra.yaml
sed -i "s/rpc_address: localhost/rpc_address: $HOST/g" /etc/cassandra/cassandra.yaml

5.Modify /etc/cassandra/cassandra-env.sh
sed -i 's/JVM_OPTS="$JVM_OPTS -Xss256k"/JVM_OPTS="$JVM_OPTS -Xss512k"/g' /etc/cassandra/cassandra-env.sh
sed -i 's/JVM_OPTS="$JVM_OPTS -XX:MaxTenuringThreshold=1"/JVM_OPTS="$JVM_OPTS -XX:MaxTenuringThreshold=30"/g' /etc/cassandra/cassandra-env.sh
sed -i 's/# JVM_OPTS="$JVM_OPTS -XX:+PrintGCDetails"/JVM_OPTS="$JVM_OPTS -XX:+PrintGCDetails"/g' /etc/cassandra/cassandra-env.sh
sed -i 's/# JVM_OPTS="$JVM_OPTS -XX:+PrintGCDateStamps"/JVM_OPTS="$JVM_OPTS -XX:+PrintGCDateStamps"/g' /etc/cassandra/cassandra-env.sh
sed -i 's/# JVM_OPTS="$JVM_OPTS -XX:+PrintHeapAtGC"/JVM_OPTS="$JVM_OPTS -XX:+PrintHeapAtGC"/g' /etc/cassandra/cassandra-env.sh
sed -i 's/# JVM_OPTS="$JVM_OPTS -XX:+PrintTenuringDistribution"/JVM_OPTS="$JVM_OPTS -XX:+PrintTenuringDistribution"/g' /etc/cassandra/cassandra-env.sh
sed -i 's/# JVM_OPTS="$JVM_OPTS -XX:+PrintGCApplicationStoppedTime"/JVM_OPTS="$JVM_OPTS -XX:+PrintGCApplicationStoppedTime"/g' /etc/cassandra/cassandra-env.sh
sed -i 's/# JVM_OPTS="$JVM_OPTS -XX:+PrintPromotionFailure"/JVM_OPTS="$JVM_OPTS -XX:+PrintPromotionFailure"/g' /etc/cassandra/cassandra-env.sh
sed -i 's/# JVM_OPTS="$JVM_OPTS -XX:PrintFLSStatistics=1"/JVM_OPTS="$JVM_OPTS -XX:PrintFLSStatistics=1"/g' /etc/cassandra/cassandra-env.sh
sed -i 's/# JVM_OPTS="$JVM_OPTS -Xloggc:\/var\/log\/cassandra\/gc-`date +%s`.log"/JVM_OPTS="$JVM_OPTS -Xloggc:\/var\/log\/cassandra\/gc-`date +%s`.log"/g' /etc/cassandra/cassandra-env.sh

6.Modify /etc/zookeeper/conf/zoo.cfg
echo "maxSessionTimeout=120000" >> /etc/zookeeper/conf/zoo.cfg
echo "autopurge.purgeInterval=3" >> /etc/zookeeper/conf/zoo.cfg
echo "server.1=$host1:2888:3888" >> /etc/zookeeper/conf/zoo.cfg
echo "server.2=$hos2:2888:3888" >> /etc/zookeeper/conf/zoo.cfg
echo "server.3=$host3:2888:3888" >> /etc/zookeeper/conf/zoo.cfg 

7.Modify /etc/zookeeper/conf/log4j.properties
sed -i 's/^#log4j.appender.ROLLINGFILE.MaxBackupIndex=/log4j.appender.ROLLINGFILE.MaxBackupIndex=/g' /etc/zookeeper/conf/log4j.properties 

8.Modify /etc/zookeeper/conf/environment
echo ZOO_LOG4J_PROP="INFO,CONSOLE,ROLLINGFILE" >> /etc/zookeeper/conf/environment

9.Modify /var/lib/zookeeper/myid
echo $ID > /var/lib/zookeeper/myid

10.Modify /usr/share/kafka/config/server.properties
sed -i "s/broker.id=0/broker.id=$ID/g" /usr/share/kafka/config/server.properties
sed -i 's/listeners=PLAINTEXT:\/\/:9092/#listeners=PLAINTEXT:\/\/:9092/g' /usr/share/kafka/config/server.properties
sed -i 's/#port=9092/port=9092/g' /usr/share/kafka/config/server.properties
sed -i "s/#advertised.host.name=/advertised.host.name=$HOST/g" /usr/share/kafka/config/server.properties
sed -i "s/zookeeper.connect=localhost:2181/zookeeper.connect=$host1:2181,$host2:2181,$host3:2181/g" /usr/share/kafka/config/server.properties

echo "log.cleanup.policy=compact" >> /usr/share/kafka/config/server.properties
echo "delete.topic.enable=true" >> /usr/share/kafka/config/server.properties

11.Modify /usr/share/kafka/config/log4j.properties
sed -i 's/DailyRollingFileAppender/RollingFileAppender/g' /usr/share/kafka/config/log4j.properties
sed -i "s/DatePattern='.'yyyy-MM-dd-HH/MaxBackupIndex=10/g" /usr/share/kafka/config/log4j.properties

12.Modify /etc/contrail/contrail-database-nodemgr.conf
sed -i "s/hostip=127.0.0.1/hostip=$HOST/g" /etc/contrail/contrail-database-nodemgr.conf
sed -i "s/server=127.0.0.1/server=$IVIP/g" /etc/contrail/contrail-database-nodemgr.conf
sed -i 's/minimum_diskGB=4/minimum_diskGB=100/g' /etc/contrail/contrail-database-nodemgr.conf

ln -s /bin/true /sbin/chkconfig
database-server-setup.sh $HOST
13.Check zookeeper status
Execute "/usr/share/zookeeper/bin/zkServer.sh status" on all of Servers. Zookeeper cluster is done if one of servers is Leader, then remaining servers are follower.
/usr/share/zookeeper/bin/zkServer.sh status 
JMX enabled by default 
Using config: /etc/zookeeper/conf/zoo.cfg 
Mode: follower 
14.Check cassandra status
Cassandra cluster is done if Execute "nodetool status" shows all of three servers.
Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address       Load       Tokens  Owns    Host ID                               Rack
UN  10.84.50.132  2.83 GB    256     ?       4022583f-bfeb-41af-a877-878468ef379a  rack1
UN  10.84.50.133  2.68 GB    256     ?       4a7a351a-9e62-4388-9c45-b66c37a4efe2  rack1
UN  10.84.50.131  2.66 GB    256     ?       ed68231d-edc0-40df-9987-411de522888a  rack1

Note: Non-system keyspaces don't have the same replication settings, effective ownership information is meaningless

Next topic is "Install Contrail Support Service".

Contrail R3.0 Manual installation on ubuntu(3)

Install HA modules

This topic describes how to install High Availability, haproxy and keepalived. In case of using external Load balancer, you don't need install them. Instead of that, properly configure the load balancer.
**Target Server: Contrail1-3

1.Set variables
Set variables of IP address, Physical NIC and so on for easy installation.
Modify sample below to IP address and servers of your actual environment.
HOST=10.84.50.131  ## Set IP address of an Installation server
IVIP=10.84.50.139  ## Set Virtual IP address of Control/Data
IVIP_name=10_84_50_139  ## An identification of keepalived for Control/Data
IDEV=eth0  ## A name of Physical NIC for Control/Data
EVIP_name=172_27_115_159  ## An identification of keepalived for Management
EVIP=172.27.115.159  ## Set Virtual IP address of Management
EDEV=eth1  ## A name of Physical NIC for Management
PRIORITY=100  ## A priority of Keepalived. Set unique number among 0-255 in each server. The server that has the biggest number elects as Master.
host1=10.84.50.131  ## Physical IP address for Control/Data of Contrail1
host2=10.84.50.132  ## Physical IP address for Control/Data of Contrail2
host3=10.84.50.133  ## Physical IP address for Control/Data of Contrail3

2.Install packages
apt-get install -y contrail-openstack-config

3.Configure haproxy
Modify configuration for Neutron, RabbitMQ, Contrail API, Discovery Server, Analytics API which needs IP address redundancy.、

External/Internal ports are below.(External port/Internal port)
Neutron
9696 / 9697
RabbitMQ
5673 / 5672
Contrail API
8082 / 9100
Discovery server
5998 / 9110
AnalyticsAPI
8081 / 9081

** Refer Juniper Official Document for Contrai Redundancy.

Copy and paste below command for Configuration.
sed -i -e '1a\ \ \ \ \ \ \ \ tune.bufsize 16384' /etc/haproxy/haproxy.cfg
sed -i -e '1a\ \ \ \ \ \ \ \ tune.maxrewrite 1024' /etc/haproxy/haproxy.cfg

echo '
listen contrail-config-stats :5937
   mode http
   stats enable
   stats uri /
   stats auth haproxy:contrail123

' >> /etc/haproxy/haproxy.cfg

echo "
##contrail-collector-marker-start
frontend  contrail-analytics-api *:8081
    default_backend    contrail-analytics-api

backend contrail-analytics-api
    option nolinger
    balance     roundrobin
    option tcp-check
    tcp-check connect port 6379
    default-server error-limit 1 on-error mark-down
    server $host1 $host1:9081 check inter 2000 rise 2 fall 3
    server $host2 $host2:9081 check inter 2000 rise 2 fall 3
    server $host3 $host3:9081 check inter 2000 rise 2 fall 3

#contrail-collector-marker-end


#contrail-config-marker-start

frontend  contrail-api *:8082
    default_backend    contrail-api-backend
    timeout client 3m

frontend neutron-server *:9696
    default_backend    neutron-server-backend

frontend  contrail-discovery *:5998
    default_backend    contrail-discovery-backend

backend neutron-server-backend
    option nolinger
    balance     roundrobin
    server $host1 $host1:9697 check inter 2000 rise 2 fall 3
    server $host2 $host2:9697 check inter 2000 rise 2 fall 3
    server $host3 $host3:9697 check inter 2000 rise 2 fall 3

backend contrail-api-backend
    option nolinger
    timeout server 3m
    balance     roundrobin
    server $host1 $host1:9100 check inter 2000 rise 2 fall 3
    server $host2 $host2:9100 check inter 2000 rise 2 fall 3
    server $host3 $host3:9100 check inter 2000 rise 2 fall 3

backend contrail-discovery-backend
    option nolinger
    balance     roundrobin
    server $host1 $host1:9110 check inter 2000 rise 2 fall 3
    server $host2 $host2:9110 check inter 2000 rise 2 fall 3
    server $host3 $host3:9110 check inter 2000 rise 2 fall 3

listen  rabbitmq 0.0.0.0:5673
    mode tcp
    maxconn 10000
    balance leastconn
    option tcpka
    option nolinger
    option forceclose
    timeout client 0
    timeout server 0
    timeout client-fin 60s
    timeout server-fin 60s
    server rabbit1 $host1:5672 weight 200 check inter 2000 rise 2 fall 3
    server rabbit2 $host2:5672 weight 100 check inter 2000 rise 2 fall 3 backup
    server rabbit3 $host3:5672 weight 100 check inter 2000 rise 2 fall 3 backup

#contrail-config-marker-end" >> /etc/haproxy/haproxy.cfg

4.Configure keepalived
echo "
vrrp_script chk_haproxy_INTERNAL_$IVIP_NAME {
        script \"pgrep -x haproxy > /dev/null\" # verify if pid exists
        interval 1
        timeout 3
        rise 2
        fall 2
}

vrrp_script chk_ctrldatanet_INTERNAL_$IVIP_NAME {
    script \"/opt/contrail/bin/chk_ctrldata.sh\"
    interval 1
    timeout 3
    rise 1
    fall 1
}

vrrp_instance INTERNAL_$IVIP_NAME {
        interface $IDEV
        state MASTER
        preempt_delay 7
        garp_master_delay 5
        garp_master_repeat 3
        garp_master_refresh 1
        advert_int 1
        virtual_router_id 100
        vmac_xmit_base
        priority  $PRIORITY
        virtual_ipaddress {
                $IVIP/24 dev $IDEV
        }
        track_script  {
                chk_haproxy_INTERNAL_$IVIP_NAME
        }

        track_script  {
            chk_ctrldatanet_INTERNAL_$IVIP_NAME
        }
        track_interface {
            $IDEV
        }
}

vrrp_script chk_haproxy_EXTERNAL_$EVIP_NAME {
        script \"pgrep -x haproxy > /dev/null\" # verify if pid exists
        interval 1
        timeout 3
        rise 2
        fall 2
}

vrrp_script chk_ctrldatanet_EXTERNAL_$EVIP_NAME {
    script \"/opt/contrail/bin/chk_ctrldata.sh\"
    interval 1
    timeout 3
    rise 1
    fall 1
}

vrrp_instance EXTERNAL_$EVIP_NAME {
        interface $EDEV
        state MASTER
        preempt_delay 7
        garp_master_delay 5
        garp_master_repeat 3
        garp_master_refresh 1
        advert_int 1
        virtual_router_id 100
        vmac_xmit_base
        priority  $PRIORITY
        virtual_ipaddress {
                $EVIP/24 dev $EDEV
        }
        track_script  {
                chk_haproxy_EXTERNAL_$EVIP_NAME
        }

        track_script  {
            chk_ctrldatanet_EXTERNAL_$EVIP_NAME
        }
        track_interface {
            $IDEV
            $EDEV
        }
}" > /etc/keepalived/keepalived.conf

5.Restart each process
Reflect modified configuration of haproxy and keepalived.
service keepalived restart
service haproxy restart

Next topic is "Install Contrail Database Node".

Contrail R3.0 Manual installation on ubuntu(2)

Create local repository of Contrail and Set common configuration

This topic describes how to create apt local repository for Contrail and set common configuration of Linux server.
This topic is same in each Linux servers. Please execute in all of them.
**Target server: Contrail1-3, Compute1

1.Install packages
Save downloaded "contrail-install-packages_3.0.0.0-2725~ubuntu-14-04kilo_all.deb" into /tmp. install deb file by dpkg commoand.
dpkg -i contrail-install-packages_3.0.0.0-2725~ubuntu-14-04kilo_all.deb

2.Unarchive Contrail packages
mkdir -p /opt/contrail/contrail_install_repo
cd /opt/contrail/contrail_install_repo
tar zxvf /opt/contrail/contrail_packages/contrail_debs.tgz

3.Install shell scripts of Contrail to /opt/contrail/bin
mkdir -p /opt/contrail/bin
cd /opt/contrail/contrail_install_repo
dpkg -i binutils_*.deb make_*.deb libdpkg-perl_*.deb dpkg-dev_*.deb patch_*.deb

4.Modify apt repository to refer local packages
sed -e '1ideb file:\/opt\/contrail\/contrail_install_repo \.\/' -i /etc/apt/sources.list

5.Allow to install unauthenticated packages
echo 'APT::Get::AllowUnauthenticated "true";' > /etc/apt/apt.conf

6.Intall local repository to /etc/apt/apt.conf from /opt/contral, then package it
cp /opt/contrail/contrail_packages/preferences /etc/apt/preferences
cd /opt/contrail/contrail_install_repo
dpkg-scanpackages . /dev/null | gzip -9c > Packages.gz

7.Install Contrail-setup
apt-get -y update
apt-get -y install contrail-setup

8.Change Linux Kernel(option)
**If you want to align Juniper recommended version, do this item.
Contrail can be installed even the version is different.
apt-get -y install linux-headers-3.13.0-40 linux-image-extra-3.13.0-40-generic

9.Modify GRUB(option)
**If you want to align Juniper recommended version, do this item.
sed -i "s/GRUB_DEFAULT=0/GRUB_DEFAULT='Advanced options for Ubuntu\>Ubuntu\,\ with Linux 3.13.0-40-generic'/g" /etc/default/grub
update-grub
reboot

10.Set common configuration of Linux server
sysctl net.ipv4.ip_forward=1
sysctl kernel.core_pattern=/var/crashes/core.%e.%p.%h.%t
sysctl fs.file-max=65535
sysctl -p
mkdir -m 0777 -p /var/crashes

echo 'fs.file-max = 65535' >> /etc/sysctl.conf
echo 'kernel.core_pattern = /var/crashes/core.%e.%p.%h.%t' >> /etc/sysctl.conf

11.Disable SE Linux(option)
sed -i 's/SELINUX=.*/SELINUX=disabled/g' /etc/selinux/config

12.Modify /etc/hosts
Add items below to /etc/hosts. Note: "host name" + "-ctrl" is used by RabbitMQ.
127.0.0.1       localhost
10.84.50.131    contrail1 contrail1-ctrl
10.84.50.132    contrail2 contrail2-ctrl
10.84.50.133    contrail3 contrail3-ctrl
10.84.50.134    controller
10.84.50.135    compute1

13.Install ntp
apt-get -y install ntp
Confirm NTP sync by "ntpq -p"

Next topic is "Install HA modules".

Contrail R3.0 Manual installation on ubuntu(1)

There are some ways of Contrail installation such as Fabric or Contrail Server Manager. But we can manually install Contrail. To learn how to install contrail by hands, we can use our own Configuration management tools like Ansible or Chef.

Today, Contrail is installed High Avaiilability. The installarion procedure is below.
  1. Create local repository of Contrail and Set common configuration.
  2. Install HA modules
  3. Install Contrail Database Node
  4. Install Contrail Support Service
  5. Install Contrail Config Node
  6. Install Contrail Control Node
  7. Install Contrail Analytics Node
  8. Install Contrail WebUI
  9. Set initial configuration
  10. Install Compute Node
  11. Operation check
There are many items, but almost configuration can be done by Copy and Paste.

Prerequisite of Installation Contrail

  • Contrail version is R3.0 that is provided at juniper.net. 
  • Finish to install/Configure Openstack except Neutron before installing Contrail.
  • Ubuntu version is 14.04.2. Finish to install OS and configure basic configuration like IP address, default gateway, ssh and so on.
Contrail is installed to Physical configuration below. There are two networks, one is Management for SSH and another is Control/Data for actual traffic of User traffic and Control traffic generated by Contrail.


IP address and Host name

Contrail
10.84.50.131    contrail1
10.84.50.132    contrail2
10.84.50.133    contrail3
Openstack
10.84.50.134    openstack
Compute
10.84.50.135    compute1
HAのVIP
172.27.115.159
10.84.50.139

User ID and Password

Neutron
neutron / password
Openstack Admin
admin / password

Installed components in Openstack node

Keysotne
Nova
Cinder
Glance
Horizon

Next topic is "Create local repository of Contrail and Set common configuration".