Aug 8, 2018

How to configure Contrail/TungstenFabric to control EVPN/VXLAN on QFX51x0

Contrail/TungstenFabric has supported EVPN/VXLAN since 4.1.
**OVSDB/VXLAN had been deprecated on 5.0 due to scaling and stability issue.

Automated configuration for EVPN/VXLAN is only supported Juniper QFX51x0 series on 5.0.
**5.1 will support multi vender configuration.

How to Configure

in this article, Juniper QFX5100 is used.

1. Register Leaf Switch

Configure > Physical Devices > Physical Routers
Select "Netconf Managed Physical Router" from "+".
Name: Host name of Leaf Switch (HVTEP)
Vender: Juniper
Model: qfx5100
Management IP: IP address of Netconf/SSH connection.
Netconf Username: User name to accept configuration by Netconf.
Netconf Password: User password to accept configuration by Netconf.
Role: Leaf
EVPN Peered TOR: Enable check
JUNOS Service Port: Listen port of Netconf on Leaf Switch

2. Configure BGP Router

Configure > Infrastructure > BGP Routers
Router Type: BGP Router
Name: Hostname of Leaf Switch (HVTEP)
Vender ID: Juniper
IP Address: Source IP address of BGP as same as Loopback.
Router ID: Router ID of BGP. Same as IP Address
Autonomous System: AS number of Leaf Switch
BGP Router ASN: AS number which is used BGP connection of BGP Peer by Leaf Switch. Same as Autonomous System.
Address Families: Configure only inet-vpn, route-target, e-vpn
Advanced Options > Physical Router: Select Leaf Switch which is configured at "Physical Device"
Assosiate Peer(s) > Peer: Select Control node

3. Configure VTEP

Configure > Physical Devices > Physical Routers
Configure VTEP information on configured Physical Router. Click right side "Gear" icon.
VTEP Address: Configure Loopback IP address
Loopback IP: Configure Loopback IP address


4. Confirm Leaf Switch

Configuration is seen under "group __contrail__" which is configured by Contrail/TungstenFabric.
"show configuration groups __contrail__" shows loopback, BGP configuration and so on.
interfaces {
    /* Interfaces Configuration */
    lo0 {
        /* Router Loopback Interface */
        unit 0 {
            family inet {
                address 10.84.54.2/32 {
                    primary;
                    preferred;
                }
            }
        }
    }
}
routing-options {
    /* Global Routing Options */
    router-id 10.84.54.2;
    route-distinguisher-id 10.84.54.2;
    autonomous-system 64519;
    resolution {
        rib bgp.rtarget.0 {
            resolution-ribs inet.0;
        }
    }
}
protocols {
    /* Protocols Configuration */
    bgp {
        /* BGP Router: qfx5100-48t-1, UUID: 5e96224d-f708-4207-9ac3-b15a2699494e */
        group _contrail_asn-64519 {
            type internal;
            local-address 10.84.54.2;
            hold-time 90;
            family evpn {
                signaling;
            }
            family route-target;
        }
        /* BGP Router: qfx5100-48t-1, UUID: 5e96224d-f708-4207-9ac3-b15a2699494e */
        group _contrail_asn-64519-external {
            type external;
            multihop;
            local-address 10.84.54.2;
            hold-time 90;
            family evpn {
                signaling;
            }
            family route-target;
            /* BGP Router: overcloud-contrailcontroller-0, UUID: 72950843-4008-4e6f-8019-0d875e06dcd6 */
            neighbor 10.84.50.91 {
                peer-as 65534;
            }
        }
    }
}
policy-options {
    community _contrail_switch_policy_ members target:64519:1;
}
switch-options {
    vtep-source-interface lo0.0;
}

5. Configure Virtual network on Leaf Switch

Configure > Physical Devices > Interfaces > {Leaf Swtich}
Name: Configure Interface. Set Physical interface with sub-interface like "xe-0/0/1.0".
Logical Interface Properties > Logical Interface Type: Server
Logical Interface Properties > VLAN ID: Configure VLAN nmber. In case of Untag, set "0"
Logical Interface Properties > Virtual Network: Select Virtual network
Logical Interface Properties > Server MAC: Configure MAC address to asign IP address to Bare metal server by DHCP. Dummy MAC address is acceptable.
** TSN is required to assign IP address by DHCP.
Logical Interface Properties > IP address: IP address of DHCP. Blank is acceptable.

6. Confirm Leaf Switch

After configuration, interface is automatically configured. Necessary configuration is also configured.
__contrail__ {
    interfaces {
        /* Interfaces Configuration */
        lo0 {
            /* Router Loopback Interface */
            unit 0 {
                family inet {
                    address 10.84.54.2/32 {
                        primary;
                        preferred;
                    }
                }
            }
        }
        xe-0/0/1 {
            flexible-vlan-tagging;
            native-vlan-id 4094;
            encapsulation extended-vlan-bridge;
            /* L2 EVPN Untagged Interface, Virtual Network: vxlan-vn1, UUID: 1e1009e1-8113-4e62-b06f-24a3f5467b3f */
            unit 0 {
                vlan-id 4094;
            }
        }
    }
    routing-options {
        /* Global Routing Options */
        router-id 10.84.54.2;
        route-distinguisher-id 10.84.54.2;
        autonomous-system 64519;
        resolution {
            rib bgp.rtarget.0 {
                resolution-ribs inet.0;
            }
        }
    }
    protocols {
        /* Protocols Configuration */
        bgp {
            /* BGP Router: qfx5100-48t-1, UUID: 5e96224d-f708-4207-9ac3-b15a2699494e */
            group _contrail_asn-64519 {
                type internal;
                local-address 10.84.54.2;
                hold-time 90;
                family evpn {
                    signaling;
                }
                family route-target;
            }
            /* BGP Router: qfx5100-48t-1, UUID: 5e96224d-f708-4207-9ac3-b15a2699494e */
            group _contrail_asn-64519-external {
                type external;
                multihop;
                local-address 10.84.54.2;
                hold-time 90;
                family evpn {
                    signaling;
                }
                family route-target;
                /* BGP Router: overcloud-contrailcontroller-0, UUID: 72950843-4008-4e6f-8019-0d875e06dcd6 */
                neighbor 10.84.50.91 {
                    peer-as 65534;
                }
            }
        }
        evpn {
            vni-options {
                vni 5 {
                    vrf-target target:64512:100000;
                }
            }
            encapsulation vxlan;
            multicast-mode ingress-replication;
            extended-vni-list all;
        }
    }
    policy-options {
        /* Policy Options */
        /* Virtual Network: vxlan-vn1, UUID: 1e1009e1-8113-4e62-b06f-24a3f5467b3f, Route Targets Type: Import */
        policy-statement _contrail_vxlan-vn1-l2-5-import {
            term _contrail_switch_policy_ {
                from community _contrail_switch_policy_;
                then accept;
            }
            term t1 {
                from community [ _contrail_target_65534_8000003 _contrail_target_64512_100000 ];
                then accept;
            }
        }
        /* L2 Switch Global Export Policy */
        policy-statement _contrail_switch_export_policy_ {
            term t1 {
                then {
                    community add _contrail_switch_export_community_;
                }
            }
        }
        community _contrail_switch_export_community_ members [ target:65534:8000003 target:64512:100000 ];
        community _contrail_target_65534_8000003 members target:65534:8000003;
        community _contrail_target_64512_100000 members target:64512:100000;
        community _contrail_switch_policy_ members target:64519:1;
    }
    switch-options {
        vtep-source-interface lo0.0;
        route-distinguisher 10.84.54.2:1;
        vrf-import _contrail_vxlan-vn1-l2-5-import;
        vrf-export _contrail_switch_export_policy_;
        vrf-target {
            target:64519:1;
            auto;
        }
    }
    vlans {
        contrail_vxlan-vn1-l2-5 {
            interface xe-0/0/1.0;
            vxlan {
                vni 5;
            }
        }
    }
}

7. Confirm table on Leaf Switch

QFX shows MAC address and Remote VTEP infomation which are advertised by either Controller or other BGP Peer.
show ethernet-switching table 
MAC flags (S - static MAC, D - dynamic MAC, L - locally learned, P - Persistent static
           SE - statistics enabled, NM - non configured MAC, R - remote PE MAC, O - ovsdb MAC)


Ethernet switching table : 5 entries, 5 learned
Routing instance : default-switch
   Vlan                MAC                 MAC      Logical                Active
   name                address             flags    interface              source
   contrail_vxlan-vn1-l2-5 00:00:5e:00:01:01 DR     esi.1736               05:00:00:fc:00:00:00:00:05:00
   contrail_vxlan-vn1-l2-5 02:ed:68:05:27:94 D      vtep.32769             10.84.50.94
   contrail_vxlan-vn1-l2-5 08:81:f4:89:72:e0 D      vtep.32771             10.84.54.1
   contrail_vxlan-vn1-l2-5 10:0e:7e:dd:8f:43 D      vtep.32770             10.84.54.4
   contrail_vxlan-vn1-l2-5 10:0e:7e:dd:8f:44 D      xe-0/0/1.0
show route table bgp.evpn.0
   bgp.evpn.0: 12 destinations, 12 routes (12 active, 0 holddown, 0 hidden)
   + = Active Route, - = Last Active, * = Both

   1:10.84.54.1:0::050000fc000000000500::FFFF:FFFF/192 AD/ESI
                      *[BGP/170] 00:03:34, localpref 100, from 10.84.50.91
                         AS path: 65534 64512 I, validation-state: unverified
                       > to 10.84.53.13 via xe-0/0/46.0
   2:10.84.50.94:20::5::02:ed:68:05:27:94/304 MAC/IP
                      *[BGP/170] 00:03:34, MED 100, localpref 100, from 10.84.50.91
                         AS path: 65534 ?, validation-state: unverified
                       > to 10.84.53.13 via xe-0/0/46.0
   2:10.84.54.1:7::5::00:00:5e:00:01:01/304 MAC/IP
                      *[BGP/170] 00:03:34, localpref 100, from 10.84.50.91
                         AS path: 65534 64512 I, validation-state: unverified
                       > to 10.84.53.13 via xe-0/0/46.0
   2:10.84.54.1:7::5::08:81:f4:89:72:e0/304 MAC/IP
                      *[BGP/170] 00:03:34, localpref 100, from 10.84.50.91
                         AS path: 65534 64512 I, validation-state: unverified
                       > to 10.84.53.13 via xe-0/0/46.0
   2:10.84.54.4:1::5::10:0e:7e:dd:8f:43/304 MAC/IP
                      *[BGP/170] 00:03:34, localpref 100, from 10.84.50.91
                         AS path: 65534 64520 I, validation-state: unverified
                       > to 10.84.53.13 via xe-0/0/46.0
   2:10.84.50.94:20::5::02:ed:68:05:27:94::10.0.0.6/304 MAC/IP
                      *[BGP/170] 00:03:34, MED 100, localpref 100, from 10.84.50.91
                         AS path: 65534 ?, validation-state: unverified
                       > to 10.84.53.13 via xe-0/0/46.0
   2:10.84.54.1:7::5::00:00:5e:00:01:01::10.0.0.1/304 MAC/IP
                      *[BGP/170] 00:03:34, localpref 100, from 10.84.50.91
                         AS path: 65534 64512 I, validation-state: unverified
                       > to 10.84.53.13 via xe-0/0/46.0
   2:10.84.54.1:7::5::08:81:f4:89:72:e0::10.0.0.14/304 MAC/IP
                      *[BGP/170] 00:03:34, localpref 100, from 10.84.50.91
                         AS path: 65534 64512 I, validation-state: unverified
                       > to 10.84.53.13 via xe-0/0/46.0
   3:10.84.50.94:20::5::10.84.50.94/248 IM
                      *[BGP/170] 00:03:34, MED 200, localpref 100, from 10.84.50.91
                         AS path: 65534 ?, validation-state: unverified
                       > to 10.84.53.13 via xe-0/0/46.0
   3:10.84.54.1:7::5::10.84.54.1/248 IM
                      *[BGP/170] 00:03:34, localpref 100, from 10.84.50.91
                         AS path: 65534 64512 I, validation-state: unverified
                       > to 10.84.53.13 via xe-0/0/46.0
   3:10.84.54.2:1::5::10.84.54.2/248 IM
                      *[EVPN/170] 00:03:33
                         Indirect
   3:10.84.54.4:1::5::10.84.54.4/248 IM
                      *[BGP/170] 00:03:34, localpref 100, from 10.84.50.91
                         AS path: 65534 64520 I, validation-state: unverified
                       > to 10.84.53.13 via xe-0/0/46.0