tor_tsn_compare uses contrail_sandeshlibs which outputs useful python dictionary from introspect information in vRouter-agent, Control-node and so on.
I will write how to use contrail_sandeshlibs in other article. Please wait a while.
How to control ToR-Switch by Contrail
Contrail configures Physical-port,VLAN, MAC, Remote-VTEP and so on in ToR Switch by OVSDB messages. OVSDB is one of the standard protocols which was defined by VMware.
OVSDB is used by open vswitch also, you can see the manual at "vtep(5)" in open vswitch.
A pic below shows the OVSDB data flow with Contrail. TSN stands for ToR service node which controls ToR Switch.
- ToR Switch connects HAProxy by OVSDB over TLS, if High availability is configured. If not, ToR Switch directly connect to TSN.
- HA Proxy connects one of the TSN by OVSDB over TLS
- ToR Agent sends some messages such as ff:ff:ff:ff:ff:ff(any-dst) for BUM, IP address of Remote VTEP for Unicast,VLAN,VNI to ToR Switch.
- ToR Agent also sends some messages such as VTEP info of ToR Switch, MAC address connecting itself to Control-node.
- Control-node sends received data to other ToR-Agent and vRouter-agent.
That is, ToR_Switch receives many Configurations and other data from TSN.
While receiving data, the data might be inconsistent between TSN and ToR Switch due to connection failure or so, in some cases, traffic might be dropped.
To confirm such consistent, we will have to confirm some tables in TSN and ToR-agent.
There are some sachems for Hardware VTEP in OVSDB-server(More detail Link). We will know lack of data by confirming which data will be synchronized or not with TSN.
For instance, ToR Switch Ucast_Macs_Remote that ToR Switch receives shows below (Use ovseb-client)
Ucast_Macs_Remote table
MAC _uuid ipaddr locator logical_switch
------------------- ------------------------------------ ------ ------------------------------------ ------------------------------------
"02:8f:d5:8e:e6:56" 26a7d358-4c43-4831-a495-a4373042776e "" 331baf5f-6b34-4a7d-8df2-fb2aa2e6eff8 bced0b50-fbaf-4757-bb4c-0ef9532e8eaa
"02:e4:d7:4c:50:cc" a7768131-4d58-4fd0-af67-ced3836011af "" a113bfee-4156-45ea-8371-f58dc2f1eab5 bced0b50-fbaf-4757-bb4c-0ef9532e8eaa
"22:22:22:00:00:01" 4b514bb4-e28c-49e8-bf27-bfeba71c638b "" df1e1788-e7a7-449e-be04-4143209b776b 6b1e1249-8d02-4bbf-aac6-da9d9906dd5e
"22:22:22:00:00:01" 8c86ef9a-3a98-4adf-bdb4-cdf087018e59 "" df1e1788-e7a7-449e-be04-4143209b776b 9a892745-09a4-434a-8bc6-b261668e8019
"22:22:22:00:00:01" b0761188-a077-4869-a6ff-78f0217951cc "" df1e1788-e7a7-449e-be04-4143209b776b ca899f3d-d5df-44f8-9126-90125f0f52e4
"22:22:22:00:00:01" d88c82db-d8a2-40b5-a59c-8610c7e75320 "" df1e1788-e7a7-449e-be04-4143209b776b e691aaae-e42f-4b3c-93a6-128b5421de58
"22:22:22:00:00:01" dea6b7d0-69e7-4da3-ab3e-f84e92efc9ed "" df1e1788-e7a7-449e-be04-4143209b776b 8957219c-ed1a-4a8a-829f-4a4ccd3ef41b
"22:22:22:00:00:01" f1f35b78-aa31-47db-90ae-fb85dee61359 "" df1e1788-e7a7-449e-be04-4143209b776b 6079900f-5e6a-45d7-b11f-adaab6442663
"22:22:22:00:00:02" 7c70dc8c-7cd7-492a-805a-b5e7519af880 "" df1e1788-e7a7-449e-be04-4143209b776b 1c6e1fd8-89f0-4752-baec-2859f409b215
"22:22:22:00:00:02" 8b8c086b-b6d0-4519-a620-0da110fc60ff "" df1e1788-e7a7-449e-be04-4143209b776b 895c18c3-3578-44e9-a0e3-939289b5a473
"22:22:22:00:00:02" 8d5cbf5a-1095-49ed-be28-0c8035762174 "" df1e1788-e7a7-449e-be04-4143209b776b cdf05c2a-7f0f-4922-a1d8-95d2d38e219f
"22:22:22:00:00:02" 9027f67f-e216-4a6b-9843-a3a40688d261 "" df1e1788-e7a7-449e-be04-4143209b776b 61f1d0a8-3889-46be-9ca0-ff45794346d5
"22:22:22:00:00:02" 9856a1e9-075e-420f-aa2f-0c45e3bb1b86 "" df1e1788-e7a7-449e-be04-4143209b776b 827da3e0-b340-4beb-8b8b-77d8a94c1409
"22:22:22:00:00:02" a25fbec4-24a6-454c-81c5-ac6b7e677682 "" df1e1788-e7a7-449e-be04-4143209b776b c90463d7-42e7-403f-a1ac-acb743d28f12locator column makes us annoyed. This locator indicates Remote VTEP, but we have to refer Phsical_locator table to get actual IP address of it.
Physical_Locator table
_uuid dst_ip encapsulation_type
------------------------------------ -------------- ------------------
ea87b2ee-150e-4b60-8374-9792b0cc1172 "10.84.50.4" "vxlan_over_ipv4"
353eb19e-6a82-4d5a-9505-e0845b820e86 "10.84.51.72" "vxlan_over_ipv4"
df1e1788-e7a7-449e-be04-4143209b776b "10.84.51.73" "vxlan_over_ipv4"
331baf5f-6b34-4a7d-8df2-fb2aa2e6eff8 "192.168.21.1" "vxlan_over_ipv4"
a113bfee-4156-45ea-8371-f58dc2f1eab5 "192.168.22.1" "vxlan_over_ipv4" We can see the actual Remote VTEP where MAC address is there by getting two tables together.
Further more, this information refers a part of table named OvsdbUnicastMacRemote in TSN
**OVSDB info in TSN can refer TSN_IPaddress:Port/ovsdb.xml.
UUID of Virtual-network needs to collect table above.
To know which MAC address is allocated in which Virtual-networks needs to confirm UUID of Virtual-network by referring Logical_Switch table from UUID of Ucast_MAC_Remote in Logical_Switch column.
In case of Contrail, UUID of Virtual-netowrk shows "Contrail-" + UUID
Logical_Switch table
_uuid description name tunnel_key
------------------------------------ ----------- ----------------------------------------------- ----------
827da3e0-b340-4beb-8b8b-77d8a94c1409 "" "Contrail-0c286caa-915e-48e5-a386-50b324c22a70" 2152
31f1c529-3c00-4066-a589-34ce67fd5e55 "" "Contrail-220539f5-cee1-44f3-8c79-1a2ad56dc04d" 1809
e691aaae-e42f-4b3c-93a6-128b5421de58 "" "Contrail-25fb215b-4819-463f-be6a-0aa13c6ca984" 2041
1c6e1fd8-89f0-4752-baec-2859f409b215 "" "Contrail-37b9045c-6411-4fdc-8ec0-62accb6856a3" 2162
61f1d0a8-3889-46be-9ca0-ff45794346d5 "" "Contrail-39fb95bc-a339-4d3e-877e-34ba35c1c8a0" 1843
13f4993a-5eda-46ae-81ff-f9aa6886b74f "" "Contrail-5818adc2-1a15-4e90-a361-1ff0494f0227" 2359
cdf05c2a-7f0f-4922-a1d8-95d2d38e219f "" "Contrail-6cfdaa57-8dea-4c95-885e-3d5b87dd832c" 2008
895c18c3-3578-44e9-a0e3-939289b5a473 "" "Contrail-76ac5c31-dda6-4fae-870f-92d48f042c95" 2028
bced0b50-fbaf-4757-bb4c-0ef9532e8eaa "" "Contrail-7bab18ee-5eac-46df-9fe6-ccaa4a5dd445" 2336
9a892745-09a4-434a-8bc6-b261668e8019 "" "Contrail-8384126e-2ec8-4ab2-8e31-45ba1e87548a" 1878
6079900f-5e6a-45d7-b11f-adaab6442663 "" "Contrail-8961314e-afc6-471f-83dc-e32153440897" 2009
ca899f3d-d5df-44f8-9126-90125f0f52e4 "" "Contrail-912590b7-cdce-41ae-bc43-b8efff5504c2" 2055
c90463d7-42e7-403f-a1ac-acb743d28f12 "" "Contrail-955c489a-9712-4ee0-95ed-03021356bde6" 2046
8957219c-ed1a-4a8a-829f-4a4ccd3ef41b "" "Contrail-b573a1b8-e322-4896-85f9-33d7b428411e" 2153
6b1e1249-8d02-4bbf-aac6-da9d9906dd5e "" "Contrail-e9291384-666f-4c6a-bc23-54a396b6813f" 1815 It is so tired to confirm each table, column, value that I created python tor_tsn_compare.py
Please download three files below in https://github.com/nakadaisuke/contrail_utils/tree/master/getnhutils . The programs basically use same libraries in Contrail.
Packages
contrail_sandeshlibs.py
ovsdb_schema_reader.py
tor_tsn_compare.py
Libraries
sys
xmltodict
urllib2
socket
json
argarse
copy
How to use it
tor_tsn_compare.py -t 'IP address of TSN' -o 'IP address of ToR-Switch' -v 'verbose'
For instance, execute command below. In case of multiple TSN in HA, IP address of TSN connects with ",".
python tor_tsn_compare.py -t '172.27.113.205:9011','172.27.113.206:9011' -o 172.27.113.51:9999
With out "-v True"、Synchronized table shows as "Sync all data". Not Synchronized table shows table as is.
python tor_tsn_compare.py -t '172.27.113.205:9011','172.27.113.206:9011' -o 172.27.113.51:9999
-------- Mcast_Macs_Remote Table --------
Sync all data
-------- Ucast_Macs_local Table --------
Sync all data
-------- Mcast_Macs_Local Table --------
Sync all data
-------- Physical_Switch Table --------
Sync all data
-------- Logical_Switch Table --------
Sync all data
-------- Ucast_Macs_Remote Table --------
Sync all data
-------- Physical_Port Table --------
Sync all dataWith -v True, all of tales show（Snipped some data）
python tor_tsn_compare.py -t '172.27.113.205:9011','172.27.113.206:9011' -o 172.27.113.51:9999 -v True
-------- Mcast_Macs_Remote Table --------
[[u'In sync(4)', u'ff:ff:ff:ff:ff:ff', u'0c286caa-915e-48e5-a386-50b324c22a70', u'10.84.50.4']]
[[u'In sync(4)', u'ff:ff:ff:ff:ff:ff', u'220539f5-cee1-44f3-8c79-1a2ad56dc04d', u'10.84.50.4']]
[[u'In sync(4)', u'ff:ff:ff:ff:ff:ff', u'25fb215b-4819-463f-be6a-0aa13c6ca984', u'10.84.50.4']]
[[u'In sync(4)', u'ff:ff:ff:ff:ff:ff', u'37b9045c-6411-4fdc-8ec0-62accb6856a3', u'10.84.50.4']]
-------- Ucast_Macs_local Table --------
[[u'In sync(4)', u'22:22:22:00:00:01', u'0c286caa-915e-48e5-a386-50b324c22a70', u'10.84.51.72']]
[[u'In sync(4)', u'22:22:22:00:00:01', u'37b9045c-6411-4fdc-8ec0-62accb6856a3', u'10.84.51.72']]
[[u'In sync(4)', u'22:22:22:00:00:02', u'b573a1b8-e322-4896-85f9-33d7b428411e', u'10.84.51.72']]
[[u'In sync(4)', u'22:22:22:00:00:02', u'e9291384-666f-4c6a-bc23-54a396b6813f', u'10.84.51.72']]
-------- Mcast_Macs_Local Table --------
[[u'In sync(4)', u'ff:ff:ff:ff:ff:ff', u'0c286caa-915e-48e5-a386-50b324c22a70', u'10.84.51.72']]
[[u'In sync(4)', u'ff:ff:ff:ff:ff:ff', u'220539f5-cee1-44f3-8c79-1a2ad56dc04d', u'10.84.51.72']]
[[u'In sync(4)', u'ff:ff:ff:ff:ff:ff', u'25fb215b-4819-463f-be6a-0aa13c6ca984', u'10.84.51.72']]
[[u'In sync(4)', u'ff:ff:ff:ff:ff:ff', u'37b9045c-6411-4fdc-8ec0-62accb6856a3', u'10.84.51.72']]
[[u'In sync(4)', u'ff:ff:ff:ff:ff:ff', u'39fb95bc-a339-4d3e-877e-34ba35c1c8a0', u'10.84.51.72']]
[[u'In sync(4)', u'ff:ff:ff:ff:ff:ff', u'b573a1b8-e322-4896-85f9-33d7b428411e', u'10.84.51.72']]
[[u'In sync(4)', u'ff:ff:ff:ff:ff:ff', u'e9291384-666f-4c6a-bc23-54a396b6813f', u'10.84.51.72']]
-------- Physical_Switch Table --------
[[u'In sync(4)', u'shinjuku', u'10.84.51.72']]
-------- Logical_Switch Table --------
[[u'In sync(4)', u'0c286caa-915e-48e5-a386-50b324c22a70', u'2152']]
[[u'In sync(4)', u'220539f5-cee1-44f3-8c79-1a2ad56dc04d', u'1809']]
[[u'In sync(4)', u'912590b7-cdce-41ae-bc43-b8efff5504c2', u'2055']]
[[u'In sync(4)', u'b573a1b8-e322-4896-85f9-33d7b428411e', u'2153']]
[[u'In sync(4)', u'e9291384-666f-4c6a-bc23-54a396b6813f', u'1815']]
-------- Ucast_Macs_Remote Table --------
[[u'In sync(4)', u'02:8f:d5:8e:e6:56', u'7bab18ee-5eac-46df-9fe6-ccaa4a5dd445', u'192.168.21.1']]
[[u'In sync(4)', u'02:e4:d7:4c:50:cc', u'7bab18ee-5eac-46df-9fe6-ccaa4a5dd445', u'192.168.22.1']]
[[u'In sync(4)', u'22:22:22:00:00:01', u'8384126e-2ec8-4ab2-8e31-45ba1e87548a', u'10.84.51.73']]
[[u'In sync(4)', u'22:22:22:00:00:01', u'8961314e-afc6-471f-83dc-e32153440897', u'10.84.51.73']]
[[u'In sync(4)', u'22:22:22:00:00:01', u'912590b7-cdce-41ae-bc43-b8efff5504c2', u'10.84.51.73']]
-------- Physical_Port Table --------
[[u'In sync(4)', [[u'4001', u'25fb215b-4819-463f-be6a-0aa13c6ca984']], u'ge-0/0/10']]
[[u'In sync(4)', [[u'4001', u'955c489a-9712-4ee0-95ed-03021356bde6']], u'ge-0/0/11']]
[[u'In sync(4)', [[u'4001', u'912590b7-cdce-41ae-bc43-b8efff5504c2']], u'ge-0/0/12']]
[[u'In sync(4)', [[u'4001', u'0c286caa-915e-48e5-a386-50b324c22a70']], u'ge-0/0/13']]
[[u'In sync(4)', [[u'10', u'5818adc2-1a15-4e90-a361-1ff0494f0227'], [u'100', u'5818adc2-1a15-4e90-a361-1ff0494f0227'], [u'1000', u'220539f5-cee1-44f3-8c79-1a2ad56dc04d']], u'ge-0/0/19']]
[[u'In sync(4)', [[u'0', u'7bab18ee-5eac-46df-9fe6-ccaa4a5dd445']], u'ge-0/0/20']]
[[u'In sync(4)', [], u'ge-0/0/23']]
[[u'In sync(4)', [], u'ge-0/0/24']]
[[u'In sync(4)', [], u'ge-0/0/25']]If you have any comment or something. Please send me email or post messages to Github.
No comments:
Post a Comment