SONiC and White Box switches in the Enterprise DC! – Part 2

As discussed during our part 1, we are trying to configure a VXLAN-EVPN fabric using SONiC on white box switches in order to determine if Open Networking is ready to be deployed in most enterprise DCs.

As a small Recap, below is the topology we are trying to bring online:

Familiarise with the OS

The most interesting thing of SONiC is its architecture!
I’ll write a blog just about it because it’s a fascinating topic, but in short, every single process is living inside a dedicated container.

Linux SONIC-Leaf301 4.9.0-11-2-amd64 #1 SMP Debian 4.9.189-3+deb9u2 (2019-11-11) x86_64
You are on
  ____   ___  _   _ _  ____
 / ___| / _ \| \ | (_)/ ___|
 \___ \| | | |  \| | | |
  ___) | |_| | |\  | | |___
 |____/ \___/|_| \_|_|\____|

-- Software for Open Networking in the Cloud --

Unauthorized access and/or use are prohibited.
All access and/or use are subject to monitoring.

Help:    http://azure.github.io/SONiC/

Last login: Thu Apr 20 12:52:21 2017 from 192.168.0.31
admin@SONIC-Leaf301:~$ show version 

SONiC Software Version: SONiC-OS-3.0.1-Enterprise_Advanced
Product: Enterprise Advanced SONiC OS - Powered by Broadcom
Distribution: Debian 9.12
Kernel: 4.9.0-11-2-amd64
Build commit: aff85dcf1
Build date: Mon Apr 20 09:48:32 UTC 2020
Built by: sonicbld@sonic-lvn-csg-004

Platform: x86_64-accton_as7326_56x-r0
HwSKU: Accton-AS7326-56X
ASIC: broadcom
Serial Number: 732656X1916020
Uptime: 13:04:04 up 12 days, 29 min,  1 user,  load average: 2.14, 2.46, 2.42

Docker images:
REPOSITORY                       TAG                         IMAGE ID            SIZE
docker-sonic-telemetry           3.0.1-Enterprise_Advanced   15838a6dd8b6        397MB
docker-sonic-telemetry           latest                      15838a6dd8b6        397MB
docker-sonic-mgmt-framework      3.0.1-Enterprise_Advanced   c30542f39b9c        445MB
docker-sonic-mgmt-framework      latest                      c30542f39b9c        445MB
docker-swss-brcm-ent-advanced    3.0.1-Enterprise_Advanced   7db851611618        338MB
docker-swss-brcm-ent-advanced    latest                      7db851611618        338MB
docker-broadview                 3.0.1-Enterprise_Advanced   547641c0c886        330MB
docker-broadview                 latest                      547641c0c886        330MB
docker-nat                       3.0.1-Enterprise_Advanced   b819906b8bb6        320MB
docker-nat                       latest                      b819906b8bb6        320MB
docker-vrrp                      3.0.1-Enterprise_Advanced   2f0615d57ea4        333MB
docker-vrrp                      latest                      2f0615d57ea4        333MB
docker-teamd                     3.0.1-Enterprise_Advanced   3487700dc8d2        318MB
docker-teamd                     latest                      3487700dc8d2        318MB
docker-fpm-frr                   3.0.1-Enterprise_Advanced   bf6e7649147e        367MB
docker-fpm-frr                   latest                      bf6e7649147e        367MB
docker-iccpd                     3.0.1-Enterprise_Advanced   1c24858c993b        320MB
docker-iccpd                     latest                      1c24858c993b        320MB
docker-l2mcd                     3.0.1-Enterprise_Advanced   b0f6db69227b        319MB
docker-l2mcd                     latest                      b0f6db69227b        319MB
docker-stp                       3.0.1-Enterprise_Advanced   c812baaadda5        316MB
docker-stp                       latest                      c812baaadda5        316MB
docker-udld                      3.0.1-Enterprise_Advanced   66c2afbe849a        316MB
docker-udld                      latest                      66c2afbe849a        316MB
docker-sflow                     3.0.1-Enterprise_Advanced   9cf4e8a00ff9        318MB
docker-sflow                     latest                      9cf4e8a00ff9        318MB
docker-dhcp-relay                3.0.1-Enterprise_Advanced   5217cd436c40        326MB
docker-dhcp-relay                latest                      5217cd436c40        326MB
docker-syncd-brcm-ent-advanced   3.0.1-Enterprise_Advanced   800a3fc3af8b        439MB
docker-syncd-brcm-ent-advanced   latest                      800a3fc3af8b        439MB
docker-lldp-sv2                  3.0.1-Enterprise_Advanced   3a2e52d444f9        309MB
docker-lldp-sv2                  latest                      3a2e52d444f9        309MB
docker-snmp-sv2                  3.0.1-Enterprise_Advanced   d5a8e1d0ba7d        342MB
docker-snmp-sv2                  latest                      d5a8e1d0ba7d        342MB
docker-tam                       3.0.1-Enterprise_Advanced   272eabe18352        361MB
docker-tam                       latest                      272eabe18352        361MB
docker-pde                       3.0.1-Enterprise_Advanced   6ff2567c42b8        495MB
docker-pde                       latest                      6ff2567c42b8        495MB
docker-platform-monitor          3.0.1-Enterprise_Advanced   0b22d6abcd9a        367MB
docker-platform-monitor          latest                      0b22d6abcd9a        367MB
docker-router-advertiser         3.0.1-Enterprise_Advanced   9d201b15eae3        288MB
docker-router-advertiser         latest                      9d201b15eae3        288MB
docker-database                  3.0.1-Enterprise_Advanced   fb46e0661772        288MB
docker-database                  latest                      fb46e0661772        288MB

Below is the list of interfaces on my leaf. Notice how the naming of such interfaces can be confusing, specifically for the ones that can be channelised (like 40/100Gbps interfaces which support breakout). The primary channel is used as the interface number like with interface Ethernet48. In case an interface is then broken out, the other channels will be listed as Ethernet49, 50 and 51, making the next physical interface Ethernet52.
Interface Aliases are really interesting, unfortunately they currently act more like a description and even switching to “alias” as the default interface name, such alias is used in very few places making it pretty much useless as of now.

admin@SONIC-Leaf301:~$ show interfaces status 
  Interface            Lanes    Speed    MTU             Alias    Vlan    Oper    Admin             Type    Asym PFC
-----------  ---------------  -------  -----  ----------------  ------  ------  -------  ---------------  ----------
  Ethernet0                3      25G   9100   twentyFiveGigE1  routed    down     down   SFP/SFP+/SFP28         N/A
  Ethernet1                2      25G   9100   twentyFiveGigE2  routed    down     down   SFP/SFP+/SFP28         N/A
  Ethernet2                4      25G   9100   twentyFiveGigE3  routed    down     down              N/A         N/A
  Ethernet3                8      25G   9100   twentyFiveGigE4  routed    down     down              N/A         N/A
  Ethernet4                7      25G   9100   twentyFiveGigE5  routed    down     down              N/A         N/A
  Ethernet5                1      25G   9100   twentyFiveGigE6  routed    down     down              N/A         N/A
  Ethernet6                5      25G   9100   twentyFiveGigE7  routed    down     down              N/A         N/A
  Ethernet7               16      25G   9100   twentyFiveGigE8  routed    down     down              N/A         N/A
  Ethernet8                6      25G   9100   twentyFiveGigE9  routed    down     down              N/A         N/A
  Ethernet9               14      25G   9100  twentyFiveGigE10  routed    down     down   SFP/SFP+/SFP28         N/A
 Ethernet10               13      25G   9100  twentyFiveGigE11  routed    down     down              N/A         N/A
 Ethernet11               15      25G   9100  twentyFiveGigE12  routed    down     down              N/A         N/A
 Ethernet12               23      25G   9100  twentyFiveGigE13  routed    down     down              N/A         N/A
 Ethernet13               22      25G   9100  twentyFiveGigE14  routed    down     down              N/A         N/A
 Ethernet14               24      25G   9100  twentyFiveGigE15  routed    down     down              N/A         N/A
 Ethernet15               32      25G   9100  twentyFiveGigE16  routed    down     down              N/A         N/A
 Ethernet16               31      25G   9100  twentyFiveGigE17  routed    down     down              N/A         N/A
 Ethernet17               21      25G   9100  twentyFiveGigE18  routed    down     down              N/A         N/A
 Ethernet18               29      25G   9100  twentyFiveGigE19  routed    down     down              N/A         N/A
 Ethernet19               36      25G   9100  twentyFiveGigE20  routed    down     down              N/A         N/A
 Ethernet20               30      25G   9100  twentyFiveGigE21  routed    down     down              N/A         N/A
 Ethernet21               34      25G   9100  twentyFiveGigE22  routed    down     down              N/A         N/A
 Ethernet22               33      25G   9100  twentyFiveGigE23  routed    down     down              N/A         N/A
 Ethernet23               35      25G   9100  twentyFiveGigE24  routed    down     down              N/A         N/A
 Ethernet24               43      25G   9100  twentyFiveGigE25  routed    down     down              N/A         N/A
 Ethernet25               42      25G   9100  twentyFiveGigE26  routed    down     down              N/A         N/A
 Ethernet26               44      25G   9100  twentyFiveGigE27  routed    down     down              N/A         N/A
 Ethernet27               52      25G   9100  twentyFiveGigE28  routed    down     down              N/A         N/A
 Ethernet28               51      25G   9100  twentyFiveGigE29  routed    down     down              N/A         N/A
 Ethernet29               41      25G   9100  twentyFiveGigE30  routed    down     down              N/A         N/A
 Ethernet30               49      25G   9100  twentyFiveGigE31  routed    down     down              N/A         N/A
 Ethernet31               60      25G   9100  twentyFiveGigE32  routed    down     down              N/A         N/A
 Ethernet32               50      25G   9100  twentyFiveGigE33  routed    down     down              N/A         N/A
 Ethernet33               58      25G   9100  twentyFiveGigE34  routed    down     down              N/A         N/A
 Ethernet34               57      25G   9100  twentyFiveGigE35  routed    down     down              N/A         N/A
 Ethernet35               59      25G   9100  twentyFiveGigE36  routed    down     down              N/A         N/A
 Ethernet36               62      25G   9100  twentyFiveGigE37  routed    down     down              N/A         N/A
 Ethernet37               63      25G   9100  twentyFiveGigE38  routed    down     down              N/A         N/A
 Ethernet38               64      25G   9100  twentyFiveGigE39  routed    down     down              N/A         N/A
 Ethernet39               65      25G   9100  twentyFiveGigE40  routed    down     down              N/A         N/A
 Ethernet40               66      25G   9100  twentyFiveGigE41  routed    down     down              N/A         N/A
 Ethernet41               61      25G   9100  twentyFiveGigE42  routed    down     down              N/A         N/A
 Ethernet42               68      25G   9100  twentyFiveGigE43  routed    down     down              N/A         N/A
 Ethernet43               69      25G   9100  twentyFiveGigE44  routed    down     down              N/A         N/A
 Ethernet44               67      25G   9100  twentyFiveGigE45  routed    down     down              N/A         N/A
 Ethernet45               71      25G   9100  twentyFiveGigE46  routed    down     down              N/A         N/A
 Ethernet46               72      25G   9100  twentyFiveGigE47  routed    down     down              N/A         N/A
 Ethernet47               70      25G   9100  twentyFiveGigE48  routed    down     down              N/A         N/A
 Ethernet48      77,78,79,80     100G   9100     hundredGigE49  routed    down     down  QSFP28 or later         N/A
 Ethernet52      85,86,87,88     100G   9100     hundredGigE50  routed    down     down  QSFP28 or later         N/A
 Ethernet56      93,94,95,96     100G   9100     hundredGigE51  routed    down     down              N/A         N/A
 Ethernet60     97,98,99,100     100G   9100     hundredGigE52  routed    down     down              N/A         N/A
 Ethernet64  105,106,107,108     100G   9100     hundredGigE53  routed    down     down              N/A         N/A
 Ethernet68  113,114,115,116     100G   9100     hundredGigE54  routed    down     down              N/A         N/A
 Ethernet72  121,122,123,124     100G   9100     hundredGigE55  routed    down     down  QSFP28 or later         N/A
 Ethernet76  125,126,127,128     100G   9100     hundredGigE56  routed    down     down  QSFP28 or later         N/A
 Ethernet80              129      10G   9100     mgmtTenGigE57  routed    down     down              N/A         N/A
 Ethernet81              128      10G   9100     mgmtTenGigE58  routed    down     down              N/A         N/

Configuring the Underlay Routing

I’m a big fan of automation and configuration simplicity. I strongly believe that if i can automate with “notepad” using blind copy/paste i have good templates for fancier automation. For this reason i really think that the use of unnumbered interfaces are a great way to configure spine/leaf interfaces.

First step then, is to configure all fabric interfaces with a proper MTU and ip unnumbered as follows in the example below. Please notice that this post isn’t meant to be a full configuration tutorial.

config loopback add Loopback0
config interface ip add Loopback0 10.0.0.1/32
 
config interface ip unnumbered add Ethernet120 Loopback0
config interface mtu Ethernet120 9216
config interface startup Ethernet120
 
... Repeat for all interfaces facing a leaf ...
config save -y

A leaf switch would be configured exactly the same way, but I need also to add a second loopback interface to be used as VTEP source interface. As this loopback will act as an MC-LAG Anycast IP, both leafs in MC-LAG will have the same exact IP on their loopback 0

config loopback add Loopback0
config interface ip add Loopback0 10.0.0.11/32
config loopback add Loopback1
config interface ip add Loopback1 11.11.11.111/32

config interface ip unnumbered add Ethernet72 Loopback0
config interface mtu Ethernet72 9216
config interface description Ethernet72 "LINK_TO_SPINE_1"
config interface startup Ethernet72
  
config interface ip unnumbered add Ethernet76 Loopback0
config interface mtu Ethernet76 9216
config interface description Ethernet76 "LINK_TO_SPINE_2"
config interface startup Ethernet76
config save -y

At this point we need to configure OSPF between leafs and spines.
Unfortunately, advanced routing configs can only be applied inside the FRR container, so we need to switch to the FRR shell using the command “vtysh” first. From there on, really there is almost no difference with the well known cisco-like cli.

The biggest downside of this lack of integration is that FRR config needs to be saved separately from the rest of SONiC’s config, and we also need to tell SONiC to look at the routing config in a different place from the rest. To do that, we also need to apply the “config routing_config_mode split” command and most importantly, you need to reboot the box as the warning message will tell you. Failure to do so, will cause the switch to loose the FRR config in case of reload.

vtysh
 conf t
  !
  bfd
  !
  router ospf
   ospf router-id 10.0.0.11
   log-adjacency-changes
   auto-cost reference-bandwidth 100000
  !
  interface Ethernet72
   ip ospf area 0.0.0.1
   ip ospf bfd
   ip ospf network point-to-point
  !
  interface Ethernet76
   ip ospf area 0.0.0.1
   ip ospf bfd
   ip ospf network point-to-point
  !
  interface Loopback0
   ip ospf area 0.0.0.1
  !
  interface Loopback1
   ip ospf area 0.0.0.1
  end
 write memory
 exit
config routing_config_mode split
config save -y

Once everything is configured, from FRR we can check our routing:

SONIC-Leaf301# show ip ospf neighbor 

Neighbor ID     Pri State           Dead Time Address         Interface            RXmtL RqstL DBsmL
10.0.0.1          1 Full/DROther      33.775s 10.0.0.1        Ethernet72:10.0.0.11     0     0     0
10.0.0.2          1 Full/DROther      33.968s 10.0.0.2        Ethernet76:10.0.0.11     0     0     0

SONIC-Leaf301# show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued route, r - rejected route, # - not installed in hardware

O>*  10.0.0.1/32 [110/11] via 10.0.0.1, Ethernet72 onlink, 00:03:08
O>*  10.0.0.2/32 [110/11] via 10.0.0.2, Ethernet76 onlink, 00:03:18
C *  10.0.0.11/32 is directly connected, Ethernet76, 00:05:08
C *  10.0.0.11/32 is directly connected, Ethernet72, 00:05:08
O    10.0.0.11/32 [110/10] via 0.0.0.0, Loopback0 onlink, 00:05:14
C>*  10.0.0.11/32 is directly connected, Loopback0, 00:05:15
O>*  10.0.0.12/32 [110/12] via 10.0.0.1, Ethernet72 onlink, 00:03:08
  *                        via 10.0.0.2, Ethernet76 onlink, 00:03:08
O>*  10.0.0.13/32 [110/12] via 10.0.0.1, Ethernet72 onlink, 00:03:08
  *                        via 10.0.0.2, Ethernet76 onlink, 00:03:08
O>*  10.0.0.14/32 [110/12] via 10.0.0.1, Ethernet72 onlink, 00:03:08
  *                        via 10.0.0.2, Ethernet76 onlink, 00:03:08
O>*  10.10.10.2/31 [110/12] via 10.0.0.1, Ethernet72 onlink, 00:03:08
  *                         via 10.0.0.2, Ethernet76 onlink, 00:03:08
O    11.11.11.111/32 [110/10] via 0.0.0.0, Loopback1 onlink, 00:05:14
C>*  11.11.11.111/32 is directly connected, Loopback1, 00:05:15
O>*  11.11.11.113/32 [110/12] via 10.0.0.1, Ethernet72 onlink, 00:03:08
  *                           via 10.0.0.2, Ethernet76 onlink, 00:03:08

SONIC-Leaf301# ping 10.0.0.1
PING 10.0.0.1 (10.0.0.1) 56(84) bytes of data.
64 bytes from 10.0.0.1: icmp_seq=1 ttl=64 time=0.243 ms
64 bytes from 10.0.0.1: icmp_seq=2 ttl=64 time=0.186 ms
^C
--- 10.0.0.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1011ms
rtt min/avg/max/mdev = 0.186/0.214/0.243/0.032 ms
SONIC-Leaf301# ping 10.0.0.2
PING 10.0.0.2 (10.0.0.2) 56(84) bytes of data.
64 bytes from 10.0.0.2: icmp_seq=1 ttl=64 time=0.215 ms
^C
--- 10.0.0.2 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.215/0.215/0.215/0.000 ms

admin@SONIC-Leaf301:~$ traceroute 10.0.0.13
traceroute to 10.0.0.13 (10.0.0.13), 30 hops max, 60 byte packets
1 10.0.0.1 (10.0.0.1) 0.218 ms 10.0.0.2 (10.0.0.2) 0.178 ms 10.0.0.1 (10.0.0.1) 0.126 ms
2 10.0.0.13 (10.0.0.13) 0.439 ms 0.461 ms 0.468 ms

Now that every loopback is reachable (and we see also ECMP across the two spines) is time to configure MC-LAG between or leafs as well as underlay routing across the peer-link. This step can be done successfully only now because the MC-LAG peer has to be reachable via the fabric.

config portchannel add PortChannel1       
config interface mtu PortChannel1 9216

config interface mtu Ethernet48 9216
config interface description Ethernet48 "Peer-link"
config interface startup Ethernet48

config interface mtu Ethernet52 9216
config interface description Ethernet52 "Peer-link"
config interface startup Ethernet52

config portchannel member add PortChannel1 Ethernet48
config portchannel member add PortChannel1 Ethernet52
config mclag add 1 10.0.0.11 10.0.0.12 PortChannel1
 
config vlan add 3965
config vlan member add 3965 PortChannel1
config mclag unique-ip add Vlan3965
config interface ip add Vlan3965 10.10.10.0/31

vtysh
 conf t
  !
  interface Vlan3965
   ip ospf area 0.0.0.1
  end
 write memory
 exit
config save -y

Once done, we should see our additional OSPF peer and a working MC-LAG cluster

admin@SONIC-Leaf301:~$ vtysh 

Hello, this is FRRouting (version 7.2-sonic).
Copyright 1996-2005 Kunihiro Ishiguro, et al.

SONIC-Leaf301# show ip ospf neighbor 

Neighbor ID     Pri State           Dead Time Address         Interface            RXmtL RqstL DBsmL
10.0.0.1          1 Full/DROther      30.645s 10.0.0.1        Ethernet72:10.0.0.11     0     0     0
10.0.0.2          1 Full/DROther      30.899s 10.0.0.2        Ethernet76:10.0.0.11     0     0     0
10.0.0.12         1 Full/DR           36.717s 10.10.10.1      Vlan3965:10.10.10.0      0     0     0

SONIC-Leaf301# exit

admin@SONIC-Leaf301:~$ sonic-cli 
SONIC-Leaf301# show mclag brief
 
Domain ID            : 1
Role                 : active
Session Status       : up
Peer Link Status     : up
Source Address       : 10.0.0.11
Peer Address         : 10.0.0.12
Peer Link            : PortChannel1
Keepalive Interval   : 1 secs
Session Timeout      : 30 secs
System Mac           : 80:a2:35:81:dd:f0
 
 
Number of MLAG Interfaces:0

Everything works as expected, but we also faced yet another SONiC problem. We needed to configure interfaces and their IP addresses, OSPF and MC-LAG, to do this, we needed access to 3 different configuration shells (Linux CLI, VTYSH and sonic-cli) either for configuration or to be able to run show commands and verify our configs.

Configuring BGP-EVPN control plane

Now it’s time to configure BGP. As per our architecture, I’ll be configuring iBGP with Route Reflectors sitting on the Spines. To do so i’ll need FRR shell.
The spine config will look something similar to this one:

vtysh
 conf t
  !
  router bgp 65000
   bgp router-id 10.0.0.1
   bgp log-neighbor-changes
   neighbor FABRIC peer-group
   neighbor FABRIC remote-as 65000
   neighbor FABRIC update-source Loopback0
   bgp listen range 10.0.0.0/24 peer-group FABRIC
  !
  address-family l2vpn evpn
   neighbor FABRIC activate
   neighbor FABRIC route-reflector-client
   advertise-all-vni
   exit-address-family
  end
 exit

And the leafs, like this:

vtysh
 conf t
  !
  router bgp 65000
   bgp router-id 10.0.0.11
   bgp log-neighbor-changes
   neighbor 10.0.0.1 remote-as 65000
   neighbor 10.0.0.1 update-source Loopback0
   neighbor 10.0.0.2 remote-as 65000
   neighbor 10.0.0.2 update-source Loopback0
  !
  address-family l2vpn evpn
   neighbor 10.0.0.1 activate
   neighbor 10.0.0.2 activate
   advertise-all-vni
   advertise ipv4 unicast
  exit-address-family
  end
 exit

Once done, I should be able to see all peering formed on my spines:

SONIC-Spine31# show bgp l2vpn evpn summary
BGP router identifier 10.0.0.1, local AS number 65000 vrf-id 0
BGP table version 0
RIB entries 16, using 3072 bytes of memory
Peers 4, using 82 KiB of memory
Peer groups 1, using 64 bytes of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
*10.0.0.11 4 65000 5 29 0 0 0 00:01:33 0
*10.0.0.12 4 65000 6 30 0 0 0 00:01:34 0
*10.0.0.13 4 65000 7 31 0 0 0 00:01:40 0
*10.0.0.14 4 65000 7 31 0 0 0 00:01:44 0
Total number of neighbors 4
- dynamic neighbor
4 dynamic neighbor(s), limit 100
Total number of neighbors established

At this point, the only missing piece is to configure the VTEP on the leaf switches as well as the anycast gateway’s mac-address; fortunately this is very simple and straight forward:

config vxlan add nve1 11.11.11.111
config vxlan evpn_nvo add nvo1 nve1
config ip anycast-mac-address add aa:aa:bb:bb:cc:cc
root@SONIC-Leaf301:/home/admin# show vxlan interface           
VTEP Information:

        VTEP Name : nve1, SIP  : 11.11.11.111
        NVO Name  : nvo1,  VTEP : nve1
        Source interface  : Loopback1

root@SONIC-Leaf301:/home/admin# show ip static-anycast-gateway 
Configured Anycast Gateway MAC address: aa:aa:bb:bb:cc:cc
IPv4 Anycast Gateway MAC address: enable

In short…

We configured a fully functional fabric providing underlay connectivity and EVPNcontrol plane as follows:

  1. A unique loopbacks on every switch
  2. Each physical interface between spine and leaf as an ip unnumbered interface
  3. OSPF area 1 within the fabric
  4. MC-LAG and underlay peering across the peer-link
  5. iBGP EVPN between leaf and spines with RR on the spines themeselves
  6. Each MC-LAG pair as a unique Virtual VTEP.

We also noticed how while the configuration isn’t complicated by any mean, the need to move between multiple shells just to apply or verify configs can be very confusing to the end user. To be fair though, the SONiC community is working on improving this part by working on delivering a single unified shell.

FRR config is always very familiar as it resembles Cisco’s IOS cli; on the other end the basic Sonic CLI can be a bit frustrating at times, especially due to the fact that it’s case sensitive making typos easy to occur.

In the next blog post we will look how to actually configure VXLANs and server facing interfaces… stay tuned!

1 thought on “SONiC and White Box switches in the Enterprise DC! – Part 2

  1. Pingback: SONiC and White Box switches in the Enterprise DC! – Part 3 | Andrea Florio's Networking Blog

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s