Difference between revisions of "Nesting NSX-T (lab) inside NSX-T (infra)"

Line 1: Line 1:
= Nesting NSX-T (lab) inside NSX-T (infra) =
 
 
 
When we build nested labs, we typically use VLANS to provide a L2 broadcast domains where we can provide networking services across.
 
When we build nested labs, we typically use VLANS to provide a L2 broadcast domains where we can provide networking services across.
 
I also did this for a long time in my home lab, when I needed a new network with a new gateway I typically configured the gateway on a routed (SVI) VLAN sub interface with a new tagged VLAN. With this I only had the gateway and then I needed to configure  the other 4 switches in my home network with this new VLAN.
 
I also did this for a long time in my home lab, when I needed a new network with a new gateway I typically configured the gateway on a routed (SVI) VLAN sub interface with a new tagged VLAN. With this I only had the gateway and then I needed to configure  the other 4 switches in my home network with this new VLAN.
Line 8: Line 6:
 
All my (nested) lab components need a network.
 
All my (nested) lab components need a network.
 
So I decided to use NSX-T Segments for these lab VM's. This all works well, but what if I want to build a nested NSX-T lab?
 
So I decided to use NSX-T Segments for these lab VM's. This all works well, but what if I want to build a nested NSX-T lab?
This means that the “GENEVE” lab network that is required to provide NSX-T *"overlay"* Networking services inside my nested lab is using an NSX-T *underlay* network as the transport layer.
+
This means that the “GENEVE” lab network that is required to provide NSX-T '''"overlay"''' Networking services inside my nested lab is using an NSX-T '''underlay''' network as the transport layer.
  
 
Yes I know …  
 
Yes I know …  
 
There is not a simple way to explain this better so I decided to illustrate this in a drawing:
 
There is not a simple way to explain this better so I decided to illustrate this in a drawing:
  
[[File:3AF8514F-F955-496F-BA44-5CBB198735D5.png]]
+
[[File:3AF8514F-F955-496F-BA44-5CBB198735D5.png|800px]]
  
 
When you look at the picture you can divide the picture into two constructs, '''INFRA''' and '''NESTED'''.
 
When you look at the picture you can divide the picture into two constructs, '''INFRA''' and '''NESTED'''.

Revision as of 18:55, 11 April 2020

When we build nested labs, we typically use VLANS to provide a L2 broadcast domains where we can provide networking services across. I also did this for a long time in my home lab, when I needed a new network with a new gateway I typically configured the gateway on a routed (SVI) VLAN sub interface with a new tagged VLAN. With this I only had the gateway and then I needed to configure the other 4 switches in my home network with this new VLAN. This took too much time and effort and since I am lazy, I did not wanted to do this anymore, so I installed NSX-v and started to play with NSX-v. Eventually I upgraded to NSX-T, I used the V2T migration coordinator but this went horribly wrong, so I did a reinstall. Bust this is another story for another time.

All my (nested) lab components need a network. So I decided to use NSX-T Segments for these lab VM's. This all works well, but what if I want to build a nested NSX-T lab? This means that the “GENEVE” lab network that is required to provide NSX-T "overlay" Networking services inside my nested lab is using an NSX-T underlay network as the transport layer.

Yes I know … There is not a simple way to explain this better so I decided to illustrate this in a drawing:

3AF8514F-F955-496F-BA44-5CBB198735D5.png

When you look at the picture you can divide the picture into two constructs, INFRA and NESTED.

  • INFRA
    • Infra switch (physical)
      • My physical switch where the infra ESXi server is connected to
      • The interfaces of this switch connecting to the infra ESXi server are trunk interfaces and allow all VLANS across the interfaces
    • infra ESXi Host (physical)
      • One of my physical ESX servers that is connected to the physical switch/
      • This physical host also host the virtual machines that will behave as nested ESXi hosts
    • infra vmnic interfaces
      • The physical NICs of the ESXi host connected to the physical switch
    • infra N-VDS -> The N-VDS offered by the NSX-T Manager that is part of my lab infrastructure
      • infra NSX-T VLAN segments -> The VLAN Segments offered by my infra NSX-T Manager
        • infra vmk interfaces -> The infra vmkernel interfaces, connected to a VLAN backed segment
      • infra NSX-T GENEVE overlay segments #1 -> The GENEVE (overlay) Segments offered by my infra NSX-T Manager
        • infra Web, App and DB VM’s -> Regular VM’s connected to NSX-T VLAN backed segments
      • infra NSX-T GENEVE overlay segments #2 -> The GENEVE (overlay) Segments offered by my infra NSX-T Manager that are going to be used as UNDERLAY for the Nested parts (explained below)
  • NESTED
    • nested ESXi Host (virtual)
      • One of my virtual nested ESX servers that is connected to the infra N-VDS switch
    • nested vmnic interfaces
      • The virtual NICs of the nested ESXi host connected to the infra N-NDS switch on an infra overlay segment
    • nested N-VDS -> The N-VDS offered by the NSX-T Manager that is part of my NESTED lab infrastructure
      • infra NSX-T GENEVE overlay segments -> The GENEVE (overlay) Segments offered by my NESTED NSX-T Manager
        • nested vmk interfaces -> The nested vmkernel interfaces, connected to a (nested) GENEVE overlay backed segment
        • nested Web, App and DB vVM’s -> Double nested vVM’s connected to NSX-T GENEVE overlay backed segment

I know this is mind-blowing and pretty challenging to write about, but I did my best and hope you understand what is going on.