CL Port Diagrams - Mixed Ethernet + InfiniBand

Prev Next

ℹ️ Info

The Green Port - the leftmost RJ45 on the Cascade Lake CNode (The Green Port) is “reserved” and is not used in a VAST Deployment.

⚠️ Warning

Networking is configured on a per-CNode basis, it is possible for a chassis to have CNodes with differing configurations. If you are unsure of the configuration of a particular CNode, please contact VAST Support for assistance.

ℹ️ Note

Skip NIC cannot be used with mixed Ethernet + IB configurations.

Cascade Lake CBox with Dual CX5 or CX5+CX6 NIC in ETH + IB mode

Cascade Lake CBox with Dual CX5 or CX5+CX6 NIC in ETH + IB mode

Cascade Lake CBox

Number

Name

Port Location

Function

PCIe B:D.F

1

CNode 1

2

CNode 2

3

CNode 3

4

CNode 4

5

PSU 1

6

PSU 2

7

enp94s0f0

Right Card – Left Port

Internal

5e:00.0

8

enp94s0f1

Right Card – Right Port

Internal

5e:00.1

9

ib1

Left NIC - Left Port

Northbound / Split

3b:00.1

10

ib0

Left NIC - Right Port

Northbound / Split

3b:00.0

11

eno1

1 GbE Port 1 (Left)

MGMT/IPMI (*TECH)

12

eno2

1 GbE Port 2 (Right)

TECH (*MGMT)

*in B2B IPMI configuration the MGMT port is moved to 1GbE port 2 (eno2) and the TECH port is moved to 1GbE port 1 (eno1).


Cascade Lake CBox with Dual CX5 or CX5+CX6 NIC in IB + ETH mode

Cascade Lake CBox with Dual CX5 or CX5+CX6 NIC in IB + ETH mode

Cascade Lake CBox

Number

Name

Port Location

Function

PCIe B:D.F

1

CNode 1

2

CNode 2

3

CNode 3

4

CNode 4

5

PSU 1

6

PSU 2

7

ib0

Right Card – Left Port

Internal

5e:00.0

8

ib1

Right Card – Right Port

Internal

5e:00.1

9

enp59s0f1

Left NIC - Left Port

Northbound / Split

3b:00.1

10

enp59s0f0

Left NIC - Right Port

Northbound / Split

3b:00.0

11

eno1

1 GbE Port 1 (Left)

MGMT/IPMI (*TECH)

12

eno2

1 GbE Port 2 (Right)

TECH (*MGMT)

*in B2B IPMI configuration the MGMT port is moved to 1GbE port 2 (eno2) and the TECH port is moved to 1GbE port 1 (eno1).


Cascade Lake CBox with CX5+CX6 NIC in ETH + IB mode with “Reverse NICs” enabled

ℹ️ Info

The “Reverse NICs” flag is a setting that can be set either in the VAST GUI Installer or via the command line configure_network.py tool.  This flag is set on a per-CNode basis.

⚠️ Warning

Since this is a per-CNode setting, it is possible for a chassis to have CNodes with differing configurations. If you are unsure of the configuration of a CNode, please contact VAST Support for assistance.

The “Reverse NICs” flag refers to a specific configuration that swaps the roles between the NIC  intended for northband traffic and the NIC intended for internal traffic. This is primarily used in scenarios where the physical cabling or hardware layout necessitates a reversal of these roles.

Cascade Lake CBox with CX5+CX6 NIC in ETH + IB mode with “Reverse NICs” enabled

Cascade Lake CBox

Number

Name

Port Location

Function

PCIe B:D.F

1

CNode 1

2

CNode 2

3

CNode 3

4

CNode 4

5

PSU 1

6

PSU 2

7

enp94s0f0

Right Card – Left Port

Northbound / Split

5e:00.0

8

enp94s0f1

Right Card – Right Port

Northbound / Split

5e:00.1

9

ib1

Left NIC - Left Port

Internal

3b:00.1

10

ib0

Left NIC - Right Port

Internal

3b:00.0

11

eno1

1 GbE Port 1 (Left)

MGMT/IPMI (*TECH)

12

eno2

1 GbE Port 2 (Right)

TECH (*MGMT)

*in B2B IPMI configuration the MGMT port is moved to 1GbE port 2 (eno2) and the TECH port is moved to 1GbE port 1 (eno1).


Cascade Lake CBox with CX5+CX6 NIC in IB + ETH mode with “Reverse NICs” enabled

ℹ️ Info

The “Reverse NICs” flag is a setting that can be set either in the VAST GUI Installer or via the command line configure_network.py tool.  This flag is set on a per-CNode basis.

⚠️ Warning

Since this is a per-CNode setting, it is possible for a chassis to have CNodes with differing configurations. If you are unsure of the configuration of a CNode, please contact VAST Support for assistance.

The “Reverse NICs” flag refers to a specific configuration that swaps the roles between the NIC  intended for northband traffic and the NIC intended for internal traffic. This is primarily used in scenarios where the physical cabling or hardware layout necessitates a reversal of these roles.

Cascade Lake CBox with CX5+CX6 NIC in IB + ETH mode with “Reverse NICs” enabled

Cascade Lake CBox

Number

Name

Port Location

Function

PCIe B:D.F

1

CNode 1

2

CNode 2

3

CNode 3

4

CNode 4

5

PSU 1

6

PSU 2

7

ib0

Right Card – Left Port

Northbound / Split

5e:00.0

8

ib1

Right Card – Right Port

Northbound / Split

5e:00.1

9

enp59s0f1

Left NIC - Left Port

Internal

3b:00.1

10

enp59s0f0

Left NIC - Right Port

Internal

3b:00.0

11

eno1

1 GbE Port 1 (Left)

MGMT/IPMI (*TECH)

12

eno2

1 GbE Port 2 (Right)

TECH (*MGMT)

*in B2B IPMI configuration the MGMT port is moved to 1GbE port 2 (eno2) and the TECH port is moved to 1GbE port 1 (eno1).