SMC Gen5 Genoa Port Diagrams - Multi-Homed

Prev Next

⚠️ Warning

Networking is configured on a per-CNode basis, it is possible for a cluster to have CNodes with differing configurations. If you are unsure of the configuration of a particular CNode, please contact VAST Support for assistance.

ℹ️ Note

Skip NIC cannot be used with multi-homed configurations.

SMC Gen5 Genoa CBox with dual CX7 NIC in ETH mode, Multi-homed ETH + IB

⚠️ Warning

Since this is a per-CNode setting, it is possible for a cluster to have CNodes with differing configurations. If you are unsure of the configuration of a CNode, please contact VAST Support for assistance.

SMC Gen5 Genoa CBox with dual CX7 NIC in ETH mode, Multi-homed ETH + IB

SMC Gen5 Genoa CBox

Number

Name

Port Location

Function

PCIe B:D.F

1

PSU 1

2

PSU 2

3

enp129s0f1

Right Card – Left Port

Internal

81:00.1

4

enp129s0f0

Right Card – Right Port

Internal

81:00.0

5

enp1s0f1

Left Card – Left Port

Northbound/Split

6

ib0

Left Card – Right Port

Northbound/Split

7

BMC

Dedicated BMC port

(*IPMI)

8

enp65s0f0

1 GbE Port 1 (Left)

MGMT/IPMI

9

enp65s0f1

1 GbE Port 2

 (*MGMT)

10

enp65s0f3

1 GbE Port 4 (Right)

TECH

*In B2B IPMI configuration, the MGMT port is moved to 1GbE port 2 (enp65s0f1) and Port 1 (enp65s0f0) is connected to the Dedicated BMC Port of the Peer Node


SMC Gen5 Genoa CBox with dual CX7 NIC in ETH mode, Multi-homed IB + ETH

⚠️ Warning

Since this is a per-CNode setting, it is possible for a cluster to have CNodes with differing configurations. If you are unsure of the configuration of a CNode, please contact VAST Support for assistance.

SMC Gen5 Genoa CBox with dual CX7 NIC in ETH mode, Multi-homed IB + ETH

SMC Gen5 Genoa CBox

Number

Name

Port Location

Function

PCIe B:D.F

1

PSU 1

2

PSU 2

3

enp129s0f1

Right Card – Left Port

Internal

81:00.1

4

enp129s0f0

Right Card – Right Port

Internal

81:00.0

5

ib0

Left Card – Left Port

Northbound/Split

6

enp1s0f0

Left Card – Right Port

Northbound/Split

7

BMC

Dedicated BMC port

(*IPMI)

8

enp65s0f0

1 GbE Port 1 (Left)

MGMT/IPMI

9

enp65s0f1

1 GbE Port 2

 (*MGMT)

10

enp65s0f3

1 GbE Port 4 (Right)

TECH

*In B2B IPMI configuration, the MGMT port is moved to 1GbE port 2 (enp65s0f1), and Port 1 (enp65s0f0) is connected to the Dedicated BMC Port of the Peer Node


SMC Gen5 Genoa CBox with dual CX7 NIC in IB mode, Multi-homed IB + ETH

⚠️ Warning

Since this is a per-CNode setting, it is possible for a cluster to have CNodes with differing configurations. If you are unsure of the configuration of a CNode, please contact VAST Support for assistance.

SMC Gen5 Genoa CBox with dual CX7 NIC in IB mode, Multi-homed IB + ETH

SMC Gen5 Genoa CBox

Number

Name

Port Location

Function

PCIe B:D.F

1

PSU 1

2

PSU 2

3

ib1

Right Card – Left Port

Internal

81:00.1

4

ib0

Right Card – Right Port

Internal

81:00.0

5

ib2

Left Card – Left Port

Northbound/Split

6

enp1s0f0

Left Card – Right Port

Northbound/Split

7

BMC

Dedicated BMC port

(*IPMI)

8

enp65s0f0

1 GbE Port 1 (Left)

MGMT/IPMI

9

enp65s0f1

1 GbE Port 2

 (*MGMT)

10

enp65s0f3

1 GbE Port 4 (Right)

TECH

*In B2B IPMI configuration, the MGMT port is moved to 1GbE port 2 (enp65s0f1), and Port 1 (enp65s0f0) is connected to the Dedicated BMC Port of the Peer Node


SMC Gen5 Genoa CBox with dual CX7 NIC in IB mode, Multi-homed ETH + IB

⚠️ Warning

Since this is a per-CNode setting, it is possible for a cluster to have CNodes with differing configurations. If you are unsure of the configuration of a CNode, please contact VAST Support for assistance.

SMC Gen5 Genoa CBox with dual CX7 NIC in IB mode, Multi-homed ETH + IB

SMC Gen5 Genoa CBox

Number

Name

Port Location

Function

PCIe B:D.F

1

PSU 1

2

PSU 2

3

ib1

Right Card – Left Port

Internal

81:00.1

4

ib0

Right Card – Right Port

Internal

81:00.0

5

enp1s0f1

Left Card – Left Port

Northbound/Split

6

ib2

Left Card – Right Port

Northbound/Split

7

BMC

Dedicated BMC port

(*IPMI)

8

enp65s0f0

1 GbE Port 1 (Left)

MGMT/IPMI

9

enp65s0f1

1 GbE Port 2

 (*MGMT)

10

enp65s0f3

1 GbE Port 4 (Right)

TECH

*In B2B IPMI configuration, the MGMT port is moved to 1GbE port 2 (enp65s0f1), and Port 1 (enp65s0f0) is connected to the Dedicated BMC Port of the Peer Node


ℹ️ Note

Using the Reverse NIC functionality together with Multi-home is supported, but has extremely limited use cases.  Thus it is not documented here.