Configuring flow control on the ESXi host

ACOS supports configuring flow control for RDMA of the storage network on hosts through the following two methods: flow control based on DSCP at Layer 3 and flow control based on Global Pause, with the former being recommended.

Precaution

The flow control configuration method used on the host and the switch must be the same.

Configuring flow control based on DSCP at Layer 3

1. Configure the parameters of the Mellanox Driver, enable PFC, and set the Trust level to DSCP. The configuration of max_vfs must be consistent with that in step 2 of the Configuring the driver to enable SR-IOV section. Replace <value> in the following command according to the actual configuration.

   # esxcli system module parameters set -m nmlx5_core -p "pfctx=0x08 pfcrx=0x08 trust_state=2 max_vfs=<value>"
   # esxcli system module parameters set -m nmlx5_rdma -p "dscp_force=26"

2. Enable maintenance mode on the ESXi host.

3. Restart the ESXi host.

4. Disable maintenance mode on the ESXi host.

5. Check and confirm that the flow control configuration is successful.

   # esxcfg-module -g nmlx5_core
   nmlx5_core enabled = 1 options = 'pfctx=0x08 pfcrx=0x08 trust_state=2 max_vfs=<value>'
   # esxcfg-module -g nmlx5_rdma
   nmlx5_rdma enabled = 1 options = 'dscp_force=26'

   The output results should show that <value> corresponds to the value in step 1.

Configuring flow control based on Global Pause

In general, as long as the Mellanox Driver does not have the pfctx and pfcrx parameters configured, the network port will enable Global-Pause flow control by default.

Run the following command to check if Global Pause is enabled on the network port. In the returned result, check the values of the Pause RX and Pause TX columns of the corresponding vmnic.

# esxcli network nic pauseParams list
NIC     Pause Params Supported  Pause RX  Pause TX  Auto Negotiation  Auto Negotiation Resolution Avail  RX Auto Negotiation Resolution  TX Auto Negotiation Resolution
------  ----------------------  --------  --------  ----------------  ---------------------------------  ------------------------------  ------------------------------
vmnic1                    true      true      true             false                              false                           false                           false
vmnic2                    true      true      true             false                              false                           false                           false

• If the returned value is true, it indicates that the Global-Pause flow control has been enabled on that network port.

• If the returned value is false, run the following command to configure the pause params for the network ports of the Mellanox NIC and enable TX and RX. vmnicX is the name of the configured network port.

  # esxcli network nic pauseParams set -t 1 -r 1 -n vmnicX