Configuring flow control on the switch

ACOS supports configuring PFC flow control based on DSCP at Layer 3 and flow control based on Global Pause on switches, and the former is highly recommended. The following instructions use Mellanox Switch as an example.

Configuring PFC flow control based on DSCP at Layer 3 (recommended)

1. Set the RoCE traffic type to 3 and turn on ECN to enable DCQCN. Set both the minimum and maximum ECN thresholds to 3000 KB.

   switch > enable
   switch # configure terminal
   switch (config) # interface ethernet 1/1-1/2 traffic-class 3 congestion-control ecn minimum-absolute 3000 maximum-absolute 3000

2. Set QoS trust mode to L3(DSCP).

   switch (config) # interface ethernet 1/1-1/2 qos trust L3

3. Enable the PFC of the switch on priority 3.

   switch (config) # dcb priority-flow-control enable force
   switch (config) # dcb priority-flow-control priority 3 enable
   switch (config) # interface ethernet 1/1-1/2 dcb priority-flow-control mode on force
   switch (config) # interface ethernet 1/1-1/2 pfc-wd

4. Confirm that PFC is enabled and save the configuration.

   switch (config) # show dcb priority-flow-control
   PFC: enabled
   Priority Enabled List: 3
   Priority Disabled List: 0 1 2 4 5 6 7   
   -------------------------------------------------
   Interface        PFC admin        PFC oper
   -------------------------------------------------
   Eth1/1           Auto             Enabled
   Eth1/2           Auto             Enabled
   ......   
   switch (config) # configuration write

Configuring flow control based on Global Pause

Set the flowcontrol of the port on the switch connected to the RDMA NIC to on, and then verify and save the configuration.

Mellanox Switch
switch > enable
switch # configure terminal
switch (config) # interface ethernet 1/1-1/2 traffic-class 0 congestion-control ecn minimum-absolute 3000 maximum-absolute 3000
switch (config) # interface ethernet 1/1-1/2 flowcontrol send on force
switch (config) # interface ethernet 1/1-1/2 flowcontrol receive on force
switch (config) # show interfaces ethernet 1/1 | include Flow-control
Flow-control                     : receive on send on
switch (config) # show interfaces ethernet 1/2 | include Flow-control
Flow-control                     : receive on send on
switch (config) # configuration write