안녕하세요,
RDMA(Roce v2) + pNFS 구성하기 위해 Cisco 구성 작업중인데, RDMA에서 pNFS가 정상적으로 작동되지 않아 문의 드리네요.
제가 한 네트워크 설정이 맞을까요 ?
조언을 부탁드립니다.
시스코 :DSCP → qos-group → Network-QoS(PFC/MTU) → Forwarding → Egress Queue(q3/q7/default) → Congestion(ECN + PFC)
서버 : MLNX Firmware Controller, prio 3, Trust dscp, cma_roce_tos 104
ervice_cs3# show running-config
!Command: show running-config
!No configuration change since last restart
!Time: Wed Mar 25 02:47:05 2026
version 10.5(4) Bios:version 05.51
hostname service_cs3
policy-map type network-qos qos_network
class type network-qos c-8q-nq3
mtu 4200
pause pfc-cos 3
class type network-qos c-8q-nq-default
mtu 9216
vdc service_cs3 id 1
limit-resource vlan minimum 16 maximum 4094
limit-resource vrf minimum 2 maximum 4096
limit-resource port-channel minimum 0 maximum 511
limit-resource m4route-mem minimum 58 maximum 58
limit-resource m6route-mem minimum 8 maximum 8
feature lldp
username admin password 5 $5$GANDCD$5ia9QbOrL.oMtcVjRmnDFmUpU.7RHvDHJHa05.hSYo. role network-admin
ip domain-lookup
class-map type qos match-any CNP
match dscp 48
class-map type qos match-any ROCEv2
match dscp 26
policy-map type qos QOS_CLASSIFICATION
class ROCEv2
set qos-group 3
class CNP
set qos-group 7
class class-default
set qos-group 0
policy-map type queuing QOS_EGRESS_PORT
class type queuing c-out-8q-q6
bandwidth remaining percent 0
class type queuing c-out-8q-q5
bandwidth remaining percent 0
class type queuing c-out-8q-q4
bandwidth remaining percent 0
class type queuing c-out-8q-q3
bandwidth remaining percent 50
random-detect minimum-threshold 150 kbytes maximum-threshold 3000 kbytes drop-probability 7 weight 0 ecn
class type queuing c-out-8q-q-default
bandwidth remaining percent 50
class type queuing c-out-8q-q2
bandwidth remaining percent 0
class type queuing c-out-8q-q1
bandwidth remaining percent 0
class type queuing c-out-8q-q7
priority level 1
system qos
service-policy type queuing output QOS_EGRESS_PORT
copp profile strict
snmp-server user admin network-admin auth md5 205EF346B5411B194B3283D7075ED3B91318 priv aes-128 0040EE43A90C494713409DCC5A4C8DA8064B localizedV2key
rmon event 1 log trap public description FATAL(1) owner PMON@FATAL
rmon event 2 log trap public description CRITICAL(2) owner PMON@CRITICAL
rmon event 3 log trap public description ERROR(3) owner PMON@ERROR
rmon event 4 log trap public description WARNING(4) owner PMON@WARNING
rmon event 5 log trap public description INFORMATION(5) owner PMON@INFO
vlan 1
vrf context management
priority-flow-control watch-dog-interval on
hardware qos nodrop-PG-thresholds PG-size 30000
interface Ethernet1/1
switchport
priority-flow-control mode on
priority-flow-control watch-dog-interval on
mtu 9216
speed 100000
fec rs-fec
no negotiate auto
service-policy type qos input QOS_CLASSIFICATION
no shutdown
interface Ethernet1/2
switchport
priority-flow-control mode on
priority-flow-control watch-dog-interval on
mtu 9216
speed 100000
fec rs-fec
no negotiate auto
service-policy type qos input QOS_CLASSIFICATION
no shutdown
interface Ethernet1/3
switchport
priority-flow-control mode on
priority-flow-control watch-dog-interval on
mtu 9216
speed 100000
fec rs-fec
no negotiate auto
service-policy type qos input QOS_CLASSIFICATION
no shutdown
interface Ethernet1/4
switchport
priority-flow-control mode on
priority-flow-control watch-dog-interval on
mtu 9216
speed 100000
fec rs-fec
no negotiate auto
service-policy type qos input QOS_CLASSIFICATION
no shutdown
interface Ethernet1/5
priority-flow-control mode on
priority-flow-control watch-dog-interval on
mtu 9216
fec rs-fec
service-policy type qos input QOS_CLASSIFICATION
no shutdown
interface Ethernet1/6
priority-flow-control mode on
priority-flow-control watch-dog-interval on
mtu 9216
fec rs-fec
service-policy type qos input QOS_CLASSIFICATION
no shutdown
interface Ethernet1/7
priority-flow-control mode on
priority-flow-control watch-dog-interval on
mtu 9216
fec rs-fec
service-policy type qos input QOS_CLASSIFICATION
no shutdown
interface Ethernet1/8
priority-flow-control mode on
priority-flow-control watch-dog-interval on
mtu 9216
fec rs-fec
service-policy type qos input QOS_CLASSIFICATION
no shutdown
interface Ethernet1/9
priority-flow-control mode on
priority-flow-control watch-dog-interval on
mtu 9216
fec rs-fec
service-policy type qos input QOS_CLASSIFICATION
no shutdown
interface Ethernet1/10
priority-flow-control mode on
priority-flow-control watch-dog-interval on
mtu 9216
fec rs-fec
service-policy type qos input QOS_CLASSIFICATION
no shutdown
interface Ethernet1/11
switchport
priority-flow-control mode on
priority-flow-control watch-dog-interval on
mtu 9216
speed 100000
fec rs-fec
no negotiate auto
service-policy type qos input QOS_CLASSIFICATION
no shutdown
interface Ethernet1/12
switchport
priority-flow-control mode on
priority-flow-control watch-dog-interval on
mtu 9216
speed 100000
fec rs-fec
no negotiate auto
service-policy type qos input QOS_CLASSIFICATION
no shutdown
interface Ethernet1/13
switchport
priority-flow-control mode on
priority-flow-control watch-dog-interval on
mtu 9216
speed 100000
fec rs-fec
no negotiate auto
service-policy type qos input QOS_CLASSIFICATION
no shutdown
interface Ethernet1/14
priority-flow-control mode on
priority-flow-control watch-dog-interval on
mtu 9216
fec rs-fec
service-policy type qos input QOS_CLASSIFICATION
no shutdown
interface Ethernet1/15
priority-flow-control mode on
priority-flow-control watch-dog-interval on
mtu 9216
fec rs-fec
service-policy type qos input QOS_CLASSIFICATION
no shutdown
interface Ethernet1/16
priority-flow-control mode on
priority-flow-control watch-dog-interval on
mtu 9216
fec rs-fec
service-policy type qos input QOS_CLASSIFICATION
no shutdown
interface Ethernet1/17
priority-flow-control mode on
priority-flow-control watch-dog-interval on
mtu 9216
fec rs-fec
service-policy type qos input QOS_CLASSIFICATION
no shutdown
interface Ethernet1/18
priority-flow-control mode on
priority-flow-control watch-dog-interval on
mtu 9216
fec rs-fec
service-policy type qos input QOS_CLASSIFICATION
no shutdown
interface Ethernet1/19
priority-flow-control mode on
priority-flow-control watch-dog-interval on
mtu 9216
fec rs-fec
service-policy type qos input QOS_CLASSIFICATION
no shutdown
interface Ethernet1/20
priority-flow-control mode on
priority-flow-control watch-dog-interval on
mtu 9216
fec rs-fec
service-policy type qos input QOS_CLASSIFICATION
no shutdown
interface Ethernet1/21
priority-flow-control mode on
priority-flow-control watch-dog-interval on
mtu 9216
fec rs-fec
service-policy type qos input QOS_CLASSIFICATION
no shutdown
interface Ethernet1/22
priority-flow-control mode on
priority-flow-control watch-dog-interval on
mtu 9216
fec rs-fec
service-policy type qos input QOS_CLASSIFICATION
no shutdown
interface Ethernet1/23
priority-flow-control mode on
priority-flow-control watch-dog-interval on
mtu 9216
fec rs-fec
service-policy type qos input QOS_CLASSIFICATION
no shutdown
interface Ethernet1/24
priority-flow-control mode on
priority-flow-control watch-dog-interval on
mtu 9216
fec rs-fec
service-policy type qos input QOS_CLASSIFICATION
no shutdown
interface Ethernet1/25
priority-flow-control mode on
priority-flow-control watch-dog-interval on
mtu 9216
fec rs-fec
service-policy type qos input QOS_CLASSIFICATION
no shutdown
interface Ethernet1/26
priority-flow-control mode on
priority-flow-control watch-dog-interval on
mtu 9216
fec rs-fec
service-policy type qos input QOS_CLASSIFICATION
no shutdown
interface Ethernet1/27
priority-flow-control mode on
priority-flow-control watch-dog-interval on
mtu 9216
fec rs-fec
service-policy type qos input QOS_CLASSIFICATION
no shutdown
interface Ethernet1/28
priority-flow-control mode on
priority-flow-control watch-dog-interval on
mtu 9216
fec rs-fec
service-policy type qos input QOS_CLASSIFICATION
no shutdown
interface Ethernet1/29
priority-flow-control mode on
priority-flow-control watch-dog-interval on
mtu 9216
fec rs-fec
service-policy type qos input QOS_CLASSIFICATION
no shutdown
interface Ethernet1/30
priority-flow-control mode on
priority-flow-control watch-dog-interval on
mtu 9216
fec rs-fec
service-policy type qos input QOS_CLASSIFICATION
no shutdown
interface Ethernet1/31
priority-flow-control mode on
priority-flow-control watch-dog-interval on
mtu 9216
fec rs-fec
service-policy type qos input QOS_CLASSIFICATION
no shutdown
interface Ethernet1/32
priority-flow-control mode on
priority-flow-control watch-dog-interval on
mtu 9216
fec rs-fec
service-policy type qos input QOS_CLASSIFICATION
no shutdown
interface Ethernet1/33
priority-flow-control mode on
priority-flow-control watch-dog-interval on
mtu 9216
fec rs-fec
service-policy type qos input QOS_CLASSIFICATION
no shutdown
interface Ethernet1/34
priority-flow-control mode on
priority-flow-control watch-dog-interval on
mtu 9216
fec rs-fec
service-policy type qos input QOS_CLASSIFICATION
no shutdown
interface Ethernet1/35
priority-flow-control mode on
priority-flow-control watch-dog-interval on
mtu 9216
fec rs-fec
service-policy type qos input QOS_CLASSIFICATION
interface Ethernet1/36
priority-flow-control mode on
priority-flow-control watch-dog-interval on
mtu 9216
fec rs-fec
service-policy type qos input QOS_CLASSIFICATION
interface mgmt0
vrf member management
ip address 192.168.1.203/24
icam monitor scale
line console
line vty
boot nxos bootflash:/nxos64-cs.10.5.4.M.bin

RDMA°¡ ¾ÊµÇ´Â °Ç°¡¿ä?
pNFS °¡ RDMA ·Î ÀÛµ¿ÇÏ·Á¸é. ÀÏ´Ü ±âº»ÀûÀ¸·Î MTU °ªÀÌ ÀÏÄ¡ÇØ¾ß ÇÕ´Ï´Ù.