【问题标题】：pktgen cannot find Mellanox portpktgen 找不到 Mellanox 端口
【发布时间】：2021-01-15 07:03:56
【问题描述】：

设置

Pktgen 版本 21.01.0
DPDK 版本 20.11
操作系统：ubuntu 18.04
网卡：Mellanox

driver: mlx5_core
version: 5.1-2.5.8
firmware-version: 16.28.2006 (MT_0000000012)
expansion-rom-version:
bus-info: 0000:03:00.1
supports-statistics: yes
supports-test: yes
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: yes

问题

当我在 dpdk 20.11 和 ./dpdk-devbind -s 中运行 dpdk-testpmd 时，它可以找到 mlx 端口

Network devices using kernel driver
===================================
0000:03:00.0 'MT27800 Family [ConnectX-5] 1017' if=ens1f0 drv=mlx5_core unused=vfio-pci
0000:03:00.1 'MT27800 Family [ConnectX-5] 1017' if=ens1f1 drv=mlx5_core unused=vfio-pci
0000:05:00.0 'I210 Gigabit Network Connection 1533' if=enp5s0 drv=igb unused=vfio-pci *Active*
0000:06:00.0 'I210 Gigabit Network Connection 1533' if=enp6s0 drv=igb unused=vfio-pci
0000:07:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' if=enp7s0f0 drv=ixgbe unused=vfio-pci
0000:07:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' if=enp7s0f1 drv=ixgbe unused=vfio-pci

$sudo build/app/dpdk-testpmd -c7 --vdev=net_pcap0,iface=eth0 --vdev=net_pcap1,iface=eth1 -- -i --nb-cores=2 --nb-ports=2 --total-num-mbufs=2048

EAL: Detected 24 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: Probe PCI driver: mlx5_pci (15b3:1017) device: 0000:03:00.0 (socket 0)
mlx5_pci: Size 0xFFFF is not power of 2, will be aligned to 0x10000.
EAL: Probe PCI driver: mlx5_pci (15b3:1017) device: 0000:03:00.1 (socket 0)
mlx5_pci: Size 0xFFFF is not power of 2, will be aligned to 0x10000.
open_iface_live(): Couldn't open eth0: eth0: No such device exists (SIOCGIFHWADDR: No such device)
open_single_iface(): Couldn't open interface eth0
vdev_probe(): failed to initialize net_pcap0 device
open_iface_live(): Couldn't open eth1: eth1: No such device exists (SIOCGIFHWADDR: No such device)
open_single_iface(): Couldn't open interface eth1
vdev_probe(): failed to initialize net_pcap1 device
EAL: Bus (vdev) probe failed.
EAL: No legacy callbacks, legacy socket not created
Interactive-mode selected
testpmd: create a new mbuf pool <mb_pool_0>: n=2048, size=2176, socket=0
testpmd: preferred mempool ops selected: ring_mp_mc
Configuring Port 0 (socket 0)
Port 0: 98:03:9B:06:AB:34
Configuring Port 1 (socket 0)
Port 1: 98:03:9B:06:AB:35
Checking link statuses...
Done
testpmd> quit

但是当我运行pktgen 时，它不起作用。

$sudo ./Builddir/app/pktgen -c 0xff -n 3 -a 0000:03:00.1 -- -p 0x1 -P -m "[1:2].0"

Copyright (c) <2010-2020>, Intel Corporation. All rights reserved. Powered by DPDK
EAL: Detected 24 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: No legacy callbacks, legacy socket not created

*** Copyright (c) <2010-2020>, Intel Corporation. All rights reserved.
*** Pktgen  created by: Keith Wiles -- >>> Powered by DPDK <<<

 Port: Name         IfIndex Alias        NUMA  PCI

!PANIC!: *** Did not find any ports to use ***
PANIC in pktgen_config_ports():
*** Did not find any ports to use ***
6: [./Builddir/app/pktgen(+0x977a) [0x556f614b477a]]
5: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7) [0x7f1721223bf7]]
4: [./Builddir/app/pktgen(+0x9319) [0x556f614b4319]]
3: [./Builddir/app/pktgen(+0x31fa7) [0x556f614dcfa7]]
2: [/usr/local/lib/x86_64-linux-gnu/librte_eal.so.21(__rte_panic+0xc5) [0x7f172221d285]]
1: [/usr/local/lib/x86_64-linux-gnu/librte_eal.so.21(rte_dump_stack+0x2e) [0x7f172223ef2e]]
Aborted

[Update-1]

我发现系统中存在librte_net_mlx5.so：

/usr/local/lib/x86_64-linux-gnu/librte_net_mlx5.so.21.0
/usr/local/lib/x86_64-linux-gnu/librte_net_mlx5.so.21
/usr/local/lib/x86_64-linux-gnu/librte_net_mlx5.so
/usr/local/lib/x86_64-linux-gnu/dpdk/pmds-21.0/librte_net_mlx5.so.21.0
/usr/local/lib/x86_64-linux-gnu/dpdk/pmds-21.0/librte_net_mlx5.so.21
/usr/local/lib/x86_64-linux-gnu/dpdk/pmds-21.0/librte_net_mlx5.so
/usr/local/lib/x86_64-linux-gnu/librte_net_mlx5.a
/home/dpdk-20.11/build/drivers/librte_net_mlx5.so.21.0
/home/dpdk-20.11/build/drivers/librte_net_mlx5.so.21
/home/dpdk-20.11/build/drivers/librte_net_mlx5.so
/home/dpdk-20.11/build/drivers/librte_net_mlx5.a.p
/home/dpdk-20.11/build/drivers/librte_net_mlx5.so.21.0.p
/home/dpdk-20.11/build/drivers/librte_net_mlx5.a

我尝试将-d librte_net_mlx5.so 添加为 @Vipin Varghese 的建议，并得到以下输出：

$sudo ./Builddir/app/pktgen -c 0xff -n 3 -a 0000:03:00.1 -d librte_net_mlx5.so -- -p 0x1 -P -m "[1:2].0"

EAL: Detected 24 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: Probe PCI driver: mlx5_pci (15b3:1017) device: 0000:03:00.1 (socket 0)
mlx5_pci: Size 0xFFFF is not power of 2, will be aligned to 0x10000.
EAL: No legacy callbacks, legacy socket not created

*** Copyright (c) <2010-2020>, Intel Corporation. All rights reserved.
*** Pktgen  created by: Keith Wiles -- >>> Powered by DPDK <<<

 Port: Name         IfIndex Alias        NUMA  PCI
    0: mlx5_pci       11                   0   15b3:1017/03:00.1

Initialize Port 0 -- TxQ 1, RxQ 1
MBUF: error setting mempool handler
!PANIC!: Cannot create mbuf pool (Default RX  0:0) port 0, queue 0, nb_mbufs 4096, socket_id 0: Invalid argument
PANIC in pktgen_mbuf_pool_create():
Cannot create mbuf pool (Default RX  0:0) port 0, queue 0, nb_mbufs 4096, socket_id 0: Invalid argument
6: [./Builddir/app/pktgen(+0x977a) [0x55fd3145f77a]]
5: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7) [0x7f5479069bf7]]
4: [./Builddir/app/pktgen(+0x9319) [0x55fd3145f319]]
3: [./Builddir/app/pktgen(+0x3198b) [0x55fd3148798b]]
2: [/usr/local/lib/x86_64-linux-gnu/librte_eal.so.21(__rte_panic+0xc5) [0x7f547a063285]]
1: [/usr/local/lib/x86_64-linux-gnu/librte_eal.so.21(rte_dump_stack+0x2e) [0x7f547a084f2e]]
Aborted

【问题讨论】：

由于您使用的是 DPDK 20.11，它使用了 ninja-meson，因此应确保您已构建了 mxl5 驱动程序。默认情况下它不会被构建，这会导致设备 PMD 探测失败。检查librte_net_mx* 的安装文件夹。请确认您是否构建了驱动程序？
你们那边有更新吗？
如果答案解决了您的问题或帮助您理解了问题，请采纳并点赞
我已经尝试了您的建议并更新了内容。 pktgen 找不到 librte_net_mlx5.so 而 .so 文件存在。
感谢您的回复，pktgen 在存在时找不到librte_net_mxl5.so，仅表示pktgen is not built with right DPDK。随意连接Skype进行调试。

标签： dpdk

【解决方案1】：

很明显pktgen 实用程序是

不是基于日志使用 Mellanox PMD mlx5 构建的
或 pktgen 未传递用于初始化 MLX5 PMD 的共享库

由于用于构建的 DPDK 是 DPDK 版本 20.11。使用共享库构建 pktgen 的概率很高。将 eal 参数作为 -d librte_net_mlx5.so 传递应该可以解决共享库问题。

不建议静态库路径的原因是因为 testpmd 的日志显示 MLX5 被识别，而 eth0 和 eth1 是不存在的接口并跳过 PCAP PMD

EAL: Probe PCI driver: mlx5_pci (15b3:1017) device: 0000:03:00.0 (socket 0)
mlx5_pci: Size 0xFFFF is not power of 2, will be aligned to 0x10000.
EAL: Probe PCI driver: mlx5_pci (15b3:1017) device: 0000:03:00.1 (socket 0)
mlx5_pci: Size 0xFFFF is not power of 2, will be aligned to 0x10000.

[EDIT-1] 也要求提供 cmets 中的正确详细信息，没有可用的。

[EDIT-2] 在控制台检查中检查 PKTGEN 21.01 是否使用 MXL5 PMD 构建

pkg-config --cflags --libs --static libdpdk | grep -i mlx5
nm [pktgen-application] | grep -i mlx5

。根据更新的日志，pktgen 不是用 DPDK 构建的，它位于/usr/local/lib/x86_64-linux-gnu

【讨论】：

根据最新更新，接口使用选项-d标识，请接受并投票。