I'm running ESXi 4.1 Update 3 on a lab machine.
I recently purchased some HP NC380T Dual Gigabit network cards and installed them.
I have no issues seeing the cards in BIOS, and just about everywhere I look in the logs the system is aware the card is there except that they are not assigned a vmnic number. I expect these ports to be assigned vmnic2 and vmnic3 since the motherboard has two gigabit ethernet ports on it (and they are working fine)
So I'm really at a loss as to why these ports are not being recognized so I can use them in ESXi. Any insight is appreciated!!!
Please find below additional information which should be helpful in understanding the problem.
Thank you,
Kevin
Only the motherboard ports are shown as NICs.
~ # esxcfg-nics -l
Name PCI Driver Link Speed Duplex MAC Address MTU Description
vmnic0 0000:02:00.00 tg3 Up 1000Mbps Full 00:1e:8c:51:62:26 1500 Broadcom Corporation NetXtreme BCM5721 Gigabit Ethernet
vmnic1 0000:03:00.00 tg3 Up 1000Mbps Full 00:1e:8c:51:63:10 1500 Broadcom Corporation NetXtreme BCM5721 Gigabit Ethernet
But lspci is showing that the NIC itself is seen at the PCI level:
~ # lspci | grep vmnic
000:002:00.0 Network controller: Broadcom Corporation NetXtreme BCM5721 Gigabit Ethernet [vmnic0]
000:003:00.0 Network controller: Broadcom Corporation NetXtreme BCM5721 Gigabit Ethernet [vmnic1]
000:006:04.0 Network controller: Broadcom Corporation NC380T PCI Express Dual Port Multifunction Gigabit Server Adapt [vmnic2]
000:007:05.0 Network controller: Broadcom Corporation NC380T PCI Express Dual Port Multifunction Gigabit Server Adapt [vmnic3]
And ethtool is showing the correct PCI addresses and drivers being assigned:
~ # ethtool -i vmnic0
driver: tg3
version: 3.110h.v41.1
firmware-version: 5721-v3.55a
bus-info: 0000:02:00.0
~ # ethtool -i vmnic1
driver: tg3
version: 3.110h.v41.1
firmware-version: 5721-v3.55a
bus-info: 0000:03:00.0
~ # ethtool -i vmnic2
driver: bnx2
version: 2.0.22f.v41.2
firmware-version: bc 1.9.6
bus-info: 0000:06:04.0
~ # ethtool -i vmnic3
driver: bnx2
version: 2.0.22f.v41.2
firmware-version: bc 1.9.6
bus-info: 0000:07:05.0
The following is from /var/log/messages. I see the error message, but I'm not sure exactly what it means and google didn't help. What is really odd is that the bnx2-cnic driver appears to be picking these ports up, but I do not see the ports listed in the storage adapters either.
Jun 12 14:56:47 vmkernel: 0:00:00:16.962 cpu7:4618)PCI: Announcing 0000:06:04.0
Jun 12 14:56:47 vmkernel: <6>Broadcom NetXtreme II Gigabit Ethernet Driver bnx2 v2.0.22f.v41.2 (Feb 11, 2011)
Jun 12 14:56:47 vmkernel: 0:00:00:16.962 cpu7:4618)socket buffer cache creation succeeded for vmnic%d
Jun 12 14:56:47 vmkernel: 0:00:00:16.962 cpu7:4618)VMK_PCI: 746: device 000:006:04.0 capType 1 capIndex 72
Jun 12 14:56:47 vmkernel: 0:00:00:16.962 cpu7:4618)VMK_PCI: 746: device 000:006:04.0 capType 7 capIndex 64
Jun 12 14:56:47 vmkernel: 0:00:00:16.962 cpu7:4618)VMK_PCI: 746: device 000:006:04.0 capType 5 capIndex 88
Jun 12 14:56:47 vmkernel: 0:00:00:16.963 cpu7:4618)<6>vmnic2: Broadcom NetXtreme II BCM5706 1000Base-T (A2) PCI-X 64-bit 100MHz found at mem d0
Jun 12 14:56:47 vmkernel: 0:00:00:16.963 cpu7:4618)PCI: driver bnx2 claimed device 0000:06:04.0
Jun 12 14:56:47 vmkernel: 0:00:00:16.963 cpu7:4618)PCI: Registering network device 0000:06:04.0
Jun 12 14:56:47 vmkernel: 0:00:00:16.963 cpu7:4618)VMK_PCI: 646: Device 000:006:04.0 name: vmnic2
Jun 12 14:56:47 vmkernel: 0:00:00:16.963 cpu7:4618)Uplink: 11078: Device vmnic2 not yet opened
Jun 12 14:56:47 vmkernel: 0:00:00:16.963 cpu7:4618)LinPCI: LinuxPCI_DeviceClaimed: Device 6:20 claimed.
Jun 12 14:56:47 vmkernel:
Jun 12 14:56:47 vmkernel: 0:00:00:16.963 cpu7:4618)PCI: Trying 0000:07:05.0
Jun 12 14:56:47 vmkernel: 0:00:00:16.963 cpu7:4618)PCI: Announcing 0000:07:05.0
Jun 12 14:56:47 vmkernel: 0:00:00:16.963 cpu7:4618)VMK_PCI: 746: device 000:007:05.0 capType 1 capIndex 72
Jun 12 14:56:47 vmkernel: 0:00:00:16.963 cpu7:4618)VMK_PCI: 746: device 000:007:05.0 capType 7 capIndex 64
Jun 12 14:56:47 vmkernel: 0:00:00:16.963 cpu7:4618)VMK_PCI: 746: device 000:007:05.0 capType 5 capIndex 88
Jun 12 14:56:47 vmkernel: 0:00:00:16.963 cpu7:4618)<6>vmnic3: Broadcom NetXtreme II BCM5706 1000Base-T (A2) PCI-X 64-bit 100MHz found at mem d4
Jun 12 14:56:47 vmkernel: 0:00:00:16.963 cpu7:4618)PCI: driver bnx2 claimed device 0000:07:05.0
Jun 12 14:56:47 vmkernel: 0:00:00:16.963 cpu7:4618)PCI: Registering network device 0000:07:05.0
Jun 12 14:56:47 vmkernel: 0:00:00:16.963 cpu7:4618)VMK_PCI: 646: Device 000:007:05.0 name: vmnic3
Jun 12 14:56:47 vmkernel: 0:00:00:16.963 cpu7:4618)Uplink: 11078: Device vmnic3 not yet opened
Jun 12 14:56:47 vmkernel: 0:00:00:16.963 cpu7:4618)LinPCI: LinuxPCI_DeviceClaimed: Device 7:28 claimed.
Jun 12 14:56:47 vmkernel:
Jun 12 14:56:47 vmkernel: 0:00:00:16.963 cpu7:4618)PCI: driver bnx2 claimed 2 devices
Jun 12 14:56:47 vmkernel: 0:00:00:16.963 cpu7:4618)Uplink: 13003: Opening device vmnic2
Jun 12 14:56:47 vmkernel: 0:00:00:16.963 cpu0:4129)WARNING: VMK_VECTOR: 104: Bad vector 0 specified
Jun 12 14:56:47 vmkernel: 0:00:00:16.963 cpu0:4129)WARNING: vmklinux26: vmklnx_request_irq: Couldn't register vector 0x0
Jun 12 14:56:47 vmkernel: 0:00:00:16.963 cpu0:4129)Uplink: 12466: Device open handler failed for : vmnic2
Jun 12 14:56:47 vmkernel:
Jun 12 14:56:47 vmkernel: 0:00:00:16.963 cpu5:4618)Uplink: 13026: Device vmnic2 failed to open
Jun 12 14:56:47 vmkernel: 0:00:00:16.963 cpu5:4618)Uplink: 12209: Failed to close vmnic2: not found
Jun 12 14:56:47 vmkernel: 0:00:00:16.963 cpu5:4618)Uplink: 13003: Opening device vmnic3
Jun 12 14:56:47 vmkernel: 0:00:00:16.963 cpu0:4129)WARNING: VMK_VECTOR: 104: Bad vector 0 specified
Jun 12 14:56:47 vmkernel: 0:00:00:16.963 cpu0:4129)WARNING: vmklinux26: vmklnx_request_irq: Couldn't register vector 0x0
Jun 12 14:56:47 vmkernel: 0:00:00:16.963 cpu0:4129)Uplink: 12466: Device open handler failed for : vmnic3
Jun 12 14:56:47 vmkernel:
Jun 12 14:56:47 vmkernel: 0:00:00:16.963 cpu5:4618)Uplink: 13026: Device vmnic3 failed to open
Jun 12 14:56:47 vmkernel: 0:00:00:16.963 cpu5:4618)Uplink: 12209: Failed to close vmnic3: not found
Jun 12 14:56:47 vmkernel: 0:00:00:16.963 cpu5:4618)Mod: 4163: Initialization of bnx2 succeeded with module ID 32.
Jun 12 14:56:47 vmkernel: 0:00:00:16.963 cpu5:4618)bnx2 loaded successfully.
Jun 12 14:56:47 vmkernel:
Jun 12 14:56:48 vmkernel: sysboot: Loading VMkernel Module 'cnic'
Jun 12 14:56:48 vmkernel: 0:00:00:17.406 cpu6:4626)Loading module cnic ...
Jun 12 14:56:48 vmkernel: 0:00:00:17.406 cpu6:4626)Elf: 2187: <cnic> provides name-space <esx:nover>
Jun 12 14:56:48 vmkernel: 0:00:00:17.407 cpu6:4626)module heap : Initial heap size : 1048576, max heap size: 68665344
Jun 12 14:56:48 vmkernel: 0:00:00:17.407 cpu6:4626)module heap vmklnx_cnic: creation succeeded. id = 0x4100bc000000
Jun 12 14:56:48 vmkernel: 0:00:00:17.407 cpu6:4626)module skb heap : Initial heap size : 524288, max heap size: 23068672
Jun 12 14:56:48 vmkernel: 0:00:00:17.408 cpu6:4626)module skb heap : creation succeeded
Jun 12 14:56:48 vmkernel: <6>Broadcom NetXtreme II CNIC Driver cnic v1.10.2q.v41.9 (Feb. 11, 2011)
Jun 12 14:56:48 vmkernel: 0:00:00:17.408 cpu6:4626)<6>cnic: Added CNIC device: vmnic3
Jun 12 14:56:48 vmkernel: 0:00:00:17.408 cpu6:4626)<6>cnic: Added CNIC device: vmnic2
Jun 12 14:56:48 vmkernel: 0:00:00:17.408 cpu6:4626)Mod: 4163: Initialization of cnic succeeded with module ID 33.
Jun 12 14:56:48 vmkernel: 0:00:00:17.408 cpu6:4626)cnic loaded successfully.
I've also installed the recommended vmware drivers
~ # esxupdate query --vib-view | grep bnx2
deb_vmware-esx-drivers-net-bnx2_400.2.0.7d-6vmw.3.26.800380 installed 2012-08-02T20:55:21+00:00
deb_vmware-esx-drivers-scsi-bnx2i_400.1.8.11t5.rc2.8.1-4vmw.3.26.800380 installed 2012-08-02T20:55:21+00:00
deb_vmware-esx-drivers-net-bnx2x_400.1.54.1.v41.1-3vmw.3.26.800380 installed 2012-08-02T20:55:21+00:00
cross_vmware-esx-drivers-net-bnx2_400.2.2.3m.v41.2-1vmw.0.0.00000 uninstalled 2013-06-11T21:29:41.637931+00:00
cross_vmware-esx-drivers-scsi-bnx2i_400.2.76.56.v41.2-1vmw.0.0.00000 uninstalled 2013-06-11T21:30:39.798582+00:00
cross_vmware-esx-drivers-net-bnx2x_400.1.76.54.v41.1-1vmw.0.0.00000 uninstalled 2013-06-11T21:31:37.731451+00:00
cross_vmware-esx-drivers-net-bnx2_400.2.0.22f.v41.2-1vmw.0.0.00000 installed 2013-06-11T21:46:02.744078+00:00
cross_vmware-esx-drivers-net-bnx2x_400.1.62.15.v41.2-1vmw.0.0.00000 installed 2013-06-11T21:46:41.777374+00:00
cross_vmware-esx-drivers-scsi-bnx2i_400.1.9.1t.v41.2-1vmw.0.0.00000 installed 2013-06-11T21:47:14.932194+00:00
~ #
Finally, I haven't done any specific configuration with the bnx2 module
~ # esxcfg-module -i bnx2
esxcfg-module module information
input file: /usr/lib/vmware/vmkmod/bnx2.o
License: GPL
Version:
Name-space:
Required name-spaces:
com.vmware.vmkapi@v1_1_0_0
Parameters:
disable_msi: int
Disable Message Signaled Interrupt (MSI)
force_netq: array of int
Option used for 5709/5716 only: Enforce the number of NetQueues per port (allowed values: -1 to 7 queues: 1-7 will force the number of NetQueues for the given device, 0 to disable NetQueue, -1 to use the default driver NetQueues value) [Maximum supported NIC's = 32] [example usage: force_net=-1,0,1,2: This corresponds to the first 5709/5716 to use the default number of NetQueues, disable NetQueue on the second 5709/5716, use 1 NetQueue on the third 5709/5716use 2 NetQueues on the fourth 5709/5716]
heap_initial: int
Initial heap size allocated for the driver.
heap_max: int
Maximum attainable heap size for the driver.
skb_heap_initial: int
Initial private socket buffer heap size allocated for the driver.
skb_heap_max: int
Maximum attainable private socket buffer heap size for the driver.
stop_on_tx_timeout: int
For debugging purposes, prevent a chip reset when a tx timeout occurs
Again, any insight or suggestions are welcome!
Kevin