Hello there dear community,

I see an odd behaviour with the card in the title and I would like to ask if anyone had the same experience or an idea?

Description of the setup:

The card is a Mellanox ConnectX-3 Pro 40G card, jumbo frames enabled. The server is an Intel R2208WFTZS.

The card's firmware is on the latest stable version from mellanox.

The vsan's vmk interface is connected to one of the ports of the 40G card, jumbo frames enabled.

I'm using a non customised ESXI 6.7 image with the nmlx4_en native driver, all the latest patches installed.

Problem:

The setup works well, however occasionally totally randomly the vmk interface can't reach the other vsan nodes in the same cluster, can't ping anyone.

When I was doing packet capture on the vmk interface, I could see the arp requests arriving from the other nodes to the vmk interface, the interface responds to them, however the responses don't make it out of the server. Seemed similar to a unidirectional link.

If I reload the vmnic interface where the vsan traffic meant to go through, it all returns to normal, the traffic from vsan vmk starts to flow, arp responses to through.

Obviously the logs don't say much other than this interesting part:

2018-08-13T20:26:23.187Z cpu25:2097436)<NMLX_INF> nmlx4_en: vmnic2: nmlx4_en_QueueRemoveFilter - (vmkdrivers/native/BSD/Network/mlnx/nmlx4/nmlx4_en/nmlx4_en_multiq.c:2294) MAC RX filter (class 1) at index 0 is removed from
2018-08-13T20:26:23.187Z cpu25:2097436)<NMLX_INF> nmlx4_en: vmnic2: nmlx4_en_QueueRemoveFilter - (vmkdrivers/native/BSD/Network/mlnx/nmlx4/nmlx4_en/nmlx4_en_multiq.c:2301) RX ring 1, QP[0x49], Mac address 00:50:56:6c:23:xx
2018-08-13T20:26:23.187Z cpu25:2097436)<NMLX_INF> nmlx4_en: vmnic2: nmlx4_en_RxQFree - (vmkdrivers/native/BSD/Network/mlnx/nmlx4/nmlx4_en/nmlx4_en_multiq.c:789) RX queue 1 is freed
2018-08-13T20:26:39.594Z cpu43:2098358)VLANMTUCheck: NMVCCheckUplinkResultDetail:880: Vlanmtuchk EVENT: DV port 408 uplink vmnic2MTU check RESULT_CHANGED
2018-08-13T20:26:39.594Z cpu43:2098358)VLANMTUCheck: NMVCCheckUplinkResultDetail:898: Vlanmtuchk EVENT: DV port 408 uplink vmnic2VLAN check RESULT_CHANGED

Seems like the vsan queue is being destroyed? Not sure why is that happening. And you see after that the dvs alarms come on.

Other than this line, no signs of driver and firmware related logs or crashes.

It happened on 2 servers already, identical ones so I'm afraid it's not just specific to one server. Don't really know where to start or how to dig deeper.

Any idea or suggestion?

Thank you very much!

Zoltan

Hello All,

After install ESXi 6.0 via "VMware-VMvisor-Installer-6.0.0.update02-4192238.x86_64-Dell_Customized-A04" on Dell PowerEdge R730, everything is okay !!!

But, if I try to install ESXi 6.5 via "VMware-VMvisor-Installer-6.5.0-4564106.x86_64-Dell_Customized-A00" on Dell PowerEdge R730, my vmnic0 and vmnic1 (Intel X710 DP 10Gb DA/SFP+) are not working !!!

Does anybody know the solution ?

Thanks for your reply.

[root@R730:~] esxcli network nic list

Name    PCI Device    Driver  Admin Status  Link Status  Speed  Duplex  MAC Address        MTU  Description                                          

------  ------------  ------  ------------  -----------  -----  ------  -----------------  ----  ---------------------------------------------------------

vmnic0  0000:01:00.0  i40en  Up            Up          10000  Full    f8:bc:12:05:85:d0  1500  Intel Corporation Ethernet Controller X710 for 10GbE SFP+

vmnic1  0000:01:00.1  i40en  Up            Up          10000  Full    f8:bc:12:05:85:d2  1500  Intel Corporation Ethernet Controller X710 for 10GbE SFP+

vmnic2  0000:0c:00.0  igbn    Up            Down            0  Half    f8:bc:12:05:85:f0  1500  Intel Corporation Gigabit 4P X710/I350 rNDC          

vmnic3  0000:0c:00.1  igbn    Up            Up            1000  Full    f8:bc:12:05:85:f1  1500  Intel Corporation Gigabit 4P X710/I350 rNDC          

vusb0  Pseudo        cdce    Up            Up            100  Full    18:fb:7b:5d:d5:ee  1500  DellTM iDRAC Virtual NIC USB Device              

[root@R730:~] esxcli network nic get -n vmnic0

  Advertised Auto Negotiation: false

  Advertised Link Modes: 1000BaseT/Full, 10000BaseT/Full, 10000BaseT/Full, 40000BaseCR4/Full, 40000BaseSR4/Full

  Auto Negotiation: false

  Cable Type:

  Current Message Level: -1

  Driver Info:

        Bus Info: 0000:01:00:0

        Driver: i40en

        Firmware Version: 5.04 0x800024bc 17.5.11

        Version: 1.1.0

  Link Detected: true

  Link Status: Up

  Name: vmnic0

  PHYAddress: 0

  Pause Autonegotiate: false

  Pause RX: false

  Pause TX: false

  Supported Ports:

  Supports Auto Negotiation: false

  Supports Pause: false

  Supports Wakeon: true

  Transceiver:

  Virtual Address: 00:50:56:52:57:61

  Wakeon: MagicPacket(tm)

I figured out recently that memory and CPU can be hot added to VM if OS is supported and VMWare tools are installed.

By default this is disabled and would like to know the pro and cons of hot add memory and CPU.

Hi,

the CPUs of the physical servers do not support hyper threading. ESX Hyperthreading is inactive .for example there are 4 vCPUs in the virtual machine but on the windows task manager also sees  1 vCPU. is this server running with 4 vcpu or 1 vcpu. what is the value assigned to the virtual server. how many virtual servers are working with vCpu.

Hi,

how is it solved to attach a rotating sata rdx tape to a virtual machine in esxi 6.5?

My known solution is to add the sata RDX tape as a virtual disk into the machine.

But if we swap out the tape with another one the virtual machine crashs.

We dont have the problem if we are using a USB 3.0 RDX controller. The problem only occurs within the sata solution.

Is it possible to configure the machine with a rotating tape as a virtual disk?

Hi everyone.

After upgrading from 6.5 to 6.7 our host displays the above error in vCenter. Everything seems to work fine, but can't make to error disappear.

I can't seem to find the service under "services", so I can enable/configure it.

Any help is much appreciated.

Kindly

Kasper

i have error when i upgrade esxi 5.5 u2 to 6.5 u2 . what is problem ?

login as: root

Using keyboard-interactive authentication.

Password:

The time and date of this login have been sent to the system logs.

VMware offers supported, powerful system administration tools.  Please

see www.vmware.com/go/sysadmintools for details.

The ESXi Shell can be disabled by an administrative user. See the

vSphere Security documentation for more information.

~ # esxcli software vib install -d /vmfs/volumes/LDS152/VMware-ESXi-6.5.0-829425

3-Custom-Cisco-6.5.2.1.zip

[MetadataDownloadError]

Could not download from depot at zip:/vmfs/volumes/LDS152/VMware-ESXi-6.5.0-8294253-Custom-Cisco-6.5.2.1.zip?index.xml, skipping (('zip:/vmfs/volumes/LDS152/VMware-ESXi-6.5.0-8294253-Custom-Cisco-6.5.2.1.zip?index.xml', '', "Error extracting index.xml from /vmfs/volumes/LDS152/VMware-ESXi-6.5.0-8294253-Custom-Cisco-6.5.2.1.zip: [Errno 2] No such file or directory: '/vmfs/volumes/LDS152/VMware-ESXi-6.5.0-8294253-Custom-Cisco-6.5.2.1.zip'"))

        url = zip:/vmfs/volumes/LDS152/VMware-ESXi-6.5.0-8294253-Custom-Cisco-6.5.2.1.zip?index.xml

Please refer to the log file for more details.

~ #

Hi

I cannot get ESXi 6.7 to install Mellanox ConnectX-4 in ESXi 6.7

I the UI on the ESXi I can see the Mellanox card under manage->hardware.

But it will not show up under NIC or under HBA, not sure where to find it.

If I from the CLI run the command

esxcli hardware pci list

I can also see the ESXi have found the controller correctly.

But as you can see Module ID and Module Name has not been set.

   Module ID: -1

   Module Name: None

I use the build in driver in ESXi 6.7 as it should support this controller and I have upgraded it to newest firmware so it fits with HCL

Any way to fix this?

0000:87:00.0
   Address: 0000:87:00.0
   Segment: 0x0000
   Bus: 0x87
   Slot: 0x00
   Function: 0x0
   VMkernel Name: vmnic4
   Vendor Name: Mellanox Technologies
   Device Name: MT27700 Family [ConnectX-4]
   Configured Owner: VMkernel
   Current Owner: VMkernel
   Vendor ID: 0x15b3
   Device ID: 0x1013
   SubVendor ID: 0x15b3
   SubDevice ID: 0x0010
   Device Class: 0x0207
   Device Class Name: Infiniband controller
   Programming Interface: 0x00
   Revision ID: 0x00
   Interrupt Line: 0x0b
   IRQ: 255
   Interrupt Vector: 0x00
   PCI Pin: 0x00
   Spawned Bus: 0x00
   Flags: 0x3201
   Module ID: -1
   Module Name: None
   Chassis: 0
   Physical Slot: 2
   Slot Description: RSC-UN4-88 SLOT2 PCI-E X8
   Passthru Capable: true
   Parent Device: PCI 0:128:3:0
   Dependent Device: PCI 0:135:0:0
   Reset Method: Function reset
   FPT Sharable: true

0000:87:00.1
   Address: 0000:87:00.1
   Segment: 0x0000
   Bus: 0x87
   Slot: 0x00
   Function: 0x1
   VMkernel Name: vmnic5
   Vendor Name: Mellanox Technologies
   Device Name: MT27700 Family [ConnectX-4]
   Configured Owner: VMkernel
   Current Owner: VMkernel
   Vendor ID: 0x15b3
   Device ID: 0x1013
   SubVendor ID: 0x15b3
   SubDevice ID: 0x0010
   Device Class: 0x0207
   Device Class Name: Infiniband controller
   Programming Interface: 0x00
   Revision ID: 0x00
   Interrupt Line: 0x0a
   IRQ: 255
   Interrupt Vector: 0x00
   PCI Pin: 0x01
   Spawned Bus: 0x00
   Flags: 0x3201
   Module ID: -1
   Module Name: None
   Chassis: 0
   Physical Slot: 2
   Slot Description: Chassis slot 2; function 1
   Passthru Capable: true
   Parent Device: PCI 0:128:3:0
   Dependent Device: PCI 0:135:0:1
   Reset Method: Function reset
   FPT Sharable: true

Hello,

I have a new build I just put together and am running ESXi 6.7 + vSphere 6.7 - just updated ESXi with latest updates from https://esxi-patches.v-front.de/ESXi-6.7.0.html

The system is performing extremely slow.  I am making my first VM, Windows 10 1803, and the speed reported in vCenter is 600 MHz.  And it really feels like it, too.

My system:

Supermicro X10SRL-F motherboard latest bios (3.1)

E5-2650 v4

2 x 8GB DDR4-2666MHz RAM 1rx8

Samsung SM953 NVMe datastore

running off USB to boot

I'm noticing as I write this that I am only using 2 DIMMs in a system with 4-channel memory.  I have more memory coming soon, so hopefully that will improve things.

But it's hard for me to believe this is all because of using two DIMMs instead of four.  Is there some sort of setting maybe in the VM or in vSphere that can be tweaked when the processor is clocking so slow?

Thank you

Avery

Hello,

i have an iSCSI LUN attached with 5TB. Now I increased this LUN to 10TB.

At the ESXi host I refreshed the storage devices - the new capacity is shown there correctly. But when I open the storage tab at vCenter there are still the 5TB shown.

How can I expand the LUN? "Increase Datastore Capacity" is empty, so I cannot select anything.

Thanks a lot

Tobias

ESXI 6.5 Free License ?

I have come across an interesting issue with a new HPE platform. The system is running within a C7000 BladeSystem, with BL460c Gen9 blades.

We have noticed some degradation in performance on iSCSI connection (using the Software iSCSI initiator), this traffic runs over vmnic1 and vmnic2 details from the NIC list are below.

Each NIC is reporting at 10000 Mb full, however I am not able to set the speed on the ESXi server. vmnic1 reports the following for advertised link modes;

[root@ESX:~] esxcli network nic get -n vmnic1

   Advertised Auto Negotiation: true

   Advertised Link Modes: 1000BaseKR2/Full, 10000BaseKR2/Full, 20000BaseKR2/Full, Auto

   Auto Negotiation: true

Where as vmnic2 reports the following modes

[root@ESXi2b-14:~] esxcli network nic get -n vmnic2

   Advertised Auto Negotiation: false

   Advertised Link Modes: 20000None/Full

   Auto Negotiation: false

Confused, the settings are identical for these within OneView. Both NIC's are using firmware - 12.0.1110.11 from SPP 2018.06.0. The HPE ESXi image has been used including driver version 12.0.1115.0 which shows as being compatible on the comparability guide VMware Compatibility Guide - I/O Device Search.

Has anyone else seen this issue? If I try and manually set the speed/duplex settings via esxcli it fails with the following error in the vmkernel.log

2018-08-14T23:49:41.361Z cpu20:65677)WARNING: elxnet: elxnet_linkStatusSet:7471: [vmnic2] Device is not privileged to do speed changes

As a result of this when using HCIBench to test the storage throughput the 95%tile_LAT value is reading excessively when traversing vmnic2 - 95%tile_LAT = 3111.7403 ms

Any thoughts??

Hello, i want to ask

recently i have problem like this :

i have storage with 6 volumes let says A,B,C,D,E,F

volumes A,B,C connected to host ESXI-1 and i have formated for datastore with vmfs format

volumes D,E,F connected to physical Server let says LINUX-1 and LINUX-2

i want to convert LINUX-1 to Virtual machine

i have connected A,B,C,D,E,F FibreChanel to ESXI-1 and ESXI-1 detected A,B,C,D,E,F Volumes

but when i want to attach D,E,F for datastore i have notice like this

There is one layout configuration available All available partition will be used

so i  assume if i click next it will formated D,E,F volumes, but LINUX-2 uses that volumes too

so how can i attach volumes D,E,F for datastore without format existing data? anyone can help me?

sorry for my bad english

Hi All

I installed custom certs in our vSphere 6.5 environment. There are 2 sites in enhanced link mode vCenter and PSC in each both with a custom cert from our local CA of SHA1 and connected SRM.

I need to upgrade these certs to SHA256. I cant find much info on the internet about it, does anyone have any experience with this and steps taken?

Thank you so much for any info

Hello

In my vmware environment I used BIOS for my virtual machine. I inject vmware drivers with his query

SELECT Model FROM Win32_ComputerSystem WHERE Model = "VMware Virtual Platform"

It always works fine and installs the drivers. Recently I upgrade my Vsphere environment to version 6.7 and when I create a new VM UEFI Bios is set by default. I thought that's fine. However when I ran the same task sequence no VMware drivers will be injected. I ran the command "wmic csproduct get name" in cmd and the result was "VMware7,1". So obviously with UEFI enabled the model name is different. I thought that's no problem and copied my inject driver task and change it to

SELECT Model FROM Win32_ComputerSystem WHERE Model = "VMware7,1"

However MDT still won't inject the drivers.

Any idea how I can get it working?

Thanks

Edy

We recently purchased 8 new HPE DL380 Gen10 servers. They came with two 10Gbe ports embedded on the motherboard and two 10Gbe ports on a PCI card. All of these use the Intel X710 controller. Before I purchased these servers I had no idea how many problems these particular nics have given people. Here is a sampling of issues:

Install ESXi 6.5 on R730 with Intel X710

https://lonesysadmin.net/2018/02/28/intel-x710-nics-are-crap/

http://www.v-strange.de/index.php/15-vmware/239-vsphere-6-5-with-intel-x710-network-adapter

VMware Knowledge Base

I have two DAC cables hooked from this server to my Juniper switch. In the ESXi console I select both of these nics for my management network, give it an IP address, mask, gateway, and dns server. I reboot the host and during the boot process pings to the management interface start working. Once the host is up and running for about 30 seconds, the pings stop. If I remove one of the nics from the management network, pings start again. In other words, when two nics are connected for the management network, it will not work. I have upgraded the driver and firmware on the HPE Ethernet 10Gb 2-port 562SFP+ Adapter (it uses the Intel X710 controller) thinking it might help. We are now sitting at driver version 1.5.8 and firmware 10.2.5. The server is running HPE customized ESXi 6.5 Update 1

Do you think the problem I am experiencing (two nics for the management network will not work) is caused by this controller?

Are there other things I can check that might be causing this behavior?

is there a way to determine what hosts logical CPUs are used for each running VM guest, or the NUMA node they are running on? Obviously you can check VM configuration, however, I wonder if there was a way to achieve this via the CLI?

I thought you might be able to extract this information through `esxtop`, but it doesn't appear to give me what I'm after (unless I'm looking at this the wrong way)

Hello Everyone,

I got here

1x vSphere 6.5 Server controlling 3x ESXi 6.5 Hosts in a HA Cluster.

My Issue is:  I updated all 3 Host and now i can´t licence the last of them because the vSphere argues with me that "DRS" is used on that Host (Evaluation Licence) and my Licence is without DRS. BUT: DRS is is disabled via vSphere and never was enabled before. What did I overlook at this Point?

Thanks in Advance.

This will be my server: A HP ProLiant DL380 Gen10 with two Intel Xeon-Silver 4114. (i.e., it has 2 physical CPUs with 10 cores each).

I want to install ESXi hypervisor 6.7. I will not be using vCenter server.

The server will be used to host about 40 Windows guest machines. I've got the Windows licenses covered.

So will I be needing a paid license for ESXi?

Hi,

I've been working with a new server, learning ESXi.  My first build was ESXi 6.5.

I was using iperf to test my network setup after following this guide: https://www.virtuallyghetto.com/2016/03/quick-tip-iperf-now-available-on-esxi.html

https://www.virtuallyghetto.com/2016/03/quick-tip-iperf-now-available-on-esxi.html

It was simple:

cp /usr/lib/vmware/vsan/bin/iperf /usr/lib/vmware/vsan/bin/iperf.copy

/usr/lib/vmware/vsan/bin/iperf.copy -s -B [IPERF-SERVER-IP]

Then I upgraded to 6.7.  I did not try iperf again using this method.  So, unfortunately, I don't know if the 'upgrade' preserved this function from 6.5.

I recently formatted and did a fresh install from ESXi 6.7 ISO.  Now, I want to iperf again... there are no traces of iperf on a fresh ESXi 6.7 install (that I can find).  But, I'm a CLI noob for all things Unix/Linux/FreeBSD/etc...

I did find a directory /usr/lib/vmware/vsan/perfsvc

But, I have no idea what it means or how it might be leveraged.

Appreciate any help here.

Br,

Mike