ESXi 6.0/6.5 and Intel X710 NIC Problem

If you are running ESXi 6.0/6.5 and using a Intel X710 network adapter, the network card port may stop forwarding packets, if you are using the native driver (i40en). I customer using both Dell, HPE and Lenovo servers where I have seee this problem. This is a huge problem.

The is due to a new function in the driver called “Malicious Driver Detection” or MDD for short.

I the vmkernet.log file you will see, this lines.

i40en: i40en_HandleMddEvent:6495: Malicious Driver Detection event 0x02 on TX queue 0 PF number 0x00 VF number 0x00
i40en: i40en_HandleMddEvent:6521: TX driver issue detected, PF reset issued

The problem is fixed in ESXi 6.7 that has a new driver version 1.7.1.

See this from Intel forum: https://communities.intel.com/community/tech/wired/blog/2018/05/23/malicious-driver-detection-mdd-event-resolved

The workaround is to uninstall og disable the native driver “i40en” and have a working version or the Linux style driver installed “i40e”.

I really hopes that Intel will fix this now, the article from the forum is dated May 23th of 2018.

Update (14th July 2018)

There is a known issue with the i40e driver versions up to 2.0.6, so you should use 2.0.7 or newer. See VMware KB: https://kb.vmware.com/s/article/2126909

Update (28th August 2018)

Intel/VMware has released a new driver version 1.7.5 that should fixes this issue. from the release notes:

Fix for MDD event and TX hang caused by TSO_MSS option smaller than 64 bytes

It can be found here:

ESXi 6.0: https://my.vmware.com/group/vmware/details?downloadGroup=DT-ESXI60-INTEL-I40EN-175&productId=491

ESXi 6.5: https://my.vmware.com/group/vmware/details?downloadGroup=DT-ESXI65-INTEL-I40EN-175&productId=614

I have not tested the driver at any customer.

If you are using LLDP, please see my Colleagues blog about Intel X710 and LLDP here: http://www.core-four.info/2018/06/lldp-not-available-on-intel-x710.html

Update (7th September 2018)

A new version has been released that fixes some other problems, also including a new MDD problem. the new version is 1.7.11, and can be downloaded for 6.0/6.5/6.7.