VM operation not allowed in current state

I have see this message some times, specially after som storage problems, and it can be a little difficult to find out how to fix this. Here is a guide to, how i usually fixes it.

The error can look like this:

And the Machine can not be connected on thru the network, or pinged.

The first task i do i look for if there is any files locked. This can be done thru SSH, you should check the vmx and/or -flat.vmdk files.

[root@esx01:~] vmkfstools -D /vmfs/volumes/Datastore/Server/Server.vmx
Lock [type 10c00001 offset 235544576 v 270, hb offset 3809280
gen 101, mode 1, owner 56f32abb-b98107ba-ba5c-ac1624ee27e8 mtime 5372
num 0 gblnum 0 gblgen 0 gblbrk 0]
Addr <4, 507, 12>, gen 13, links 1, type reg, flags 0, uid 0, gid 0, mode 100755
len 3728, nb 1 tbz 0, cow 0, newSinceEpoch 1, zla 2, bs 8192

The information that we grab from this is the "mode 1" telling os that the file i locked, and "ac1624ee27e8" this is the MAC address of the a physical NIC i the host, that owns the lock.

To find the host i use RVTools to look for the MAC address in the vNIC tab, or us PowerCLI.

PowerCLI C:\> $vmhost = Get-VMHost | Get-VMHostNetworkAdapter | Where-Object {$_.Mac -eq "ac:16:24:ee:27:e8"}
PowerCLI C:\> $vmhost.vmhost
:
Name                 ConnectionState PowerState NumCpu CpuUsageMhz CpuTotalMhz   MemoryUsageGB   MemoryTotalGB Version
----                 --------------- ---------- ------ ----------- -----------   -------------   ------------- -------
esx1.domain.local    Connected       PoweredOn      12        2965       36792          71,176          95,990   6.0.0
esx1.domain.local    Connected       PoweredOn      12        2965       36792          71,176          95,990   6.0.0

Now we know that it's the esx1.domain.local host, we have to investigate, using the we check if the VM is running as a process, that is not reflected in vCenter or the host using the vSphere Client, or Web Client.

I do that from the SSH on the host i found here:

[root@esx01:~] esxcli vm process list | grep Server -A 1 | grep -v "Display Name" | grep -v "Config File"
Server
   World ID: 112261247
—
The information we finds here is the World ID: 112261247

We now want to kill this process.

[root@esx01:~] esxcli vm process kill -t hard -w 112261247

Normally hard is good enough, ellers try "force" instead.