Virtualized VCenter 5.1 and Paravirtual SCSI Driver problem

Hi there

We have been investigating a very strange problem, likely the strangest we have ever seen, and wanted to know if anyone else has seen this.

The Setup is as follows:

Software:

1. Two ESXI servers version 5.1 1021289 (HA Cluster no DRS and it is standard licensed)
2. Virtualized VCenter running Windows Server 2008 R2 SP1 and latest security updates, VCenter version 5.1.0 build 947673 running SSO,Inventory  and VCenter server.
3. VI Client build 5.1.0 build 942893
4. DB's are offloaded to a SQL 2008 R2 SP1 failover Cluster

Hardware:

1. Dell R620 Servers
2. Dell PowerVault MD 3620I ISCSI Storage

After the installation of VCenter we noticed a problem that only occurs after the VCenter VM is restarted. Once the Guest OS restarts, and VCenter services are started up, everything seems to be working well, except for when creating a new Virtual machine and editing the Virtual machine hardware we get an error thrown in VI Client stating "The server fault ‘InvalidArgument’ had no message" . The reconfiguration tasks still completes successfully. Existing Virtual machines are not affected by the problem, only newly created ones

Ok so what is causing the issue. We removed the new Virtual machines from the inventory , re added them, problem persists, we removed the hosts from vCenter, restarted them, and added them back to vCenter, problem persists, but only after vCenter itself is rebooted.

After about two or three days of investigating this we decided to reinstall the vCenter Virtual machine from scratch. So we reinstalled the OS, Reinstalled vCenter, created the cluster again, and tested. Everything seemed ok, the error no longer happened after vCenter reboots. Then I realized that we never installed VMware Tools  in the Guest OS of the vCenter VM. So we installed it, and restarted the Guest OS, and immediately found the problem was occurring again ? Vmware Tools causing this, surely not ? So we uninstalled VMWare Tools, and the problem disappears after a reboot.

After a lot of testing this, we have found that it only occurs if VMware tools installs the PVSCSI driver. Without the PVSCSI driver presence the issue does not occur. What is even stranger is that the VCenter VM is configured to use the LSI Logic SCSI adapter, not the VMware Paravirtual one. But as soon as a full install of VMware tools is done, and the PVSCSI driver is installed, the issue starts occurring.

Bizarre, very bizarre. As anyone else seen the PVSCSI driver do anything strage like this ?

We are about to log a call with VMWare support on this, but wanted to know if anyone else have seen this before we do.