During some consultancy projects @ SMB customers where we needed to install and configure VMware VDR to do some backups, we had some issues with Windows Machines and VSS. The result of the these errors with VSS was that the VDR backup jobs all failed.
Here some possible solutions (gonna try to keep the list up to date.)
- VSS could not be started on a Windows 2003 server
We had to reregister all VSS DLL’s, restart the server and everything was working again. (check this)
VMware KB Articles about possible VDR / VSS issues.
Because VDR is taking snapshots of your virtual machines this can also be a interesing KB
Doing a P2V is very easy using the VMware Converter (Standard / Enterprise). But after the machine has been migrated the P2V is not yet finished, we need to do some extra stuff.
- Remove all hardware related software via add / remove programs (if possible) or disable the service.
- Old unused devices should be removed using device manager after setting a special SET parameter (set devmgr_show_nonpresent_devices=1). This can be done manually are using a script.
- Update virtual hardware settings (CPU / Memory)
- Enter the IP configuration on the new NIC. (Can be automated)
- Depending on the MS license you are using you maybe need to reactivate.
In my home lab I have a setup with has 1 VC VM and 2 ESXi hosts (Dell PowerEdge T410) which are connected to a Equallogic PS5000 SAN.
The VC and the VC database are both on a different VM.
While I was debugging a powerCLI script for a customer I noticed that my vCenter Service stopped working (and tried to restart automaticaly) without success.
In the Eventviewer I saw following errors:
- In the System Log
The VMware VirtualCenter Server service terminated unexpectedly. It has done this 1 time(s). The following corrective action will be taken in 300000 milliseconds: Restart the service.
- In the Application Log
An unrecoverable problem has occurred, stopping the VMware VirtualCenter service. Error: Error[VdbODBCError] (-1) “ODBC error: (42000) – [Microsoft][SQL Native Client][SQL Server]The transaction log for database ‘VC’ is full. To find out why space in the log cannot be reused, see the log_reuse_wait_desc column in sys.databases” is returned when executing SQL statement “UPDATE VPX_DATASTORE WITH (ROWLOCK) SET NAME = ? , STORAGE_URL = ? , CAPACITY = ? , FREE_SPACE = ? , TYPE = ? , DATACENTER_ID = ? , IORM_ENABLED = ? , IORM_CONGESTION_THRESHOLD = ? WHERE ID = ?”
- In the C:\Users\All Users\VMware\VMware VirtualCenter\Logs I saw erros like “Connection refused”
The answer to all this was lack of space on the drive the vCenter database was located. This because SQL Recovery option on the vCenter database was set to “Full Recovery” and there was NO maintenance plan active. This was the cause of a growing LOG file. After running the maintenance plan the log was emptied and a lot of disk space became available.
At a customer site I have an ESXi server which give me the possibility in the Memory configuration to change the Service Console memory size. But as you know in ESXi there is no more Service Console.
Before I changed something on this server I have put it in maintenance mode. This is a server in Production.
Steps I did without result:
- Restart of the management agents did not fix the issue
- Restart of ESX did not change any thing, setting still available.
- Disconnected the server from vCenter, still Service Console Memory available.
- Reinstalled vpx Agent on the ESXi using VMware procedure (see VMware KB 1003714), still no fix.
This last step fixed the issue
Removed the ESXi from vCenter and added it again. This fixed the issue. Now it looks like a normal ESXi server.
Now we know ESXi is the way to go we need to start to trancition all our ESX servers to ESXi. This is said quicker then done. During a certain period we have build a lot of experiance managing these ESX servers. On of the biggest changes is that there is no more Service Console to login to to watch (using tail / less / more) the log files.
There are now some other possibilities to watch your log files.
Via the DCUI (Consolee of the ESXi). Press F2 (and the password to get to the different options
Using following URL http://<insert ESXi host name or IP here>/host
Using the vSphere Client direct to the ESXi
Using the vSphere client connected to the vCenter and then select the ESXi server you want to export logs from.
Export the logs using PowerCLI using the cmdlet get-log
And the last way is redirect all logs to a Syslog server. Here some examples of Syslog servers you can use. Depending on the needs you can use the FREE version of these syslog servers
Vmware vMA Appliance www.vmware.com/support/developer/vima/vima40/doc/vma_40_guide.pdf#page=26
KIWI syslog server (Windows Service)
SyslogAppliance (Linux Appliance)
www.splunk.com/ (Linux Appliance)
Other suggestions of nice syslog setups are welcome and will be added in here.