Introduction
I’ve been managing multiple vCenter servers protected by VMware Heartbeat:
VMware® vCenter™ Server Heartbeat™ protects your virtual infrastructure from downtime related to application, configuration, operating system, network or hardware problems for high availability.
It has been calming for the last few months but all of a sudden, Heartbeat started complaining about C-Drive is filling up via email (I highly recommend you to configure SMTP settings on VMware Heartbeat, details can be found here). There was a potential risk that might kill the virtual machine.
In this post, I will be going through the workaround and permanent fix for this issue.
Investigation
First thing I had to check was what was filling up the C-Drive and I used a product called TreeSize. My guess was the Volume Shadow Copy was filling up the C-Drive but it wasn’t, the root cause was the log files located under C:\Program Files\VMware\VMware vCenter Server Heartbeat\tomcat\apache-tomcat-6.0.32\logs. Specifically, the files were called catalina.* and one of them was 30GB.
Workaround
To free up the C-Drive, the log files had to be deleted. Tried to delete them, however, it was unsuccessful. Why? It’s because VMware vCenter Heartbeat WebService was locking up the log files. Hence, I had to stop the service and then delete the files.
In summary:
- Stop VMware vCenter Server Heartbeat WebService
- Delete the log files starts with catlina
- Start VMware vCenter Server Heartbeat WebService
It does not impact vCenter Heartbeat Service that it can be performed at any time.
Permanent Fix
Permanent fix is quite simple, upgrade Heartbeat to 6.6. The fix is not listed on the release notes but I haven’t seen catlina.* logs filling up the C-Drive (it has been 4 months since Heartbeat was upgraded to 6.6).
Hope this helps to people who are facing the issue above.
Steven Kang 