No Host is Compatible with Virtual Machine ESXi 5.5

Recently, while upgrading ESXi 5.0 Update 2 to ESXi 5.5, faced an issue that several virtual machines were not able to be vMotioned to any ESXi 5.5 servers. The error message shown on the vSphere client task pane was:

No host is compatible with the virtual machine.

Yes, it looks so obvious that something isn’t compatible with ESXi 5.5 on these virtual machines. Checked the settings on a virtual machine and found that:

      • SCSI controller: BusLogic Parallel.
      • Operating System: Debian Linux 5 (32-bit)

Odd thing was that the clone of this virtual machine was fine, i.e. one for production and another for test/dev. The settings were exactly the same. What’s the next step? Of course, vmkernel.log.

vmkernel.log

2014-02-09T23:02:02.970Z cpu6:3550541)P2MCache: 545: vm 4895049: GetPhysMemRange failed for PPN 0x1c9af4 canBlock 1 count 98 status Out of slots

2014-02-09T23:02:02.970Z cpu6:3550541)VmMemCow: 1599: p2m update: cannot reserve – cur 2309 2432 rsvd 0 req 1 avail 2432

2014-02-09T23:02:02.970Z cpu6:3550541)P2MCache: 545: vm 4895049: GetPhysMemRange failed for PPN 0x18bff5 canBlock 1 count 99 status Out of slots

2014-02-09T23:02:02.970Z cpu6:3550541)VmMemCow: 1599: p2m update: cannot reserve – cur 2309 2432 rsvd 0 req 1 avail 2432

2014-02-09T23:02:02.970Z cpu6:3550541)VmMemCow: 1599: p2m update: cannot reserve – cur 2309 2432 rsvd 0 req 1 avail 2432

2014-02-09T23:02:02.970Z cpu6:3550541)VmMemCow: 1599: p2m update: cannot reserve – cur 2309 2432 rsvd 0 req 1 avail 2432

2014-02-09T23:02:23.480Z cpu9:5982)Config: 346: “SIOControlFlag2” = 1, Old Value: 0, (Status: 0x0)

2014-02-09T23:03:53.550Z cpu4:12576)Config: 346: “SIOControlFlag2” = 0, Old Value: 1, (Status: 0x0)

Looking at the vmkernel.log, I could see that reserving memory was causing an issue. To fix the problem, I did the following:

  1. Remove memory reservation
  2. Storage vMotion only the configuration file to another VMFS volume
  3. vMotion
  4. Storage vMotion back the configuration file to the original VMFS volume
  5. Create memory reservation

After upgrading to ESXi 5.5 and see the error above, don’t panic. It can be simply fixed with vMotion and storage vMotion.

webCommander Tip #2

This entry is part 2 of webCommander Tip blog series and those who have missed the first one, check it out here webCommander Tip #1

Introduction

By default, webCommander could output a result in a table format on a browser. What if the user wants to send the result to him/herself or to another via email to keep it? It could be achieved by adding/modifying few lines and functions.

Pre-requisites

Before going through the post, the users must meet the pre-requisites below:

    • An email user that will be sending an email out, i.e. email sender
    • SMTP server

Configuration

Send-Mail function

The below attached is the function called Send-Mail that has 4 arguments:

    • $To => The user who wants to receive the result.
    • $Subject => The subject of the email.
    • $Body => The body of the email, this will be for the result.
    • $Attachement => If the user wants to keep the result in .csv format, it can be sent as an attachment. For our case, this is by default even the $Attachments parameter is set to Mandatory=$false.

It uses a native PowerShell Send-MailMessage function (more details can be found by Get-Help Send-MailMessage) and written in the objects.ps1 file that can be loaded by any PowerCLI functions:

Function Send-Mail
{
  param(
    [Parameter(Mandatory=$true)]
    [string[]]$To,
    [string]$Subject,
    [string]$Body,
    [Parameter(Mandatory=$false)]
    [string]$Attachments
  )

  $mail_sender = "sender@test.com"
  $mail_smtp_server = "mailhost.test.com"

  if ($Attachments)
  {
    Send-MailMessage -from $mail_sender -to $To -subject $Subject -BodyAsHTML -Body $Body -smtpServer $mail_smtp_server -Attachments $Attachments
  }
  else
  {
    Send-MailMessage -from $mail_sender -to $To -subject $Subject -BodyAsHTML -Body $Body -smtpServer $mail_smtp_server
  }
}

webCmd.xml

Obviously, an input box (parameter) has to be provided to the users. I created an entity called Email that is optional:

 <!ENTITY Email '
    <parameter optional="1" name="EmailTo" description="Email address to send the report" />
  '>

Now this can be added to any commands i.e. PowerCLI scripts, an example below:

<command name="Query-Powered-Off-VM" description="[vSphere] Query powered off virtual machines">
<script>Query-Powered-Off-VM</script>
<parameters>
  &ConnectvCenter;
  &Cluster;
  &Email;
</parameters>
</command>

Actual Script

By now, the input parameter is created and Send-Mail function is added to objects.ps1. Let’s go through how to use them on the actual scripts.

The following is an example script that queries datastores under several conditions ($expression is the regular expression that I used for the filtering):

$output = foreach ($cluster in (Get-Cluster -Name $Cluster -Server $server.viserver -wa 0 -EA stop -ErrorVariable error_message)) { 
  $cluster | Get-VMHost | Get-Datastore | Where-Object {$_.Name -match $expression -and $_.FreespaceGB -ge $Freespace} | Sort Name | Select @{N="Cluster";E={$cluster.Name}}, Name, FreespaceGB, CapacityGB }

Once the report is saved to $output variable, it can be sent via email by converting the output into HTML as a string format:

$output_html = $output | ConvertTo-Html | Out-String
Send-Mail -To $EmailTo -Subject "Query-Datastore Report" -Body $output_html

On your email, it will look like this:

Cluster Name FreeSpaceGB CapacityGB
Cluster1 Datastore1 559.3359375 1023.75
Cluster1 Datastore2 525.53125 1023.75
Cluster1 Datastore3 394.1640625 1023.75
Cluster1 Datastore4 344.9453125 1023.75
Cluster1 Datastore5 236.609375 511.75

Now, let’s go through how to send the result as an attachment. One thing to consider is that when the output is saved as a .csv file on C: drive somewhere, depends on the amount of data the user queries, the size of the file might be big which will eventually fill up the drive. So, when attaching a file I decided to:

    • Export the output as a .csv file
    • Send an email with the attachment
    • Delete the file

This way, the administrator doesn’t have to worry about filling up the drive 🙂 The script is attached below:

$output | Export-CSV "C:\script\output\Query-Datastore.csv" -NoTypeInformation -UseCulture
$output_html = $output | ConvertTo-Html | Out-String
Send-Mail -To $EmailTo -Subject "Query-Datastore Report" -Body $output_html -Attachments "C:\script\output\Query-Datastore.csv"
Remove-Item -Path "C:\script\output\Query-Datastore.csv"

Summary

Not only outputting the result on a web browser, it can also be sent to the users via email. This way, people could save the report and utilise it using Excel…etc.

On the next series, I will be going through how to upload a .csv file to make changes to virtual machines.

webCommander Tip #1

I will be posting several tips on webCommander by #VMware #Fling for the next few weeks. For more information on the software, refer to http://labs.vmware.com/flings/web-commander. Also, it’s good to install a SSL certificate which can be directed to my old post https://ssbkang.com/2014/02/11/webcommander-ssl-certificate-installation/

Introduction

Using webCommander to output a table i.e. a result, the administrator must create a result table defined in webCmd.xsl, an example below:

<xsl:if test="result/datastore">
  <center>
    <table width="100%">
      <tr>
        <th>Datastore</th>
        <th>freespace</th>
      </tr>
      <xsl:for-each select="result/datastore">
        <tr>
          <td><xsl:value-of select="datastore" /></td>
          <td><xsl:value-of select="freespace" /></td>
        </tr>
      </xsl:for-each>
    </table>
  </center>
</xsl:if>

And on your script:

write-output "<datastore>"
write-output "<datastore>$datastore</datastore>"
write-output "<freespace>$freespace</freespace>"
write-output "</datastore>"

It’s quite simple to output a result but one problem is that when the number of output table increases, the webCmd.xsl file becomes massive and hard to manage. Later on, I ask myself “Did I add this and this and that and that” and takes a time to look for the table.

configuration

To make the script simpler, under webCmd.xsl file, added a simple if statement. There will be only one table defined in webCmd.xsl file which I named custom and has all variables used for all scripts, shown below:

<!-- Modify Here !-->
<xsl:if test="result/custom">
  <center>
    <table class="exceptionTable" width="100%">
      <tr>
        <xsl:if test="result/custom/cluster"><th>Cluster</th></xsl:if>
        <xsl:if test="result/custom/esxi"><th>ESXi</th></xsl:if>
        <xsl:if test="result/custom/vm"><th>Virtual Machine</th></xsl:if>
        <xsl:if test="result/custom/powerstate"><th>PowerState</th></xsl:if>                                
        <xsl:if test="result/custom/vmdk"><th>VMDK</th></xsl:if>
        <xsl:if test="result/custom/datastore"><th>Datastore</th></xsl:if>
        <xsl:if test="result/custom/freespacegb"><th>FreespaceGB</th></xsl:if>
        <xsl:if test="result/custom/capacitygb"><th>CapacityGB</th></xsl:if>
        <xsl:if test="result/custom/scsicontroller"><th>SCSIController</th></xsl:if>
        <xsl:if test="result/custom/customattribute"><th>Custom Attribute</th></xsl:if>
        <xsl:if test="result/custom/customattributevalue"><th>Custom Attribute Value</th></xsl:if>
        <xsl:if test="result/custom/ntpserver"><th>ntpserver</th></xsl:if>
        <xsl:if test="result/custom/service"><th>service</th></xsl:if>
        <xsl:if test="result/custom/status"><th>status</th></xsl:if>
        <xsl:if test="result/custom/resourcepool"><th>resourcepool</th></xsl:if>
        <xsl:if test="result/custom/path"><th>path</th></xsl:if>
        <xsl:if test="result/custom/networkadatper"><th>networkadatper</th></xsl:if>
        <xsl:if test="result/custom/macaddress"><th>macaddress</th></xsl:if>
      </tr>
      <xsl:for-each select="result/custom">
        <tr>
          <xsl:if test="cluster"><td><xsl:value-of select="cluster" /></td></xsl:if>
          <xsl:if test="esxi"><td><xsl:value-of select="esxi" /></td></xsl:if>
          <xsl:if test="vm"><td><xsl:value-of select="vm" /></td></xsl:if>
          <xsl:if test="powerstate"><td><xsl:value-of select="powerstate" /></td></xsl:if>
          <xsl:if test="vmdk"><td><xsl:value-of select="vmdk" /></td></xsl:if>
          <xsl:if test="datastore"><td><xsl:value-of select="datastore" /></td></xsl:if>
          <xsl:if test="freespacegb"><td><xsl:value-of select="freespacegb" /></td></xsl:if>
          <xsl:if test="capacitygb"><td><xsl:value-of select="capacitygb" /></td></xsl:if> 
          <xsl:if test="scsicontroller"><td><xsl:value-of select="scsicontroller" /></td></xsl:if>
          <xsl:if test="customattribute"><td><xsl:value-of select="customattribute" /></td></xsl:if>
          <xsl:if test="customattributevalue"><td><xsl:value-of select="customattributevalue" /></td></xsl:if>
          <xsl:if test="ntpserver"><td><xsl:value-of select="ntpserver" /></td></xsl:if>
          <xsl:if test="service"><td><xsl:value-of select="service" /></td></xsl:if>
          <xsl:if test="status"><td><xsl:value-of select="status" /></td></xsl:if>
          <xsl:if test="resourcepool"><td><xsl:value-of select="resourcepool" /></td></xsl:if>
          <xsl:if test="path"><td><xsl:value-of select="path" /></td></xsl:if>
          <xsl:if test="networkadatper"><td><xsl:value-of select="networkadatper" /></td></xsl:if>
          <xsl:if test="macaddress"><td><xsl:value-of select="macaddress" /></td></xsl:if>
        </tr>
      </xsl:for-each>
    </table>
  </center>
</xsl:if>
<!-- Modify Ends !-->

Now, your script can have something like:

write-output "<custom>"
write-output "<cluster>$cluster</cluster>"
write-output "<datastore>$datastore</datastore>"
write-output "<freespacegb>$freespacegb</freespacegb>"
write-output "<capacitygb>$capacitygb</capacitygb>"
write-output "</custom>"

Or:

write-output "<custom>"
write-output "<cluster>$cluster</cluster>"
write-output "<vm>$vm</vm>"
write-output "<customattribute>$customattribute</customattribute>"
write-output "<customattributevalue>$customattributevalue</customattributevalue>"
write-output "</custom>"

Summary

Using an if statement in the table makes the script much simpler and easier to manage. There must be a better way but this is how I think 🙂

Update

Check out the latest comment by Jason, it also could be done by using JSON here comment