PowerCLI Report – vMSC Health Check

Introduction

I recently wrote a PowerCLI script to check the health state of Metro Storage Cluster. The intention of this script is to:

• Ensure the cluster setting’s correct
• Check compute resources on each datacenters
• Check uniform access
• Check DRS group

Throughout this post, it will be going over the script & explanation and sample output.

One assumption is that this report is for uniform access design.

Script

As mentioned in the introduction, the script consists of 4 reports. The first one will show the cluster configuration:

1. Admission control policy enabled?
2. Admission control policy CPU/Memory
3. Isolation Addresses
4. Number of Heartbeat datastores
5. List of Heartbeat datastores

This will allow the administrators to check and make sure the cluster setting is correct.

The second one is the resource report. This will represent CPU/Memory usage of ESXi servers of each datacenter that will help administrators to decided which datacenter needs to be used when deploying virtual machines to balance out the resource usage.

The third one is the uniform check report. The purpose of this report is to ensure that virtual machines are uniformly accessing ESXi servers as well as datastores. For example, if a virtual machine is running in site 1 but using datastore(s) in site 2, it introduces extra latency between the virtual machine and datastore. DRS rule should handle this but if the virtual machine is not in DRS group or somehow DRS didn’t do the job, it has to be corrected manually.

Lastly, DRS group report. Any virtual machines not in DRS group will be shown. It will list ESXi and datastore(s) so that administrators know which DRS group to put in.

In this script, there are inputs to be replaced, in bold and underline:

• vCenter server, username and password
• Cluster name
• Site expression, 1 and 2. This totally depends on your naming convention, an example is below
  • “s1|site1”
  • “s2|site2”
• Mail settings

This script is per cluster base so if you have more than 1 cluster, it will be required to run the script multiple times. Attached below:

Connect-VIServer -Server "vCenter Address" -User "vCenter User" -Password "vCenter User Password"

$cluster = Get-Cluster -Name "Cluster Name"
$site_1_expression = "^s1|^site1"
$site_2_expression = "^s2|^site2"
$esxi_list = Get-VMHost -Location $cluster
$datastore_list = Get-Datastore -VMHost $esxi_list
$virtual_machine_list = Get-VM -Location $cluster | Sort Name
$drs_group_list = $cluster.ExtensionData.Configurationex.Group | ?{$_.VM}

$configuration_report = @()
$resource_report = @()
$uniform_report = @()
$drs_report = @()

## 1st Report
## Cluster Settings

$heartbeat_datastore_list = $cluster.ExtensionData.Configuration.DasConfig.HeartbeatDatastore.value | ForEach-Object { $id = $_; $datastore_list | where {$_.id -replace "datastore_list-" -match $id } | %{$_.Name} } 

$configuration_report = $cluster | select @{N="Cluster";E={$_.Name}}, 
                                          @{N="Admission Control Policy";E={$_.ExtensionData.Configuration.DasConfig.AdmissionControlEnabled}},
                                          @{N="Admission Control Policy CPU";E={$_.ExtensionData.Configuration.DasConfig.AdmissionControlPolicy.CpuFailoverResourcesPercent}}, 
                                          @{N="Admission Control Policy Memory";E={$_.ExtensionData.Configuration.DasConfig.AdmissionControlPolicy.MemoryFailoverResourcesPercent}},
                                          @{N="Isolation Addresses";E={[string]::Join(",", ($_.ExtensionData.Configuration.DasConfig.Option | where {$_.Key -match "isolation"} | %{$_.Value}))}},
                                          @{N="Heartbeat Datastore #";E={$_.ExtensionData.Configuration.DasConfig.Option | where {$_.Key -match "heartbeat"} | %{$_.Value}}},
                                          @{N="Heartbeat Datastore";E={[string]::Join(",", ($heartbeat_datastore_list))}}


## 2nd Report
## Resource Compute Usage

$resource_report = "" | select @{N="Site1 CPU Usage";E={ "{0:P1}" -f ( (($esxi_list | where {$_.Name -match $site_1_expression}).CpuUsageMHz | Measure-Object -Sum).Sum / (($esxi_list | where {$_.Name -match $site_1_expression}).CpuTotalMHz | Measure-Object -Sum).Sum ) }},
                               @{N="Site1 Memory Usage";E={ "{0:P1}" -f ( (($esxi_list | where {$_.Name -match $site_1_expression}).MemoryUsageGb | Measure-Object -Sum).Sum / (($esxi_list | where {$_.Name -match $site_1_expression}).MemoryTotalGB | Measure-Object -Sum).Sum ) }},
                               @{N="Site2 CPU Usage";E={ "{0:P1}" -f ( (($esxi_list | where {$_.Name -match $site_2_expression}).CpuUsageMHz | Measure-Object -Sum).Sum / (($esxi_list | where {$_.Name -match $site_2_expression}).CpuTotalMHz | Measure-Object -Sum).Sum ) }},
                               @{N="Site2 Memory Usage";E={ "{0:P1}" -f ( (($esxi_list | where {$_.Name -match $site_2_expression}).MemoryUsageGb | Measure-Object -Sum).Sum / (($esxi_list | where {$_.Name -match $site_2_expression}).MemoryTotalGB | Measure-Object -Sum).Sum ) }} 

## 3rd/4th Report
## DRS Group Report & Uniform Report

foreach ($vm in $virtual_machine_list) {
    $esxi = $vm.VMHost.Name

  $datastore = $vm.DatastoreIdList | Foreach-Object {
        $datastore_id = $_
        $datastore_list | where {$_.id -match $datastore_id} | %{$_.Name}
    }

    foreach ($d in $datastore) { 
        if ( ($esxi -match $site_1_expression -and $d -match $site_1_expression) ) {
            $uniform = "Yes"
        } elseif ( ($esxi -match $site_2_expression -and $d -match $site_2_expression) ) {
            $uniform = "Yes"
        } else {
            $uniform = "No"
            break
        }
    }
    
    $drs_group = $drs_group_list | where {$_.VM -eq $vm.id} | %{$_.Name}

    if (!$drs_group) {
        $drs_group = "No DRS Group"
        
        $drs_report += ("" | select    @{N="VM";E={$vm.Name}},
                                       @{N="ESXi";E={$esxi}},
                                       @{N="VMFS";E={[string]::Join(",", $datastore)}},
                                       @{N="DRS Group";E={$drs_group}} )    
    }
    
    if ($uniform -eq "No") {         
        $uniform_report += ("" | select @{N="VM";E={ $vm.Name }},
                                        @{N="ESXi";E={$esxi}},
                                        @{N="VMFS";E={ [string]::Join(",", $datastore) }},
                                        @{N="DRS Group Name";E={ $drs_group }},
                                        @{N="Uniform Access";E={ $uniform }} )
    }
}    

$header = @"
    <style>
    TABLE {border-width: 1px;border-style: solid;border-color: black;border-collapse: collapse;}
    TH {border-width: 1px;padding: 3px;border-style: solid;border-color: black;background-color: #6495ED;}
    TD {border-width: 1px;padding: 3px;border-style: solid;border-color: black;}
    </style>
"@

$body = "<h1>VMware Metro Storage Cluster Health Check</h1>"
$body += "<h2>Configuration Report</h2>" + ($configuration_report | ConvertTo-HTML -Head $header | Out-String)
$body += "<h2>Resource Report</h2>" + ($resource_report | ConvertTo-HTML -Head $header | Out-String)
$body += "<h2>Uniform Check Report</h2>"

if ($uniform_report.count -eq 0) {
  $uniform_report = "All virtual machines are uniformly accessing ESXi servers and VMFS volumes"
    $body += ConvertTo-HTML -Body $uniform_report | Out-String
} else {
    $body += "<h3>Please correct the following virtual machines</h3>"
    $body += $uniform_report | Sort VM | ConvertTo-HTML -Head $header | Out-String
}

$body += "<h2>DRS Group Report</h2>"
if ($drs_report.count -eq 0) { 
    $drs_report = "All virtual machines are in correct DRS groups"
    $body += ConvertTo-HTML -Body $drs_report | Out-String
} else {
    $body += "<h3>Please put the following virtual machines in appropriate DRS groups</h3>"
    $body += $drs_report | Sort VM | ConvertTo-HTML -Head $header | Out-String
}

Send-MailMessage -From "Sender mail address" -To "To email address" -Subject "vMSC Report" -BodyasHtml -Body $body -SmtpServer "SMTP server address"

Disconnect-VIServer * -Confirm:$false

Sample Output

Attaching sample outputs below:

The above example shows you that nonuniform_vm is running on Site 1 ESXi server but accessing Site 2 datastore. It’s in Site 1 DRS group so the VMDK should be storage vMotioned to a datastore in Site 1. Also, test1 and test2 virtual machines are not in DRS group. Based on ESXi and VMFS location, you know which DRS group to put these virtual machines in.

Attaching another example below:

In this case, all virtual machines are running uniformly that you don’t have to worry about it. However, still there are two virtual machines test1 and test2 need to be put into proper DRS group.

Hope this helps and always welcome to ask me any questions or issues with regard to this report.