How To Safely Shut Down Your ESXi and Nutanix AOS Cluster

esxi-aos-cluster-shutdown.jpg

About

Shut down all hosts in a Nutanix cluster that is running VMware ESXi for maintenance.

General Process

1. Run NCC Health Check via Nutanix Web Console
2. Shut down all VMs in the vCenter
3. Shut down the vCenter (VM)
4. Stop the Nutanix Cluster and Shut down CVM
5. Shut down each Node (Host)
6. Powering on the nodes and cluster after a shutdown
7. Start Nutanix cluster
8. Start All VMs

1. Run NCC Health Check via Nutanix Web Console

Note: Upgrade to the latest version of NCC before proceeding with the following steps.

i. Upgrade NCC to latest version

2023-07-06_132540.png

ii. Run NCC Health Check

2023-07-06_132601.png

Check that everything is working well, then could proceed to the next step.

2. Shut down All VMs in the vCenter

NOTE : Do not shut down the Nutanix Controller VMs (CVM).

i. The DRS setting must be set to 'Manual' before shutting down all VMs.

2022-12-29_160831.png

ii. Start the process of shutting down all virtual machines using vCenter.

3. Shut down the vCenter (VM)

Note: Remember the host on which the vCenter is currently running.

* Host will be assigned automatically based on the cluster, and their names should fall within the range of vms01-08.

2023-07-07_143216.png

Method 1: Login - vCenter Server Management to shut down (via Browser)

2023-07-06_133219.png

Method 2: To shut down via Remote Console

2023-07-06_133304.png

4. Stop the Nutanix Cluster and Shut down CVM

i. Stop the Nutanix cluster

Log in to one of CVM using SSH and execute the following command to stop the Nutanix cluster:

 cluster stop

Wait until output similar to the following is displayed for every Controller VM in the cluster before proceeding:

2023-07-07_143944.png

ii. Shut down CVM:

Referring to the 'HCI IP Information Table', log in to each CVM using SSH and execute the following command to shut down:

 sudo shutdown -P now

5. Shut down each Node (Host)

Method01: via GUI

Log in to each node (ESXi) using a browser to follow the indicators.

i. Put host in maintenance mode:

2023-07-07_150321.png

ii. Shut down the host:

2023-07-07_150334.png

Method02: via Command line

Log in to each Node (ESXi) using SSH and execute the following command

i. Put host in maintenance mode:

 esxcli system maintenanceMode set --enable true

ii. Shut down the host:

 poweroff

6. Powering on the Nodes and Cluster

After the shutdown

i. Power On all Nodes via IPMI. (Or turn it on by pressing the physical power button on the front)

2023-07-07_164950.png

ii. Log in to each node (ESXi) using a browser and select Exit Maintenance Mode.

2023-07-07_150321-1688971682024.png

​ Or, log in to each Node using SSH and execute the following command

 esxcli system maintenanceMode set --enable false

iii. Log in the node that previous was recorded (e.g.vms01) , then Right-click the vCenter and select Power > Power on.

2023-07-07_170424.png

​ Or, log in to the Node (vms01) using SSH and execute the following command

vim-cmd vmsvc/power.on "$(vim-cmd vmsvc/getallvms | grep -ia "vCenter" | cut -d ' ' -f 1)"

Wait for approximately 5 minutes for all services to start on the VM.

iv. Right-click each CVM and select Power > Power on via vCenter

2023-07-07_171706.png

​ Or, log in to Each Node (ESXi) using SSH and execute the following command

vim-cmd vmsvc/power.on "$(vim-cmd vmsvc/getallvms | grep -ia "NTNX-*-CVM" | cut -d ' ' -f 1)"

7. Start Nutanix Cluster

i. Log on to any of the CVMs in the cluster via SSH with the nutanix user credentials.

ii. Start the Nutanix cluster by issuing the following command:

cluster start

When the cluster starts, output similar to the following is displayed for each Controller VM in the cluster:

2023-07-10_103716.png

iii. Confirm that all cluster services are running on the Controller VMs.

cluster status

2023-07-11 03:38:45,982Z INFO MainThread cluster:3088 Success! (*The last in line.)

iv. Confirm that all Nutanix datastores are available:

Right-click the ESXi host in the vSphere client and select Rescan for Datastores.

2023-07-10_104707.png

8. Start All VMs

i. Power On all virtual machines using vCenter.

ii. The DRS setting must be set to 'Fully Automated' after Power On all VMs.

2023-07-07_143123.png

Reference

• ESXi - Shutting down the cluster​ - Maintenance or Re-location
• Shutting down Nutanix cluster running VMware vSphere for maintenance or relocation
• Shutting Down an ESXi Node in a Nutanix Cluster
• Shutting Down a Node in a Cluster (vSphere Command Line)
• Failing Over a Protection Domain (Planned Failover)