There are a few reasons that will degrade the performance of virtual machines such as a disk I/O bottleneck, resource contention, misconfiguration and other problems.
When too many VMs compete for limited resources, results will inevitably suffer.
The effort it takes to solve the storage I/O bottleneck is a variety of ways, it may be from physical infrastructure choices to configuration changes.
The starting point for solving your I/O problems will be VM monitoring and establishing baseline performance.
From there, you can examine the IOPS read vs. write IOPS, network cards, and certain points which may contribute to the I/O bottlenecks.
With the right approach and the right technology, administrators can resolve the problems that caused the dismal performance of the infrastructure.
Using ESXi Shell
When performing troubleshooting with ESXi/ESX storage, use command line tools which require you to identify a specific disk or LUN connected to ESXi/ESX.
This article provides information on different ways to identify these disks which ais via ESXI Shell. This can be part of troubleshooting, primarily due to the storage component of the virtual environment.
ESXi Shell provides a command-line interface that allows administrators to interact with the ESXi host directly. To list volumes using the ESXi Shell, follow these steps:
Display availabke command for esxcli storage :
~ # esxcli storage
Usage: esxcli storage {cmd} [cmd options]
Available Namespaces:
core VMware core storage commands.
nfs Operations to create, manage, and remove Network Attached Storage filesystems.
nmp VMware Native Multipath Plugin (NMP). This is the VMware default implementation of the Pluggable Storage Architecture.
san IO device management operations to the SAN devices on the system.
vflash virtual flash Management Operations on the system.
vmfs VMFS operations.
filesystem Operations pertaining to filesystems, also known as datastores, on the ESX host.
Display availabke command for esxcli storage -> vmfs :
~ # esxcli storage vmfs
Usage: esxcli storage vmfs {cmd} [cmd options]
Available Namespaces:
snapshot Manage VMFS snapshots.
extent Manage VMFS extents.
Available Commands:
unmap Reclaim the space by unmapping free blocks from VMFS Volume
upgrade Upgrade a VMFS3 volume to VMFS5.
~ # esxcli storage vmfs extent
Usage: esxcli storage vmfs extent {cmd} [cmd options]
Available Commands:
list List the VMFS extents available on the host.
List the VMFS extents available on the host :
~ # esxcli storage vmfs extent list Volume Name VMFS UUID Extent Number Device Name Partition ----------- ----------------------------------- ------------- -------------------------------------------------------------------------- --------- datastore1 52a0db3f-d87636dc-61bb-28924a2f1bc0 0 t10.ATA_____VB0250EAVER_____________________________Z3TDY30B____________ 3 datastore2 53f6103b-e2c46411-13e9-28924a2f1bc0 0 t10.ATA_____WDC_WD1600AAJS2D00L7A0________________________WD2DWCAV36165769 1
~ # esxcli storage filesystem
Usage: esxcli storage filesystem {cmd} [cmd options]
Available Commands:
automount Request mounting of known datastores not explicitly unmounted.
list List the volumes available to the host. This includes VMFS, NAS, VFAT and UFS partitions.
mount Connect to and mount an unmounted volume on the ESX host.
rescan Issue a rescan operation to the VMkernel to have is scan storage devices for new mountable filesystems.
unmount Disconnect and unmount and existing VMFS or NAS volume. This will not delete the configuration for the volume, but will remove the
volume from the list of mounted volumes.
List the volumes available to the host. This includes VMFS, NAS and VFAT partitions :
~ # esxcli storage filesystem list Mount Point Volume Name UUID Mounted Type Size Free ------------------------------------------------- ----------- ----------------------------------- ------- ------ ------------ ----------- /vmfs/volumes/52a0db3f-d87636dc-61bb-28924a2f1bc0 datastore1 52a0db3f-d87636dc-61bb-28924a2f1bc0 true VMFS-5 244544700416 52979302400 /vmfs/volumes/53f6103b-e2c46411-13e9-28924a2f1bc0 datastore2 53f6103b-e2c46411-13e9-28924a2f1bc0 true VMFS-5 159987531776 14871953408 /vmfs/volumes/556a99d3-4dc8997f-b819-28924a2f1bc0 556a99d3-4dc8997f-b819-28924a2f1bc0 true vfat 4293591040 4255121408 /vmfs/volumes/4a052e7f-ed60f807-e327-66c4dcb34a59 Hypervisor1 4a052e7f-ed60f807-e327-66c4dcb34a59 true vfat 261853184 96874496 /vmfs/volumes/25230d7e-4dcb47f9-390e-25a3db175fe7 Hypervisor2 25230d7e-4dcb47f9-390e-25a3db175fe7 true vfat 261853184 96976896 /vmfs/volumes/2da668ef-40e5d96b-90bf-855ddb9c5547 Hypervisor3 2da668ef-40e5d96b-90bf-855ddb9c5547 true vfat 299778048 96681984
~ # esxcli storage nmp
Usage: esxcli storage nmp {cmd} [cmd options]
Available Namespaces:
psp Operations pertaining to the Path Selection Policy Plugins for the VMware Native Multipath Plugin.
satp Operations pertaining to the Storage Array Type Plugins for the VMware Native Multipath Plugin.
device Operations pertaining to the devices currently claimed by the VMware Native Multipath Plugin.
path Operations pertaining to the paths currently claimed by the VMware Native Multipath Plugin.
~ # esxcli storage nmp device
Usage: esxcli storage nmp device {cmd} [cmd options]
Available Commands:
list List the devices currently controlled by the VMware NMP Multipath Plugin and show the SATP and PSP information associated with that
device.
set Allow setting of the Path Selection Policy (PSP) for the given device to one of the loaded policies on the system.
List the devices currently controlled by the VMware NMP Multipath Plugin and show the SATP and PSP information associated with that device :
~ # esxcli storage nmp device list
t10.ATA_____VB0250EAVER_____________________________Z3TDY30B____________
Device Display Name: Local ATA Disk (t10.ATA_____VB0250EAVER_____________________________Z3TDY30B____________)
Storage Array Type: VMW_SATP_LOCAL
Storage Array Type Device Config: SATP VMW_SATP_LOCAL does not support device configuration.
Path Selection Policy: VMW_PSP_FIXED
Path Selection Policy Device Config: {preferred=vmhba0:C0:T0:L0;current=vmhba0:C0:T0:L0}
Path Selection Policy Device Custom Config:
Working Paths: vmhba0:C0:T0:L0
Is Local SAS Device: false
Is USB: false
Is Boot USB Device: false
t10.ATA_____WDC_WD1600AAJS2D00L7A0________________________WD2DWCAV36165769
Device Display Name: Local ATA Disk (t10.ATA_____WDC_WD1600AAJS2D00L7A0________________________WD2DWCAV36165769)
Storage Array Type: VMW_SATP_LOCAL
Storage Array Type Device Config: SATP VMW_SATP_LOCAL does not support device configuration.
Path Selection Policy: VMW_PSP_FIXED
Path Selection Policy Device Config: {preferred=vmhba32:C0:T0:L0;current=vmhba32:C0:T0:L0}
Path Selection Policy Device Custom Config:
Working Paths: vmhba32:C0:T0:L0
Is Local SAS Device: false
Is USB: false
Is Boot USB Device: false