Top 4 – ESXi one-liner scripts

bash-148836_960_720I have caught myself quiet a few times running a bunch of one-liner scripts on ESXi.
Whether I setup ESXi, troubleshoot something or just test a feature, one-liners can often simplify whatever task you’re trying to complete.

Below is the list of one-liner scripts, which I use at least once a week.

Read More

ESXi Fibre Channel Configuration Maximums

Today we ran into a issue where we could not see all the LUNs presented to our hosts. In this case, we had multiple Nimble arrays connected already. We added 6 more LUNs from one of the Nimbles to our initiator group that is going to the ESXi host. We checked the logged in initiators, the initiator group, and the zones and EVERYTHING appeared to be connected correctly. We have 4 initiators coming out of the ESXi host and all 4 were logging into the fabric, array, and showing connections to the LUNs.  However… not all of the LUNs were showing up under storage adapters on the host.  This is what we saw:

HBA1 sees 0, 1, 2 and 3

HBA2 sees 0, 1 and 2

HBA3 sees 0, 1, 2, 3, 4 and 5

HBA4 sees 0, 1 and 2

Needless to say… That was kind of odd since everything was showing up as logged in. We rescanned multiple times and we restarted the vSphere Client for good measure.  Eventually, we ran the command:  esxcli storage core adapter rescan –all from the command line.  When we did THAT… the system spit out a bunch of errors that were pretty close to this:

The maximum number of supported paths of 1024 has been reached. Path vmhba2:C0:T0:L3 could not be added.
The number of paths allocated has reached the maximum: 1024. Path: vmhba5:C0:T6:L28 will not be allocated. This warning won’t be repeated until some paths are removed.

 If you look here:

 http://www.vmware.com/pdf/vsphere5/r55/vsphere-55-configuration-maximums.pdf

You will see that ESXs supports a MAXIMUM of 1024 paths per SERVER.  This is not a per ADAPTER thing…

We’ve seen some of the ESX Maximums in the past with iSCSI but usually we hit the device limit long before we hit the path limit.  

So two things to learn from this:

  • Be aware that the vSphere Client won’t always be verbose about what’s going on.
  • Be aware of the ESXi Configuration Maximums. The way they have it is that whichever limit you hit first wins. It’s not a this, this, and this sorta thing.  It’s a this or this or this kind of thing.

Crucial Data In Your VMware ESXi 5 Log Files

As an Escalation Engineer, part of my daily work is reviewing log files of various systems and vendors. In my first blog post, I would like to show which VMware ESXi 5 log files are most relevant for troubleshooting storage and networking related problems.

All current ESXi 5 logs are located under /var/log and as they rotate, they’ll be available under /scratch/logs

 

vmware_esxi_logs

/var/log/vmkernel.log:

  • VMkernel related activities, such as:
    • Rescan and unmount of storage devices and datastores
    • Discovery of new storage like iSCSI and FCP LUNs
    • Networking (vmnic and vmks connectivity)

/var/log/vmkwarning.log:

  • Extracted warning and alert messages from the vmkernel.log

/var/log/hostd.log::

  • Logs related to the host management service
  • SDK connections
  • vCenter tasks and events
  • Connectivity to vpxa service, which is the vCenter agent on the ESXi server

/var/log/vobd.log:

  • VMkernel observations
  • Useful for network and performance issues

Also, if you have a VM which is affected in particular, it might be worth looking into the vmware.log which is stored with the Virtual Machine. You can find the log under /vmfs/volumes/datastore_name/VM_name/vmware.log.

For the location of ESXi 3.5 and 4.x log files, can be found here.