Friday, November 25, 2016

Get-LockedOutLocation

Download lockout location finder

#Requires -Version 2.0 
Function Get-LockedOutLocation 
{ 
<# 
.SYNOPSIS 
    This function will locate the computer that processed a failed user logon attempt which caused the user account to become locked out. 
 
.DESCRIPTION 
    This function will locate the computer that processed a failed user logon attempt which caused the user account to become locked out.  
    The locked out location is found by querying the PDC Emulator for locked out events (4740).   
    The function will display the BadPasswordTime attribute on all of the domain controllers to add in further troubleshooting. 
 
.EXAMPLE 
    PS C:\>Get-LockedOutLocation -Identity mr.vayu
 
 
 
#> 
    [CmdletBinding()] 
 
    Param( 
      [Parameter(Mandatory=$True)] 
      [String]$Identity       
    ) 
 
    Begin 
    {  
        $DCCounter = 0  
        $LockedOutStats = @()    
                 
        Try 
        { 
            Import-Module ActiveDirectory -ErrorAction Stop 
        } 
        Catch 
        { 
           Write-Warning $_ 
           Break 
        } 
    }#end begin 
    Process 
    { 
         
        #Get all domain controllers in domain 
        $DomainControllers = Get-ADDomainController -Filter * 
        $PDCEmulator = ($DomainControllers | Where-Object {$_.OperationMasterRoles -contains "PDCEmulator"}) 
         
        Write-Verbose "Finding the domain controllers in the domain" 
        Foreach($DC in $DomainControllers) 
        { 
            $DCCounter++ 
            Write-Progress -Activity "Contacting DCs for lockout info" -Status "Querying $($DC.Hostname)" -PercentComplete (($DCCounter/$DomainControllers.Count) * 100) 
            Try 
            { 
                $UserInfo = Get-ADUser -Identity $Identity  -Server $DC.Hostname -Properties AccountLockoutTime,LastBadPasswordAttempt,BadPwdCount,LockedOut -ErrorAction Stop 
            } 
            Catch 
            { 
                Write-Warning $_ 
                Continue 
            } 
            If($UserInfo.LastBadPasswordAttempt) 
            {     
                $LockedOutStats +New-Object -TypeName PSObject -Property @{ 
                        Name                   = $UserInfo.SamAccountName 
                        SID                    = $UserInfo.SID.Value 
                        LockedOut              = $UserInfo.LockedOut 
                        BadPwdCount            = $UserInfo.BadPwdCount 
                        BadPasswordTime        = $UserInfo.BadPasswordTime             
                        DomainController       = $DC.Hostname 
                        AccountLockoutTime     = $UserInfo.AccountLockoutTime 
                        LastBadPasswordAttempt = ($UserInfo.LastBadPasswordAttempt).ToLocalTime() 
                    }           
            }#end if 
        }#end foreach DCs 
        $LockedOutStats | Format-Table -Property Name,LockedOut,DomainController,BadPwdCount,AccountLockoutTime,LastBadPasswordAttempt -AutoSize 
 
        #Get User Info 
        Try 
        {   
           Write-Verbose "Querying event log on $($PDCEmulator.HostName)" 
           $LockedOutEvents = Get-WinEvent -ComputerName $PDCEmulator.HostName -FilterHashtable @{LogName='Security';Id=4740} -ErrorAction Stop | Sort-Object -Property TimeCreated -Descending 
        } 
        Catch  
        {           
           Write-Warning $_ 
           Continue 
        }#end catch      
                                  
        Foreach($Event in $LockedOutEvents) 
        {             
           If($Event | Where {$_.Properties[2].value -match $UserInfo.SID.Value}) 
           {  
               
              $Event | Select-Object -Property @( 
                @{Label = 'User';               Expression = {$_.Properties[0].Value}} 
                @{Label = 'DomainController';   Expression = {$_.MachineName}} 
                @{Label = 'EventId';            Expression = {$_.Id}} 
                @{Label = 'LockedOutTimeStamp'; Expression = {$_.TimeCreated}} 
                @{Label = 'Message';            Expression = {$_.Message -split "`r" | Select -First 1}} 
                @{Label = 'LockedOutLocation';  Expression = {$_.Properties[1].Value}} 
              ) 
                                                 
            }#end ifevent 
             
       }#end foreach lockedout event 
        
    }#end process 
    
}#end function

PowerShell script to list all installed Microsoft Windows Updates

Create a function and then call it with specific parameters.

########################################################
Function Get-MSHotfix 
{ 
    $outputs = Invoke-Expression "wmic qfe list" 
    $outputs = $outputs[1..($outputs.length)] 
     
     
    foreach ($output in $Outputs) { 
        if ($output) { 
            $output = $output -replace 'y U','y-U' 
            $output = $output -replace 'NT A','NT-A' 
            $output = $output -replace '\s+',' ' 
            $parts = $output -split ' ' 
            if ($parts[5] -like "*/*/*") { 
                $Dateis = [datetime]::ParseExact($parts[5], '%M/%d/yyyy',[Globalization.cultureinfo]::GetCultureInfo("en-US").DateTimeFormat) 
            } else { 
                $Dateis = get-date([DateTime][Convert]::ToInt64("$parts[5]", 16))-Format '%M/%d/yyyy' 
            } 
            New-Object -Type PSObject -Property @{ 
                KBArticle = [string]$parts[0] 
                Computername = [string]$parts[1] 
                Description = [string]$parts[2] 
                FixComments = [string]$parts[6] 
                HotFixID = [string]$parts[3] 
                InstalledOn = Get-Date($Dateis)-format "dddd d MMMM yyyy" 
                InstalledBy = [string]$parts[4] 
                InstallDate = [string]$parts[7] 
                Name = [string]$parts[8] 
                ServicePackInEffect = [string]$parts[9] 
                Status = [string]$parts[10] 
            } 
        } 
    } 
} 
#############################################################


now call this function

Get-MSHotfix|Where-Object {$_.Installedon -gt ((Get-Date).Adddays(-2))}|Select-Object -Property Computername, KBArticle,InstalledOn, HotFixID, InstalledBy|Format-Table

Get-RemoteProgram Get list of installed programs on remote or local computer


.EXAMPLE 
Get-RemoteProgram 
 
Description: 
Will generate a list of installed programs on local machine 
 
.EXAMPLE 
Get-RemoteProgram -ComputerName server01,server02 
 
Description: 
Will generate a list of installed programs on server01 and server02 
 
.EXAMPLE 
Get-RemoteProgram -ComputerName Server01 -Property DisplayVersion,VersionMajor 
 
Description: 
Will gather the list of programs from Server01 and attempts to retrieve the displayversion and versionmajor subkeys from the registry for each installed program 
 
.EXAMPLE 
'server01','server02' | Get-RemoteProgram -Property Uninstallstring 
 
Description 
Will retrieve the installed programs on server01/02 that are passed on to the function through the pipeline and also retrieves the uninstall string for each program 

PS2EXE : Convert PowerShell Scripts to EXE Files

Download PS2EXE

Original description of version 0.1 with updates - especially in the "Usage" section

In the last days I created the tool “PS2EXE”. It is able to “convert” PowerShell scripts to “standalone” EXE files.
But: It does not convert the PowerShell script to an other language! It encapsulates the script with a lightweight PowerShell host written in C# and compiles the dynamically generated C# source code in memory to an EXE file. The resulting EXE is an .NET assembly that contains the source script encoded in Base64. The EXE includes all stuff that is needed to execute an PowerShell through the .NET object model. It is based on classes in the namespace System.Management.Automation that reperents the PowerShell engine. – Therefore the EXE file is not a real “standalone” EXE file. It needs PowerShell to be installed!!! And – of course – it needs .NET Framework v2.0. Furthermore “script execution” have to be allowed (see cmdlet: set-execultionpolicy). – The resulting EXE is “MSIL” and is able to execute as x64 or x86.
The tool “PS2EXE” itself is a PowerShell script! – It does the in-memory compilation and generates the EXE file. It uses the CSharpCodeProvider class of namespace Microsoft.CSharp.
The script is really simple. I contains a multiline string that represents the PowerShell host I’ve written. This is much more interesting than the PS2EXE.ps1 script itself. – Have a look into it!

Usage:
Call  the script with this parameters:
-inputFile PowerShell script file
-outputFile file name (with path) for the destination EXE file
-debug (switch) generate debug info in the destination EXE file. The dynamically generated .CS file will stored beside the output EXE file. Furthermore a .PDB file will be generated for the EXE file
-verbose (switch) shows also verbose informations – if any.
-x86 (switch) compile EXE to run as 32 bit application
-x64 (switch) compile EXE to run as 64 bit application
-runtime20 (switch) force running the EXE in PowerShell 2.0 using .NET 2.0
-runtime30 (switch) force running the EXE in PowerShell 3.0 using .NET 4.0
-lcid specify language ID for threads
-sta run PowerShell environment in Single Thread Apartment mode
-mta run PowerShell environment in Multithread Apartment mode
-noconsole compile PS script as Windows application
Sample:
 image_thumb[2]

Get product key of windows machine through powershell

function Get-ProductKey {
     <#  
   
    .EXAMPLE
     Get-ProductKey -Computername Server1
    

        Retrieves the product key information from 'Server1'
    #>        
    [cmdletbinding()]
    Param (
        [parameter(ValueFromPipeLine=$True,ValueFromPipeLineByPropertyName=$True)]
        [Alias("CN","__Server","IPAddress","Server")]
        [string[]]$Computername = $Env:Computername
    )
    Begin {  
        $map="BCDFGHJKMPQRTVWXY2346789"
    }
    Process {
        ForEach ($Computer in $Computername) {
            Write-Verbose ("{0}: Checking network availability" -f $Computer)
            If (Test-Connection -ComputerName $Computer -Count 1 -Quiet) {
                Try {
                    Write-Verbose ("{0}: Retrieving WMI OS information" -f $Computer)
                    $OS = Get-WmiObject -ComputerName $Computer Win32_OperatingSystem -ErrorAction Stop               
                } Catch {
                    $OS = New-Object PSObject -Property @{
                        Caption = $_.Exception.Message
                        Version = $_.Exception.Message
                    }
                }
                Try {
                    Write-Verbose ("{0}: Attempting remote registry access" -f $Computer)
                    $remoteReg = [Microsoft.Win32.RegistryKey]::OpenRemoteBaseKey([Microsoft.Win32.RegistryHive]::LocalMachine,$Computer)
                    If ($OS.OSArchitecture -eq '64-bit') {
                        $value = $remoteReg.OpenSubKey("SOFTWARE\Microsoft\Windows NT\CurrentVersion").GetValue('DigitalProductId4')[0x34..0x42]
                    } Else {                       
                        $value = $remoteReg.OpenSubKey("SOFTWARE\Microsoft\Windows NT\CurrentVersion").GetValue('DigitalProductId')[0x34..0x42]
                    }
                    $ProductKey = "" 
                    Write-Verbose ("{0}: Translating data into product key" -f $Computer)
                    for ($i = 24; $i -ge 0; $i--) {
                      $r = 0
                      for ($j = 14; $j -ge 0; $j--) {
                        $r = ($r * 256) -bxor $value[$j]
                        $value[$j] = [math]::Floor([double]($r/24))
                        $r = $r % 24
                      }
                      $ProductKey = $map[$r] + $ProductKey
                      if (($i % 5) -eq 0 -and $i -ne 0) {
                        $ProductKey = "-" + $ProductKey
                      }
                    }
                } Catch {
                    $ProductKey = $_.Exception.Message
                }       
                $object = New-Object PSObject -Property @{
                    Computername = $Computer
                    ProductKey = $ProductKey
                    OSDescription = $os.Caption
                    OSVersion = $os.Version
                }
                $object.pstypenames.insert(0,'ProductKey.Info')
                $object
            } Else {
                $object = New-Object PSObject -Property @{
                    Computername = $Computer
                    ProductKey = 'Unreachable'
                    OSDescription = 'Unreachable'
                    OSVersion = 'Unreachable'
                } 
                $object.pstypenames.insert(0,'ProductKey.Info')
                $object                          
            }
        }
    }
}

Monday, November 21, 2016

Windows Process Activation Service stops running unexpectedly in Windows Server 2008

Download hotfix to resolve this issue
Click here to download Hotfix


Symptoms
Consider the following scenario:
  • You have a server that is running Windows Server 2008.
  • You have Internet Information Services (IIS) 7.0 installed on the server.
In this scenario, the Svchost.exe process of Windows Process Activation Service (WAS) stops running unexpectedly.

Because the World Wide Web Publishing Service (W3SVC) depends on WAS, these services and your web application stop. Additionally, the following events are logged in the Application and System event logs:

Log Name: Application
Source: Application Error

Event ID: 1000
Level: Error
Description:
Faulting application name: svchost.exe_WAS, version: 6.0.6001.18000, time stamp: 0x47919291
Faulting module name: nativerd.dll, version: 7.0.6002.18139, time stamp: 0x4af812b4
Exception code: 0xc0000005
Fault offset: 0x000000000000e49f
Faulting process id: 0xf08
Faulting application start time: 0x01cebeec8563f730
Faulting application path: %11
Faulting module path: %12
Report Id: %13Log Name: System
Source: Service Control Manager

Event ID: 7034
Level: Error
Description:
The World Wide Web Publishing Service service terminated unexpectedly. It has done this 1 time(s).
Note This problem does not occur on a server that is running Windows Server 2008 R2 and has IIS 7.5 installed.
Cause
When this problem occurs, the Windows Process Activation Service refers to the null pointer during configuration changes. This causes the Svchost.exe process to crash and the websites to restart.
 
Resolution

Hotfix information:

A supported hotfix is available from Microsoft Support. However, this hotfix is intended to correct only the problem that is described in this article. Apply this hotfix only to systems that are experiencing the problem described in this article. This hotfix might receive additional testing. Therefore, if you are not severely affected by this problem, we recommend that you wait for the next software update that contains this hotfix.

Wednesday, November 9, 2016

Linux based file systems become read-only

As a workaround, remount the Linux file system using this command to return it to the proper state:


mount -o remount /

Virtual machines stop responding when any LUN on the host is in an all-paths-down (APD)

Details

The issue addressed in this article occurs when a rescan is issued while an all-paths-down state exists for any LUN in the vCenter Server cluster. Therefore, a virtual machine on one LUN stops responding (temporarily or permanently) because a different LUN in the vCenter Server cluster is in an all-paths-down (APD) state.

These symptoms may indicate you have an all-paths-down state:
  • You may see intermittent Request timed out (out of 6 or 7 successful pings) while trying to ping the virtual machine.
  • When powering on a virtual machine with a raw device mapping (RDM) the progress bar stops at 50% and the virtual machine console becomes unresponsive at the VMware splash/loading/BIOS screen.
  • In vSphere 4.0, virtual machines drop packets intermittently.
  • Virtual machines drop ping packets.
  • Network becomes temporarily unresponsive.
  • In the vmkernel.log file, you see entries similar to:

    NMP: nmp_DeviceAttemptFailover: Retry world failover device "naa._______________" - failed to issue command due to Not found (APD)
    NMP: nmp_DeviceUpdatePathStates: Activated path "NULL" for NMP device "naa.__________________".

Note
: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

All-Paths-Down State (APD)

The all-paths-down state is a condition where no working path exists to a storage device (LUN). These situations can cause an all-paths-down state to a LUN:
  • A hardware failure (permanent or transient)
  • Removal of a LUN

Rescanning

This article is concerned with the type of rescanning that can cause virtual machines on other LUNs to become unresponsive:
  • Configuration changes in ESXi/ESX 4.x involving VMFS (Virtual Machine File System)

    Configuration of VMFS that involves creating, deleting, or increasing datastores can cause an automated rescan. vCenter Server issues a vCenter Server-wide rescan as part of the workflow to discover storage changes. This rescan allows automatic discovery to maintain a consistent view of storage across all hosts in the cluster. For example, to perform a rescan in ESX 4.0, use this command:

    esxcfg-rescan -d vmhba#

    Note: The command to perform a rescan on the vmhba using various methods, such as RCLI, vMA, and PowerCLI varies. For appropriate commands, refer the documentation.

  • Removal of a LUN

    The removal of a LUN through an array-based administration action followed by a manual rescan can cause virtual machines on other LUNs to also become unresponsive.

Solution

The issue is resolved in ESXi/ESX 4.1 Update 1 and the fix has also been included with ESXi 5.0.

This issue is resolved in the patch release for ESX 4.0. For more information see, VMware ESX 4.0, Patch ESX400-200912401-BG: Updates vmkernel, vmklinux, tools, CIM, and perftools (1016291).

Notes:

Workaround

ESXi/ESX 4.x can list all of the LUNs it detects, as well as the state of these LUNs. If none of the paths to a storage device are in the ACTIVE state, then ESXi/ESX considers the device to be in an all-paths-down state. If an all-paths-down state does exist, then this is likely the issue causing LUNs to be unresponsive, either for a limited period of time or permanently, when a rescan occurs. For more information, see Identifying disks when working with VMware ESX (1014953).

If virtual machines are not responding on an ESXi/ESX 4.0 host, determine if an all-paths-down condition exists by running the command:
# esxcfg-mpath --list-paths --device device_naa | grep state

or

# esxcfg-mpath --list-paths --device device_mpx | grep state
Where:
  • device_naa is the Network Addressing Authority (NAA) unique address for the full storage device
  • device_mpx is the identifier if a NAA ID is not available
Note: For information about using the command line with ESXi, see Tech Support Mode for Emergency Support (1003677).

Starting with ESXi/ESX 4.0 Update 1, you can set an advanced configuration option on all hosts in the vCenter Server cluster to reduce rescan times and to prevent virtual machines from not responding. By default this option is disabled.

Caution: Not every all-paths-down condition is permanent. Some all-paths-down conditions, such as those that occur briefly during a network re-configuration, are transient. Enabling this option can cause devices in a transient all-paths-down state to become unavailable. VMware recommends disabling this option after the rescan operation completes.

To enable this option, run the command:
# esxcfg-advcfg -s 1 /VMFS3/FailVolumeOpenIfAPD
To disable and reset to the default value without requiring downtime, run the command:
# esxcfg-advcfg -s 0 /VMFS3/FailVolumeOpenIfAPD
To check the value of this option, run the command:
# esxcfg-advcfg -g /VMFS3/FailVolumeOpenIfAPD

Note: This does not apply for ESXi/ESX 4.0 Update 2 and 4.1 because the patch is integrated in these versions.

With ESX 4.1 Update 1 and ESX 4.0 Update 3, you no longer have to make the modification to the advanced setting. Virtual machines that are not associated with the APD Volume(s) do not become unresponsive upon a rescan.

Changing the location of snapshot delta files for virtual machines in ESXi 5.x

o configure the snapshot delta files to be kept in the same directory as the location specified in the workingDir parameter, add the new parameter snapshot.redoNotWithParent to the virtual machine configuration file(.vmx).

To set the snapshot.redoNotWithParent parameter:

  1. Power off the virtual machine and to commit any existing snapshots
  2. Right-click the virtual machine and click Edit Settings.
  3. Click the Options tab.
  4. In the Advanced Options, click General.
  5. Click Configuration Parameters.
  6. Click Add Row.
  7. In the Name field, type snapshot.redoNotWithParent.
  8. In the Value field, type true.
  9. Click OK to save the configuration parameters.
  10. Click OK to save the virtual machine settings.
  11. Power on the virtual machine. 

Important details to keep in mind when working with virtual machine snapshots and Storage vMotion in vSphere 5.x:
  • If you have a virtual machine running on snapshots and you move a virtual disk from one datastore to another, its associated snapshot files are moved with the virtual disk to the new datastore (even if they were initially created in the working directory). Newly created snapshots are created in the working directory again(as per the advanced setting).
  • If you move virtual machine configuration files to another datastore, the related snapshot files do not move. They remain on the original datastore.


Understanding virtual machine snapshots in VMware ESXi

What is a snapshot?

A snapshot preserves the state and data of a virtual machine at a specific point in time.
  • The state includes the virtual machine’s power state (for example, powered-on, powered-off, suspended).
  • The data includes all of the files that make up the virtual machine. This includes disks, memory, and other devices, such as virtual network interface cards.
A virtual machine provides several operations for creating and managing snapshots and snapshot chains. These operations let you create snapshots, revert to any snapshot in the chain, and remove snapshots. You can create extensive snapshot trees.

In VMware Infrastructure 3 and vSphere 4.x, the virtual machine snapshot delete operation combines the consolidation of the data and the deletion of the file. This caused issues when the snapshot files are removed from the Snapshot Manager, but the consolidation failed. This left the virtual machine still running on snapshots, and the user may not notice until the datastore is full with multiple snapshot files.

In vSphere 4.x, an alarm can be created to indicate if a virtual machine was running in snapshot mode.

In vSphere 5.0, enhancements have been made to the snapshot removal. In vSphere 5.0, you are informed via the UI if the consolidation part of a RemoveSnapshot or RemoveAllSnapshots operation has failed. A new option, Consolidate, is available via the Snapshot menu to restart the consolidation.

Creating a snapshot

When creating a snapshot, there are several options you can specify:
  • Name: This is used to identify the snapshot.
  • Description: This is used to describe the snapshot.
  • Memory: If the flag is 1 or true, a dump of the internal state of the virtual machine is included in the snapshot. Memory snapshots take longer to create, but allow reversion to a running virtual machine state as it was when the snapshot was taken. This option is selected by default. If this option is not selected, and quiescing is not selected, the snapshot will create files which are crash-consistent, which you can use to reboot the virtual machine.

    Note: When taking a memory snapshot, the entire state of the virtual machine will be stunned. 
  • Quiesce: If the flag is 1 or true, and the virtual machine is powered on when the snapshot is taken, VMware Tools is used to quiesce the file system in the virtual machine. Quiescing a file system is a process of bringing the on-disk data of a physical or virtual computer into a state suitable for backups. This process might include such operations as flushing dirty buffers from the operating system's in-memory cache to disk, or other higher-level application-specific tasks.

    Note: Quiescing indicates pausing or altering the state of running processes on a computer, particularly those that might modify information stored on disk during a backup, to guarantee a consistent and usable backup. Quiescing is not necessary for memory snapshots; it is used primarily for backups.
When a snapshot is created, it is comprised of these files:
  • -.vmdk and --delta.vmdk
    A collection of .vmdk and -delta.vmdk files for each virtual disk is connected to the virtual machine at the time of the snapshot. These files can be referred to as child disks, redo logs, or delta links. These child disks can later be considered parent disks for future child disks. From the original parent disk, each child constitutes a redo log pointing back from the present state of the virtual disk, one step at a time, to the original.

    Note:
    • The value may not be consistent across all child disks from the same snapshot. The file names are chosen based on filename availability.
    • If the virtual disk is larger than 2TB in size, the redo log file is of  --sesparse.vmdk format.

  • .vmsd
    The .vmsd file is a database of the virtual machine's snapshot information and the primary source of information for the Snapshot Manager. The file contains line entries which define the relationships between snapshots as well as the child disks for each snapshot.

  • Snapshot.vmsn
    The .vmsn file includes the current configuration and optionally the active state of the virtual machine. Capturing the memory state of the virtual machine lets you revert to a turned on virtual machine state. With nonmemory snapshots, you can only revert to a turned off virtual machine state. Memory snapshots take longer to create than nonmemory snapshots.
Notes:
  • The preceding files will be placed in the working directory by default in ESXi/ESX 3.x and 4.x. This behavior can be changed if desired.
  • In ESXi 5.x and later, snapshots descriptor and delta VMDK files will be stored in the same location as the virtual disks (which can be in a different directory to the working directory). To change this behavior, see Changing the location of snapshot delta files for virtual machines in ESXi 5.0 (2007563).

What products use the snapshot feature?

In addition to being able to use Snapshot Manager to create snapshots, snapshots are used by many VMware and third-party products and features. Some VMware products that use snapshots extensively are:
  • VMware Data Recovery
  • VMware Lab Manager
  • VMware vCenter and the VMware Infrastructure Client (Snapshot Manager, Storage vMotion)
Note: This is not an exhaustive list.

How do snapshots work?

Our VMware API allows VMware and third-party products to perform operations with virtual machines and their snapshots. This is a list of common operations that can be performed on virtual machines and snapshots using our API:
  • CreateSnapshot: Creates a new snapshot of a virtual machine. As a side effect, this updates the current snapshot.
  • RemoveSnapshot: Removes a snapshot and deletes any associated storage.
  • RemoveAllSnapshots: Remove all snapshots associated with a virtual machine. If a virtual machine does not have any snapshots, then this operation simply returns successfully.
  • RevertToSnapshot: Changes the execution state of a virtual machine to the state of this snapshot. This is equivalent to the Go To option under the Snapshot Manager while using vSphere/VI client GUI.
  • Consolidate: Merges the hierarchy of redo logs. This is available in vSphere 5.0 and later.
This is a high-level overview of how to create, remove, or revert snapshot requests that are processed within the VMware environment:
  1. A request to create, remove, or revert a snapshot for a virtual machine is sent from the client to the server using the VMware API.
  2. The request is forwarded to the VMware ESX host that is currently hosting the virtual machine that has issue.

    Note: This only occurs if the original request was sent to a different server, such as vCenter, which is managing the ESX host.

  3. If the snapshot includes the memory option, the ESX host writes the memory of the virtual machine to disk.

    Note: The virtual machine is stunned throughout the duration of time the memory is being written. The length of time of the stun cannot be pre-calculated, and is dependent on the performance of the disk that has issue and the amount of memory being written. ESXi/ESX 4.x and later have shorter stun times when memory is being written.

  4. If the snapshot includes the quiesce option, the ESX host requests the guest operating system to quiesce the disks via VMware Tools.

    Note: Depending on the guest operating system, the quiescing operation can be done by the sync driver, the vmsync module, or Microsoft's Volume Shadow Copy (VSS) service.

  5. The ESX host makes the appropriate changes to the virtual machine's snapshot database (.vmsd file) and the changes are reflected in the Snapshot Manager of the virtual machine.

    Note: When removing a snapshot, the snapshot entity in the Snapshot Manager is removed before the changes are made to the child disks. The Snapshot Manager does not contain any snapshot entries while the virtual machine continues to run from the child disk.

  6. The ESX host calls a function similar to the Virtual Disk API functions to make changes to the child disks (-delta.vmdk and .vmdk files) and the disk chain.

    Note: During a snapshot removal, if the child disks are large in size, the operation may take a long time. This can result in a timeout error message from either VirtualCenter or the VMware Infrastructure Client.

The child disk

The child disk, which is created with a snapshot, is a sparse disk. Sparse disks employ the copy-on-write (COW) mechanism, in which the virtual disk contains no data in places, until copied there by a write. This optimization saves storage space. The grain is the unit of measure in which the sparse disk uses the copy-on-write mechanism. Each grain is a block of sectors containing virtual disk data. The default size is 128 sectors or 64 KB.
 
Note: The sparse disk is usually created as a VMFSSPARSE type. Starting with vSphere 5.5, for any vmdk of 2TB or larger, the sparse disk is of type SESPARSE.

Child disks and disk usage

It is important to note these points regarding the space utilization of child disks:
  • If a virtual machine is running off of a snapshot, it is making changes to a child or sparse disk. The more write operations made to this disk, the larger it grows, to an upper limit of the size of the base disk plus a small amount of overhead.
  • The space requirements of the child disk are in addition to the parent disk on which it depends. If a virtual machine has a 10 GB disk with a child disk, the space used can be 10 GB + the child disk size + .vmsn file size + overhead.
  • Child disks are known to grow large enough to fill an entire datastore, but this is because the LUN containing the datastore was insufficiently large to contain the base disk, the number of snapshots created, and the overhead and .vmsn files created.
  • The speed at which child disks grow is directly dependent on the amount of I/O being done to the disk.
  • The size of the child disk has a direct impact on the length of time it takes to delete the snapshot associated to the child disk.
For more information on child disks and disk usage, see:

The disk chain

Generally, when you create a snapshot for the first time, the first child disk is created from the parent disk. Successive snapshots generate new child disks from the last child disk on the chain. The relationship can change if you have multiple branches in the snapshot chain.
This diagram is an example of a snapshot chain. Each square represents a block of data or a grain as described in the preceding section:
 
Caution: Manually manipulating the individual child disks or any of the snapshot configuration files may compromise the disk chain. VMware does not recommend manually modifying the disk chain as it may result in data loss.

Additional Information

  • To determine if a virtual machine is running on snapshots, see Determining if a virtual machine is using snapshots (1004343).

  • There are specific considerations when hosting a Microsoft Active Directory controller in a virtual environment. For a full list of considerations, see the Microsoft Knowledge Base article 888794.

    Note: The preceding link was valid as of August 1, 2012. If you find the link to be broken, provide feedback on the article and a VMware employee will update the article as necessary.
  • Time-sensitive applications may be impacted by reverting to a previous snapshot. Reverting the snapshot will revert the virtual machine to the point in time when the snapshot was created. This includes any operations conducted by the time-sensitive service or application in the guest operating system.

  • Reverting virtual machines to a snapshot causes all settings configured in the guest operating system since that snapshot to be reverted. The configuration which is reverted includes, but is not limited to, previous IP addresses, DNS names, UUIDs, guest OS patch versions, etc.
  • A snapshot operation should not be performed on a virtual machine which uses third-party iSCSI software initiators and is running in VMware Infrastructure 3. You can perform a snapshot operation in a vSphere environment, but it requires additional steps.
  • For earlier versions prior to VMware ESX 4.0 Update-2, the task of consolidating all snapshots (Remove All Snapshots task) caused unique changes stored only in the second snapshot delta disk to be copied upward through the snapshot chain and into the first snapshot, or its parent. This effect is recursive for each preceding parent file.

    For example: You have a base disk of size 8 GB and 2 levels of snapshots, each of 4 GB each. During a Remove All Snapshot Tasks, the first snapshot delta disk file can grow, worse-case scenario, to 8 GB, as all new blocks from the second snapshot are written. Any common changes stored in both snapshot levels do not require additional space.

  • From ESX4.0 Update 2 onwards, the snapshot mechanism has changed. VMware ESX now incorporates improved consolidation procedures which lessen the demand of free space. You are able to consolidate virtual machine delta disks even while minimal free space on your datastore is available.

Testing virtual machine storage I/O performance for ESX and ESXi

Symptoms

  • You see slow performance with a virtual machine on a SAN
  • Slow virtual machine performance
  • Completing a task takes a significant amount of time

Purpose

This article provides steps to test the speed/throughput on ESX/ESXi and virtual machines as well as steps to narrow down the issue.

Resolution

Doing speeds tests are always relative to the environment and hardware being used.
This article cannot provide specific test result numbers to expect as every environment is unique and the total load for an array is different.
You must establish a baseline throughput. You need to make an apples to apples comparison between an ESX host running a single virtual machine and a physical host. If the hardware between the two are identical and the operating system version between the physical host and the virtual machine are also the same, these two systems perform as close to each other as possible. If they do not perform similarly, other factors (BUSlogic vs. LSIlogic) need to be taken into consideration to explain the difference.
 
To establish a baseline:
  1. Create a virtual machine and put the operating system vmdk on local storage. This must be the same operating system as your physical host. If you choose a Windows operating system, ensure that you select LSIlogic as your controller type.
  2. Carve out a small LUN to attach to this virtual machine via RDM. A 5GB LUN is a sufficient size to work with, though the size really does not play a factor. If you have an RDM LUN that is already available, you can use it instead.
  3. Attach the LUN in RDM mode to the virtual machine. Whether you use physical, virtual, or none for BUS sharing does not matter.
  4. After you have booted the virtual machine, install a program similar to IOMeter ( http://www.iometer.org/) or HD_Speed( http://www.steelbytes.com/?mid=20) in the virtual machine.

    Note: The preceding links were correct as of October 13, 2016. If you find the link is broken, provide a feedback and the VMware employee will update the link.

    IOMeter is a more intuitive application and allows you to run various different kinds of tests however HD Speed is much easier to work with and provides accurate results that are simple to read and understand. Both applications are free to download and use.


    Note: VMware does not endorse or recommend any particular third party utility, nor is the list above meant to be exhaustive.

  5. Load up the application and select the RDM LUN to test. The following steps are using HD Speed for this test.
  6. Leave everything set to default and begin your read test.

    Caution: A write test is destructive to the data on the RDM LUN.

  7. Allow this test to run for at least 60 to 300 seconds. You have now established your baseline numbers.
  8. Power off the virtual machine.
  9. Remove the RDM LUN from the virtual machine.
  10. Present the LUN to your physical host.
  11. Repeat steps 4 through 7.
 
The results must be similar. If they are not, you need to review why.
Possible factors include:
  • Incorrect virtual machine test configuration (using BUSlogic instead of LSIlogic)
  • ESX host and physical host are connected to different switches
  • ESX host and physical host do not have identical hardware/firmware
  • Incorrect host mode settings on the SAN for the ESX host
If the virtual machine is not performing well, review /var/log/vmkernel for possible reasons for this degradation. If both the physical host and virtual machine are performing poorly but identically, review the array and switch configuration.
A possible reason for equally poor performance may be write cache on the array controllers or over commitment of the array in general.

Performing a disk check


Symptoms

  • Virtual machine disconnects from local hard disk.
  • You see the error:

    VMware Workstation unrecoverable error: (vcpu-0)
    Exception 0xc0000006 (disk error while paging) has occurred.
  • Power on errors reporting virtual machine cannot locate the virtual disk.
  • Virtual machine stops responding with a black screen.
  • In the vmware.log file, you see entries similar to:

    T

Purpose

This article describes how to perform a disk check. This is required to address problems encountered with an operating system as a result of file system errors. Problems can include data loss, virtual machine crashes, slow performance, virtual machine resume and suspend failures, and other unexpected behavior.

Resolution

Determine if there are problems with your file system by performing a disk check. A disk check can be done by using a third party application or by using tools native to your operating system.

Disk check methods differs between operating systems. Refer to the section below that matches your operating system.

Windows

Note: The exact procedure differs between versions of Windows. If one procedure below does not work try the other. If neither method works, consult the manual for your version of Windows.
 
To perform a disk check from the user interface:
  1. Double-click the My Computer icon (Windows XP) or Computer icon (Windows 7).
  2. Right-click the entry for your local disk.
  3. Click Properties.
  4. Click the Tools tab.
  5. Click Check Now.
  6. Select Scan for and attempt recovery of bad sectors.
  7. Click Start.
To perform a disk check from a command line:
  1. Click Start > Run, type cmd, and click OK. The Command Prompt window opens. For more information, see Opening a command or shell prompt (1003892).
  2. Type chkdsk c: /r and press Enter.

    Note: If the local disk being scanned is not c:, replace c: with its drive letter.
Note: A scan of the system drive requires that the operating system be rebooted.

Linux

Note: The exact procedure may differ between distributions of Linux. If these commands do not work, consult the manual for your distribution of Linux. These commands may also fail if you are not logged in as a user with root access.
  1. Open a shell prompt. For more information, see Opening a command or shell prompt (1003892).
  2. Type touch /forcefsck and press Enter.
  3. Type shutdown -r now and press Enter.

    Note: Issuing the shutdown command restarts your operating system.

Mac OS

To perform a disk check:
  1. Navigate to: [Macintosh HD]/Applications/Utilities.
  2. Open Disk Utility by double clicking it.
  3. Click the entry for the disk or volume to check.

    Note: This is usually the disk which Macintosh HD is indented below. It is named with the size of your hard drive, and the manufacturer's name.
  4. Click Verify Disk.

    Note: You can use this utility to verify the integrity of the startup volume, but not to repair any issues that may be discovered. If you need to repair the disk, use Safe Boot. For more information, see Using Safe Boot (1004017).

    Note: You can also click on Verify Disk Permissions to confirm that there are no problems being experienced due to incorrect permissions. If you find that there are permission problems, they can be corrected by clicking on Repair Disk Permissions

Booting a Windows guest operating system into Safe Mode

To start any version of a Windows operating system in Safe Mode:
Caution: Depending on your problem, performing this procedure may remove a software environment that is required to test the health of your operating system.
  1. Restart the operating system or power off and power on the computer.
  2. When the computer starts, press and hold F8.

    Notes:
    • Depending how your keyboard is configured, you may need to press fn+F8 rather than just F8.
    • You may see a series of messages that display information about hardware and memory. This is called POST information. If you see POST information, you do not need to press F8 until the screen goes black. Ensure the mouse focus is inside the virtual machine by clicking inside the console window and that you press F8 in a timely fashion. Windows can start very rapidly, and it may take several attempts to catch the Safe Mode screen. To delay the POST information screen, see Accessing the BIOS when the POST screen clears too quickly (1004129).
    You are presented with a text menu of start options.

    Note: If you do not see this text menu and Windows boots normally, repeat steps 1-2.

  3. Select a safe mode and press enter.

    1. If the operating system issues involve networking, select Safe Mode with Networking.
    2. If the operating system issues do not involve networking, select Safe Mode.
If the symptoms are resolved when using Safe Mode, then the source of the symptoms are related to a third-party application or nonessential Windows service, not Windows itself. You can try selectively disabling individual services and startup applications to narrow the cause of the problem

Choosing a network adapter for your virtual machine

Details

Network adapter choices depend on the version number and the guest operating system running on the virtual machine. This article discusses the different network adapter options available for virtual machines.

Solution

Available Network Adapters

Only those network adapters that are appropriate for the virtual machine you are creating are available configuration options in the Choose Networks window.
  • Vlance: This is an emulated version of the AMD 79C970 PCnet32- LANCE NIC, and it is an older 10 Mbps NIC with drivers available in most 32-bit guest operating systems except Windows Vista and later. A virtual machine configured with this network adapter can use its network immediately.

  • VMXNET: The VMXNET virtual network adapter has no physical counterpart. VMXNET is optimized for performance in a virtual machine. Because operating system vendors do not provide built-in drivers for this card, you must install VMware Tools to have a driver for the VMXNET network adapter available.

  • Flexible: The Flexible network adapter identifies itself as a Vlance adapter when a virtual machine boots, but initializes itself and functions as either a Vlance or a VMXNET adapter, depending on which driver initializes it. With VMware Tools installed, the VMXNET driver changes the Vlance adapter to the higher performance VMXNET adapter.

  • E1000: An emulated version of the Intel 82545EM Gigabit Ethernet NIC. A driver for this NIC is not included with all guest operating systems. Typically Linux versions 2.4.19 and later, Windows XP Professional x64 Edition and later, and Windows Server 2003 (32-bit) and later include the E1000 driver.

    Note: E1000 does not support jumbo frames prior to ESXi/ESX 4.1.

  • E1000e: This feature emulates a newer model of Intel Gigabit NIC (number 82574) in the virtual hardware. This is known as the "e1000e" vNIC. e1000e is available only on hardware version 8 (and newer) virtual machines in vSphere 5. It is the default vNIC for Windows 8 and newer (Windows) guest operating systems. For Linux guests, e1000e is not available from the UI (e1000, flexible vmxnet, enhanced vmxnet, and vmxnet3 are available for Linux).

  • VMXNET 2 (Enhanced): The VMXNET 2 adapter is based on the VMXNET adapter but provides some high-performance features commonly used on modern networks, such as jumbo frames and hardware offloads. This virtual network adapter is available only for some guest operating systems on ESXi/ESX 3.5 and later. Because operating system vendors do not provide built-in drivers for this card, you must install VMware Tools to have a driver for the VMXNET 2 network adapter available.

    VMXNET 2 is supported only for a limited set of guest operating systems.

    To determine if the the VMXNET 2 (Enhanced) adapter is supported for your guest operating system and vSphere ESXi version, see the VMware Compatibility Guide.

    Notes
    :
    • You can use enhanced VMXNET adapters with other versions of the Microsoft Windows 2003 operating system, but a workaround is required to enable the option in the VMware Infrastructure (VI) Client or vSphere Client. If Enhanced VMXNET is not offered as an option, see Enabling enhanced vmxnet adapters for Microsoft Windows Server 2003 (1007195).
    • Jumbo frames are not supported in the Solaris Guest OS for VMXNET 2.
  • VMXNET 3: The VMXNET 3 adapter is the next generation of a paravirtualized NIC designed for performance, and is not related to VMXNET or VMXNET 2. It offers all the features available in VMXNET 2, and adds several new features like multiqueue support (also known as Receive Side Scaling in Windows), IPv6 offloads, and MSI/MSI-X interrupt delivery. For information about the performance of VMXNET 3, see Performance Evaluation of VMXNET3 Virtual Network Device. Because operating system vendors do not provide built-in drivers for this card, you must install VMware Tools or open-vm-tools to have a driver for the VMXNET 3 network adapter available.  For more information about open-vm-tools, see Choosing a network adapter for your virtual machine (1001805).

    VMXNET 3 is supported only for virtual machines version 7 and later, with a limited set of guest operating systems.

    To determine if the the VMXNET3 adapter is supported for your guest operating system and vSphere ESXi version, see the VMware Compatibility Guide.

    Notes:
    • In ESXi/ESX 4.1 and earlier releases, jumbo frames are not supported in the Solaris Guest OS for VMXNET 2 and VMXNET 3. The feature is supported starting with ESXi 5.0 for VMXNET 3 only. For more information, see Enabling Jumbo Frames on the Solaris guest operating system (2012445).
    • Fault Tolerance is not supported on a virtual machine configured with a VMXNET 3 vNIC in vSphere 4.0, but is fully supported on vSphere 4.1.
    • Windows Server 2012 is supported with e1000, e1000e, and VMXNET 3 on ESXi 5.0 Update 1 or higher.

Adapter Caveats

  • Migrating virtual machines that use enhanced VMXNET

    VMXNET 2 was introduced with ESX 3.5. Virtual machines configured to have VMXNET 2 adapters cannot migrate to earlier ESX hosts, even though virtual machines can usually migrate freely between ESX 3.0 and ESX 3.0.x.

    If you must migrate a virtual machine between later and earlier hosts, do not choose VMXNET 2.

  • Upgrading from ESX 2.x to ESX 3.x

    When a virtual hardware upgrade operation transforms a virtual machine created on an ESX 2.x host to an ESX 3.x host, Vlance adapters are automatically upgraded to Flexible. In contrast, VMXNET adapters are not upgraded automatically because most Linux guest operating system versions do not reliably preserve network settings when a network adapter is replaced. Since the guest operating system thinks a Flexible adapter is still Vlance, it retains the settings in that case. If the upgrade replace a VMXNET adapter with a Flexible adapter, the guest operating system erroneously discards the settings.

    After the virtual hardware upgrade, the network adapter is still VMXNET, without the fall back compatibility of the Flexible adapter. Just as on the original earlier host, if VMware Tools is uninstalled on the virtual machine, it cannot access its network adapters.

  • Adding virtual disks

    Adding an existing earlier (ESX 2.x) virtual disk to an ESX 3.x virtual machine results in a de facto downgrade of that virtual machine to ESX 2.x. If you are using ESX 3.x features, such as enhanced VMXNET or Flexible network adapters, the virtual machine becomes inconsistent. When you add an existing ESX 2.x virtual disk to an ESX 3.x machine, immediately use the Upgrade Virtual Hardware command to restore the virtual machine to the ESX 3 version. This problem does not arise when you add earlier virtual disks to an ESXi/ESX 4.0 virtual machine.

    Note: Executing the Upgrade Virtual Hardware command changes the ESX 2 virtual disk so that it is no longer usable on an ESX 2 virtual machine. Consider making a copy of the disk before you upgrade one of the two copies to ESX 3 format.

Installing VMware Tools in an Ubuntu virtual machine

To install VMware Tools, you must mount the VMware Tools CD image, extract the contents (VMware Tools), and then run the installer.

Note: If VMware Tools is already installed, use these steps to uninstall and then reinstall VMware Tools. When there is product updates, the VMware Tools package is also updated, so an update of the installed version of VMware Tools is required.

Ubuntu or Ubuntu Server with a graphical user interface

To mount the CD image and extract the contents:
  1. Power on the virtual machine.
  2. Log in to the virtual machine using an account with administrator or root privileges.
  3. Select:
    For Fusion: Virtual Machine > Install VMware Tools.
    For Workstation: VM > Install VMware Tools.
    For Player: Player > Manage > Install VMware Tools.

    Note: If you are running the light version of Fusion, a version of Workstation without VMware Tools, or VMware Player, you are prompted to download VMware Tools before they can be installed. Click Download Now to begin the download.

  4. Open the VMware Tools CD mounted on the Ubuntu desktop.
  5. Right-click the file name that is similar to VMwareTools.x.x.x-xxxx.tar.gz, click Extract to, and select Ubuntu Desktop to save the extracted contents.

    The vmware-tools-distrib folder is extracted to the Ubuntu Desktop.
To install VMware Tools in Ubuntu:
  1. Open a Terminal window. For more information, see Opening a command or shell prompt (1003892).
  2. In the Terminal, run this command to navigate to the vmware-tools-distrib folder:

    cd Desktop/vmware-tools-distrib

  3. Run this command to install VMware Tools:

    sudo ./vmware-install.pl -d

    Note: The -d switch assumes that you want to accept the defaults. If you do not use -d, press Return to accept the defaults or supply your own answers.

  4. Enter your Ubuntu password.
  5. Restart the Ubuntu virtual machine after the VMware Tools installation completes.

Ubuntu Server with only a command line interface

  1. Go to Virtual Machine > Install VMware Tools (or VM > Install VMware Tools).

    Note: If you are running the light version of Fusion, or a version of Workstation without VMware Tools, or VMware Player, you are prompted to download the Tools before they can be installed. Click Download Now to begin the download.

  2. In the Ubuntu guest, run these commands:

    1. Run this command to create a directory to mount the CD-ROM:

      sudo mkdir /mnt/cdrom

      When prompted for a password, enter your Ubuntu admin user password.

      Note: For security reasons, the typed password is not displayed. You do not need to enter your password again for the next five minutes.

    2. Run this command to mount the CD-ROM:

      sudo mount /dev/cdrom /mnt/cdrom or sudo mount /dev/sr0 /mnt/cdrom

    3. The file name of the VMware Tools bundle varies depending on your version of the VMware product. Run this command to find the exact name:

      ls /mnt/cdrom

    4. Run this command to extract the contents of the VMware Tools bundle:

      tar xzvf /mnt/cdrom/VMwareTools-x.x.x-xxxx.tar.gz -C /tmp/

      Note: x.x.x-xxxx is the version discovered in the previous step.

    5. Run this command to change directories into the VMware Tools distribution:

      cd /tmp/vmware-tools-distrib/

    6. Run this command to install VMware Tools:

      sudo ./vmware-install.pl -d

      Note: The -d switch assumes that you want to accept the defaults. If you do not use -d, press Return to accept each default or supply your own answers.
  3. Run this command to reboot the virtual machine after the installation completes:

    sudo reboot

Restarting the Management agents in ESXi


ymptoms

  • Cannot connect directly to the ESXi host or manage under vCenter server.
  • vCenter Server displays the error:

    Virtual machine creation may fail because agent is unable to retrieve VM creation options from the host

Purpose

For troubleshooting ESXi connectivity issue, restart the management agents on your ESXi host.
Warning: If LACP is configured on the VSAN network, do not restart management agents on ESXi hosts running Virtual SAN.
  • Restarting the management agents may impact any tasks that are running on the ESXi host at the time of the restart.
  • Check for any storage issues before restarting the Host deamon hostd service or services.sh

Resolution

Restart Management agents in ESXi Using Direct Console User Interface (DCUI):
  1. Connect to the console of your ESXi host.
  2. Press F2 to customize the system.
  3. Log in as root.
  4. Use the Up/Down arrows to navigate to Troubleshooting Options > Restart Management Agents.
  5. Press Enter.
  6. Press F11 to restart the services.
  7. When the service restarts, press Enter.
  8. Press Esc to log out.

Restart Management agents in ESXi Using ESXi Shell or Secure Shell (SSH):

  1. Log in to ESXi Shell or SSH as root.

    For Enabling ESXi Shell or SSH, see Using ESXi Shell in ESXi 5.x and 6.x (2004746).

  2. Restart the ESXi host daemon and vCenter Agent services using these commands:

    /etc/init.d/hostd restart

    /etc/init.d/vpxa restart
Caution:
  • If LACP is enabled and configured, do not restart management services using services.sh command. Instead restart independent services using the /etc/init.d/module restart command.
  • If the issue is not resolved, and you are restarting all the services that are a part of the services.sh script, take a downtime before proceeding to the script.

Content of this blog has been moved to GITHUB

Looking at current trends and to make my content more reachable to people, I am moving all the content of my blog https://tech-jockey.blogsp...