Binary Nature where the analog and digital bits of nature connect

  • Subscribe to our RSS feed.
  • Twitter
  • StumbleUpon
  • Reddit
  • Facebook
  • Digg

Tuesday, 13 August 2013

PowerShell Function for Windows System Memory Statistics

Posted on 00:27 by Unknown
Memory is one of the four primary hardware resources an operating system manages. The other three are cpu, disk, and network. Analysis of system memory statistics is a common task for any Windows system administrator as odd behavior can occur when left unchecked. A few built-in tools (Task Manager, Resource Monitor, and Performance Monitor) may be sufficient depending on what memory metrics you need to track, but many sysadmins eventually turn to the tool often referred to as “Task Manager on steroids”.

Process Explorer provides transparency into the Windows operating system from the scope of a single process to the system as a whole. I often use the System Information feature when I need a broad, but also detailed, overview of the “big four” components.
System Information
Process Explorer is an excellent solution when used interactively on the local computer, but what if you prefer to gather system memory statistics for hundreds of remote computers all from the comfort of your management workstation? Think large-scale fan-out remoting.

Design and Implementation
The following tables display the system memory attributes the custom PowerShell object type will be comprised of. As you well may notice, they’re essentially the same properties from the Commit Charge, Physical Memory, and Kernel Memory sections of Process Explorer’s System Information Memory tab.

Commit Charge
AttributeDefinition
CurrentAmount of committed virtual memory. Committed memory is physical memory for which space has been reserved on the disk paging file in case it must be written back to disk.
LimitAmount of virtual memory that can be committed without having to extend the paging file(s). If the paging file(s) are be expanded, the CommitLimit value increases accordingly.
PeakThe maximum amount of memory in the committed state since the last system reboot.

Physical Memory
AttributeDefinition
TotalThe amount of actual physical memory.
AvailableThe amount of physical memory currently available. This is the amount of physical memory that can be immediately reused without having to write its contents to disk first.
Cache Working SetShows the sum of the values of System Cache Resident Bytes, System Driver Resident Bytes, System Code Resident Bytes, and Pool Paged Resident Bytes.
Kernel Working SetShows the size of operating system code currently in physical memory that can be written to disk when not in use.
Driver Working SetShows the size of pageable physical memory being used by device drivers. It is the working set (physical memory area) of the drivers.

Kernel Memory
AttributeDefinition
Paged Working SetShows the current size of the paged pool. Space used by the paged and nonpaged pools is taken from physical memory, so a pool that is too large denies memory space to processes.
Paged VirtualThe memory currently in the paged kernel pool.
NonpagedThe memory currently in the nonpaged kernel pool.


I've also incorporated some inline C# code within the PowerShell function. When I was designing the function, I figured I would be able to retrieve all necessary data from WMI for the properties of the object, and that is true for all except one.

The Peak Commit Charge property value can only be programmatically accessed from the Windows API (PERFORMANCE_INFORMATION structure) with the GetPerformanceInfo PSAPI function. PowerShell (and more specifically C#) can’t do this directly. Platform Invocation Services (P/Invoke) is required. Luckily, Antonio Bakula created a C# wrapper for the GetPerformanceInfo function, so our PowerShell function can leverage his code.

The original design had the PowerShell function output a single custom PS object type that would encase all properties, but there is a slight performance penalty with instantiating the Win32_PerfRawData_PerfOS_Memory type. The operation needs to load performance monitor counters which translates into longer runtime of the command. You can test this yourself by running the following command:

PS> Measure-Command { gwmi Win32_PerfRawData_PerfOS_Memory } | fl sec*,mill*

On my Windows 2008 R2 Server test system, it takes an average of five seconds to complete the operation, but it only takes an average of two seconds for my Windows 2012 Server test system. This happens to be a minor nuisance for a single computer, but those seconds start to add up when accessing dozens of remote computers for a single batch job. It should also be noted this only happens with the initial launch of the command. The process is cached (and/or counters stay loaded) for a little while (~10 minutes), so each successive execution of the command is nearly instantaneous within that timespan.

My solution is to have the function output two separate custom PS object types. One is “light” and the other is “heavy”. The default output is the "light" object. If you run the function with the -Detailed option, it will output the "heavy" object with all properties. The primary differences being the "light" object has a subset of the "heavy" object properties, but it only references data from the object created with the inline C# code. It is the preferred object to use if all you need are core memory statistics and execution speed is essential. If you require "everything but the kitchen sink" memory statistics, then the "heavy" object is the way to go.

PS> (Get-Memory).pstypenames[0]
BinaryNature.Memory
PS> (Get-Memory -Detailed).pstypenames[0]
BinaryNature.MemoryDetailed

The Usage section will go over examples of both.

001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207
function Get-Memory{ [CmdletBinding()] param ( [switch]$Detailed, [switch]$Format ) $signature = @' /* * Item: Windows PSAPI GetPerformanceInfo C# Wrapper * Source: http://www.antoniob.com/windows-psapi-getperformanceinfo-csharp-wrapper.html * Author: Antonio Bakula */ using System; using System.Runtime.InteropServices; public struct PerfomanceInfoData { public Int64 CommitTotalPages; public Int64 CommitLimitPages; public Int64 CommitPeakPages; public Int64 PhysicalTotalBytes; public Int64 PhysicalAvailableBytes; public Int64 SystemCacheBytes; public Int64 KernelTotalBytes; public Int64 KernelPagedBytes; public Int64 KernelNonPagedBytes; public Int64 PageSizeBytes; public int HandlesCount; public int ProcessCount; public int ThreadCount; } public static class PsApiWrapper { [DllImport("psapi.dll", SetLastError = true)] [return: MarshalAs(UnmanagedType.Bool)] private static extern bool GetPerformanceInfo([Out] out PsApiPerformanceInformation PerformanceInformation, [In] int Size); [StructLayout(LayoutKind.Sequential)] public struct PsApiPerformanceInformation { public int Size; public IntPtr CommitTotal; public IntPtr CommitLimit; public IntPtr CommitPeak; public IntPtr PhysicalTotal; public IntPtr PhysicalAvailable; public IntPtr SystemCache; public IntPtr KernelTotal; public IntPtr KernelPaged; public IntPtr KernelNonPaged; public IntPtr PageSize; public int HandlesCount; public int ProcessCount; public int ThreadCount; } public static PerfomanceInfoData GetPerformanceInfo() { PerfomanceInfoData data = new PerfomanceInfoData(); PsApiPerformanceInformation perfInfo = new PsApiPerformanceInformation(); if (GetPerformanceInfo(out perfInfo, Marshal.SizeOf(perfInfo))) { Int64 pageSize = perfInfo.PageSize.ToInt64(); // data in bytes data.CommitTotalPages = perfInfo.CommitTotal.ToInt64() * pageSize; data.CommitLimitPages = perfInfo.CommitLimit.ToInt64() * pageSize; data.CommitPeakPages = perfInfo.CommitPeak.ToInt64() * pageSize; data.PhysicalTotalBytes = perfInfo.PhysicalTotal.ToInt64() * pageSize; data.PhysicalAvailableBytes = perfInfo.PhysicalAvailable.ToInt64() * pageSize; data.KernelPagedBytes = perfInfo.KernelPaged.ToInt64() * pageSize; data.KernelNonPagedBytes = perfInfo.KernelNonPaged.ToInt64() * pageSize; } return data; } } '@ function Format-HumanReadable { param ($size) switch ($size) { {$_ -ge 1PB}{"{0:#.#'P'}" -f ($size / 1PB); break} {$_ -ge 1TB}{"{0:#.#'T'}" -f ($size / 1TB); break} {$_ -ge 1GB}{"{0:#.#'G'}" -f ($size / 1GB); break} {$_ -ge 1MB}{"{0:#.#'M'}" -f ($size / 1MB); break} {$_ -ge 1KB}{"{0:#'K'}" -f ($size / 1KB); break} default {"{0}" -f ($size) + "B"} } } # Create PerformanceInfoData object Add-Type -TypeDefinition $signature [PerfomanceInfoData]$w32perf = [PsApiWrapper]::GetPerformanceInfo() if ($Detailed) { try { # Create Win32_PerfRawData_PerfOS_Memory object $query = 'SELECT * FROM Win32_PerfRawData_PerfOS_Memory' $wmimem = Get-WmiObject -Query $query -ErrorAction Stop # Create "detailed" PS memory object # Value in bytes for memory attributes $memd = New-Object -TypeName PSObject -Property @{ TotalPhysicalMem=$w32perf.PhysicalTotalBytes AvailPhysicalMem=$w32perf.PhysicalAvailableBytes CacheWorkingSet=[long]$wmimem.CacheBytes KernelWorkingSet=[long]$wmimem.SystemCodeResidentBytes DriverWorkingSet=[long]$wmimem.SystemDriverResidentBytes CommitCurrent=$w32perf.CommitTotalPages CommitLimit=$w32perf.CommitLimitPages CommitPeak=$w32perf.CommitPeakPages PagedWorkingSet=[long]$wmimem.PoolPagedResidentBytes PagedVirtual=$w32perf.KernelPagedBytes Nonpaged=$w32perf.KernelNonPagedBytes Computer=$env:COMPUTERNAME } } catch { $msg = ("Error: {0}" -f $_.Exception.Message) Write-Warning $msg } if ($Format) { # Format output in human-readable form # End of PS pipeline/format right rule option Write-Output '-------------' Write-Output 'Commit Charge' Write-Output '-------------' "Current : $(Format-HumanReadable $memd.CommitCurrent)" "Limit`t : $(Format-HumanReadable $memd.CommitLimit)" "Peak`t : $(Format-HumanReadable $memd.CommitPeak)" "Peak/Limit : $("{0:P2}" ` -f ($memd.CommitPeak / $memd.CommitLimit))" "Curr/Limit : $("{0:P2}" ` -f ($memd.CommitCurrent / $memd.CommitLimit))" [Environment]::NewLine Write-Output '---------------' Write-Output 'Physical Memory' Write-Output '---------------' "Total`t : $(Format-HumanReadable $memd.TotalPhysicalMem)" "Available : $(Format-HumanReadable $memd.AvailPhysicalMem)" "CacheWS`t : $(Format-HumanReadable $memd.CacheWorkingSet)" "KernelWS : $(Format-HumanReadable $memd.KernelWorkingSet)" "DriverWS : $(Format-HumanReadable $memd.DriverWorkingSet)" [Environment]::NewLine Write-Output '-------------' Write-Output 'Kernel Memory' Write-Output '-------------' "PagedWS : $(Format-HumanReadable $memd.PagedWorkingSet)" "PagedVirt : $(Format-HumanReadable $memd.PagedVirtual)" "Nonpaged : $(Format-HumanReadable $memd.Nonpaged)" [Environment]::NewLine } else { $memd.PSObject.TypeNames.Insert(0,'BinaryNature.MemoryDetailed') Write-Output $memd } } else { # Create custom "core" memory object # Value in bytes for memory attributes $memb = New-Object -TypeName PSObject -Property @{ TotalPhysicalMem=$w32perf.PhysicalTotalBytes AvailPhysicalMem=$w32perf.PhysicalAvailableBytes Computer=$env:COMPUTERNAME } if ($Format) { $memb | Select-Object @{n="Name"; e={$_.Computer}}, @{n="Total"; e={Format-HumanReadable $_.TotalPhysicalMem}}, @{n="InUse"; e={Format-HumanReadable ($_.TotalPhysicalMem - ` $_.AvailPhysicalMem)}}, @{n="Avail"; e={Format-HumanReadable $_.AvailPhysicalMem}}, @{n="Use%"; e={[int]((($_.TotalPhysicalMem - ` $_.AvailPhysicalMem) / $_.TotalPhysicalMem) * 100)}} } else { $memb.PSObject.TypeNames.Insert(0,'BinaryNature.Memory') Write-Output $memb } } }

Test Environment
The following tasks provide an example of how to configure a test environment for utilization of the Get-Memory function. I included this section for those individuals that want a "quick hack" method of getting things staged for following along in the Usage section. It is required that all computers be members of an Active Directory domain for this scenario. In my test environment, all commands will be run from a management workstation except for the Configure PowerShell Remoting task which will need to be run on each computer < Windows 2012.

# Create the PowerShell script module
We will create a basic PowerShell script module by running the following commands from an elevated PowerShell prompt. In my example, the module will be named BNTools. The notepad $bnmod command will open the notepad application (to edit the BNTools.psm1 file) where you will then copy/paste/save the Get-Memory function from the above code block.

PS> New-Item -Path $pshome\Modules\BNTools\BNTools.psm1 -Type file -Force -OutVariable bnmod
PS> notepad $bnmod

# Copy the PowerShell script module to remote computers
The PowerShell script module will need to be stored on each of the remote computers we connect to. In my example, I will save it in the default system location ($PSHOME\Modules). Verify the File and Printer Sharing (SMB-In) firewall rule, for the Inbound Rules set, is enabled on each computer. I've also created a $servers variable that contains the computer names for my test environment. Run the following commands from an elevated PowerShell prompt:

PS> $servers = 'dc01','db01','exch01','sp01'
PS> $servers | % { Copy-Item -Recurse -Force -Verbose -Path $pshome\Modules\BNTools -Destination \\$_\c$\Windows\System32\WindowsPowerShell\v1.0\Modules }

# Configure PowerShell Remoting
Remember that Windows PowerShell Remoting is enabled by default for Windows Server 2012 and later (within a trusted or domain network), so you can skip this task for those members. I'm working with Windows Server 2008 R2 VMs in my test environment, so I enable PowerShell Remoting by running the following command from an elevated PowerShell prompt on each computer:

PS> Enable-PSRemoting -Force

# PowerShell Execution Policy
Now that we have PowerShell Remoting enabled, we can verify/set the execution policy for PowerShell. My example will use RemoteSigned for the value to set on each of the computers. The following commands can be run from a normal (non-elevated) PowerShell prompt, but the credential object will need to contain the credentials of an AD administrator user account.

PS> $cred = Get-Credential 'example\administrator'
PS> Invoke-Command -Credential $cred -ComputerName $servers -ScriptBlock { if ((Get-ExecutionPolicy) -ne 'RemoteSigned') { Set-ExecutionPolicy RemoteSigned -Force } }
PS> Invoke-Command -Credential $cred -ComputerName $servers -ScriptBlock { Get-ExecutionPolicy } | ft pscomp*,value -auto

PSComputerName Value
-------------- -----
exch01 RemoteSigned
dc01 RemoteSigned
db01 RemoteSigned
sp01 RemoteSigned


Usage
The Get-Memory PowerShell function emits PowerShell objects, so we have a multitude of choices for processing and output. In addition to the "raw" output of the objects, each also has a -Format option. The option formats the numbers into human-readable form. For example, 1048576 bytes becomes 1 MB. My implementation actually omits the space and B (ex. 1M).

# Default Output

PS> Import-Module BNTools
PS> Get-Memory | fl

AvailPhysicalMem : 294473728
Computer : MGMT
TotalPhysicalMem : 1073205248

PS> Get-Memory -Detailed

KernelWorkingSet : 3588096
AvailPhysicalMem : 293928960
Nonpaged : 34111488
PagedVirtual : 113336320
CacheWorkingSet : 37961728
Computer : MGMT
DriverWorkingSet : 8429568
CommitPeak : 1307279360
CommitCurrent : 955514880
TotalPhysicalMem : 1073205248
CommitLimit : 2146947072
PagedWorkingSet : 102797312


# Output with Format option
The -Format option comes with a caveat. The human-readable value is the result of a number to string data type conversion, so keep this limitation in mind when further processing (sorting, comparison, etc.) is attempted on the value.

PS> Get-Memory | gm -MemberType prop*

TypeName: BinaryNature.Memory

Name MemberType Definition
---- ---------- ----------
AvailPhysicalMem NoteProperty System.Int64 AvailPhysicalMem
Computer NoteProperty System.String Computer
TotalPhysicalMem NoteProperty System.Int64 TotalPhysicalMem

PS> Get-Memory -Format | gm -MemberType prop*

TypeName: Selected.System.Management.Automation.PSCustomObject

Name MemberType Definition
---- ---------- ----------
Avail NoteProperty System.String Avail
InUse NoteProperty System.String InUse
Name NoteProperty System.String Name
Total NoteProperty System.String Total
Use% NoteProperty System.Int32 Use%


Unless I need to perform further operations on a property of the memory object, I usually enable the -Format option for easier readability of the output. If you're a true geek and love to do mental calculations of bytes to $x, then feel free to stick with the default output. :-)

PS> Get-Memory -Format

Name : DC01
Total : 2G
InUse : 906.1M
Avail : 1.1G
Use% : 44


The following example uses PowerShell Remoting to connect to a remote server, retrieve the detailed system memory statistics, and have the information returned and displayed in formatted output:

PS> $cred = Get-Credential 'example\administrator'
PS> Invoke-Command -Credential $cred -ComputerName exch01 -ScriptBlock { ipmo bntools; Get-Memory -Detailed -Format }
-------------
Commit Charge
-------------
Current : 3.8G
Limit : 6G
Peak : 3.9G
Peak/Limit : 64.90 %
Curr/Limit : 62.72 %


---------------
Physical Memory
---------------
Total : 3G
Available : 805.6M
CacheWS : 59.6M
KernelWS : 32K
DriverWS : 6M


-------------
Kernel Memory
-------------
PagedWS : 94.3M
PagedVirt : 94.8M
Nonpaged : 37.9M


# Fan-Out Remoting (One-to-Many)
The previous examples have demonstrated interactive or one-to-one operations, but what if we need to scale out to multiple computers? This example will connect to multiple servers, return specified data, sort on usage, and display the result in a formatted table.

PS> $cred = Get-Credential 'example\administrator'
PS> $servers = 'dc01','db01','exch01','sp01'
PS> Invoke-Command -Credential $cred -ComputerName $servers -HideComputerName -ScriptBlock { ipmo bntools; Get-Memory -Format } | select * -excl run* | sort 'Use%' -Descending | ft -auto

Name Total InUse Avail Use%
---- ----- ----- ----- ----
EXCH01 3G 2.1G 911.2M 70
SP01 2G 1G 983.8M 52
DC01 2G 923M 1.1G 45
DB01 2G 772.8M 1.2G 38


What if we need a list of servers with a Kernel Working Set value greater than 1 MB?

PS> $cred = Get-Credential 'example\administrator'
PS> $servers = 'dc01','db01','exch01','sp01'
PS> Invoke-Command -Credential $cred -ComputerName $servers -ScriptBlock { ipmo bntools; Get-Memory -Detailed } | ? { $_.KernelWorkingSet -gt 1mb } | select comp*

Computer
--------
SP01
DC01
DB01


# Out-GridView
Pipe to an interactive table instead of the console?

PS> $cred = Get-Credential 'example\administrator'
PS> $servers = 'dc01','db01','exch01','sp01'
PS> Invoke-Command -Credential $cred -ComputerName $servers -HideComputerName -ScriptBlock { ipmo bntools; Get-Memory -Format } | select * -excl run* | sort 'Use%' -Descending | Out-GridView -Title 'Server Memory Stats'


# Output to CSV
An interactive table is nice, but sometimes we just need to output to the tried-and-true comma-separated values (CSV) file type.

PS> $cred = Get-Credential 'example\administrator'
PS> $servers = 'dc01','db01','exch01','sp01'
PS> Invoke-Command -Credential $cred -ComputerName $servers -HideComputerName -ScriptBlock { ipmo bntools; Get-Memory -Format } | select * -excl run*,ps* | sort Name | Export-Csv -Path $HOME\Documents\server_memory_stats.csv -NoTypeInformation


# Other Cool Output

Get-Memory + Get-DiskFree + Get-OSInfo + XHTML =

Read More
Posted in PowerShell, Windows | No comments
Newer Posts Older Posts Home
Subscribe to: Comments (Atom)

Popular Posts

  • Cisco ASA SSL VPN with Active Directory
    There is little doubt the bring-your-own-device (BYOD) strategy is becoming a popular method to access company resources. As technical prof...
  • PowerShell Function for Windows System Memory Statistics
    Memory is one of the four primary hardware resources an operating system manages. The other three are cpu, disk, and network. Analysis of sy...
  • Integrate VMware Fusion with GNS3 on your Mac
    At long last, we can finally integrate VMware Fusion with GNS3. VMware Workstation for Windows and Linux has had this capability for quite s...
  • Configure Inter-VLAN routing on a Cisco L3 Catalyst Switch
    I recently had to configure inter-VLAN routing at a client's site. I don't have to perform this task on a regular basis, so I figur...
  • SSL VPN configuration on Cisco ASA with AnyConnect VPN client
    This post will describe how to setup a Cisco Adaptive Security Appliance (ASA) device to perform remote access SSL VPN with the stand-alone ...
  • Enable sudo for RHEL and CentOS
    Sudo is an arguably safer alternative to logging in (or using the su command) to the root account. Sudo allows you to partition and delegat...
  • Get Exchange Server Version and Update Info with PowerShell
    I prefer not to "reinvent the wheel", so I spent quite a bit of time searching the web for available code that would perform the t...
  • Cisco Security Device Manager on the Mac
    Cisco Router and Security Device Manager (SDM) is a Web-based device-management tool that enables you to deploy and manage the services on a...
  • Install Request Tracker 4 on Ubuntu Server
    The CentOS6/RT4 blog post has generated terrific feedback, so I figure an Ubuntu (and Debian) distribution port is essential. The core com...
  • Install Request Tracker 4
    The argument could be made Request Tracker is the de facto standard when it comes to issue tracking systems. Maybe the only drawback of RT ...

Categories

  • AD
  • Apache
  • AWS
  • Cisco
  • Exchange
  • FFmpeg
  • GNS3
  • Linux
  • Mac
  • MariaDB
  • MySQL
  • PowerShell
  • RT
  • Security
  • SSH
  • VMware
  • Windows
  • Zenoss

Blog Archive

  • ▼  2013 (8)
    • ►  October (1)
    • ►  September (1)
    • ▼  August (1)
      • PowerShell Function for Windows System Memory Stat...
    • ►  May (1)
    • ►  April (1)
    • ►  March (1)
    • ►  February (1)
    • ►  January (1)
  • ►  2012 (3)
    • ►  December (1)
    • ►  November (1)
    • ►  April (1)
  • ►  2011 (3)
    • ►  June (1)
    • ►  May (2)
  • ►  2010 (8)
    • ►  August (1)
    • ►  July (1)
    • ►  June (1)
    • ►  May (1)
    • ►  April (1)
    • ►  March (1)
    • ►  February (1)
    • ►  January (1)
  • ►  2009 (3)
    • ►  December (1)
    • ►  November (1)
    • ►  October (1)
Powered by Blogger.

About Me

Unknown
View my complete profile