AIX High Page Faults

Heavy Virtual Memory Page Faults occur on an AIX 5.3 64bit system.

Listing 1.1

┌─topas_nmon──b=Black&White──────Host=aixprd02───────Refresh=2 secs───16:16.11────
│ Hints-Mode-(verbose) ───────────────────────────────────────────────────────────
│  Code    Resource            Stats   Now       Warn    Danger
│      OK -> CPU               %busy  25.9%      >80%    >90%
│      OK -> Paging size       %free  99.7%      <20%    <10%
│      OK -> Paging Space      RAM:pg100.0%      <50%    <10%
│  DANGER -> Page Faults       faults 11402.0    >16/s   >160/s
│ CPU-Utilisation-Small-View ───────────EntitledCPU=  3.00 UsedCPU=  1.656────────
│Logical  CPUs              0----------25-----------50----------75----------100
│CPU User%  Sys% Wait% Idle%|           |            |           |            |
│  0  94.9   4.5   0.1   0.5|UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUss>
│  1  98.1   1.6   0.0   0.3|UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU>
│  2  91.1   8.5   0.1   0.3|UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUssss>
│  3  47.9  34.9   1.5  15.7|UUUUUUUUUUUUUUUUUUUUUUUsssssssssssssssss         >
│  4  89.4   9.7   0.2   0.8|UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUssss>|
│  5  51.9  24.4   0.1  23.7|UUUUUUUUUUUUUUUUUUUUUUUUUssssssssssss            >
│  6  27.6  54.9   3.1  14.4|UUUUUUUUUUUUUsssssssssssssssssssssssssssW        >
│  7  96.8   2.6   0.0   0.5|UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUs>
│EntitleCapacity/VirtualCPU +-----------|------------|-----------|------------+
│ EC  51.5   3.3   0.1   0.3|UUUUUUUUUUUUUUUUUUUUUUUUUs-----------------------|
│ VP  38.6   2.4   0.0   0.3|UUUUUUUUUUUUUUUUUUUs-----------------------------|
│EC=  55.2%  VP=  41.4%     +--No Cap---|------------|-----------100% VP=4 CPU+
│ Memory ─────────────────────────────────────────────────────────────────────────
│          Physical  PageSpace |        pages/sec  In     Out | FileSystemCache
│% Used       99.8%      0.3%  | to Paging Space   0.0    0.0 | (numperm) 16.8%
│% Free        0.2%     99.7%  | to File System   59.5  242.0 | Process   68.0%
│MB Used   16350.0MB    43.2MB | Page Scans        0.0        | System    15.0%
│MB Free      34.0MB 16340.8MB | Page Cycles       0.0        | Free       0.2%
│Total(MB) 16384.0MB 16384.0MB | Page Steals       0.0        |           ------
│                              | Page Faults   11402.0        | Total    100.0%
│------------------------------------------------------------ | numclient 16.8%
│Min/Maxperm     777MB(  5%)  13986MB( 85%) <--% of RAM       | maxclient 85.4%
│Min/Maxfree     960   1088       Total Virtual   32.0GB      | User      81.4%
│Min/Maxpgahead    2      8    Accessed Virtual   12.9GB 40.2%| Pinned    15.9%
│ Disk-KBytes/second-(K=1024,M=1024*1024) ────────────────────────────────────────
│Disk     Busy  Read  Write 0----------25-----------50------------75--------100
│ Name          KB/s   KB/s |           |            |             |          |
│hdisk3     0%      0      0|                                                 |
│hdisk1     0%      0      0|                                                 |
│hdisk6     0%      0      0|                                                 |
│hdisk7     0%      0      0|                                                 |
│hdisk2     0%      0      0|                                                 |
│hdisk4     0%      0      0|                                                 |
└─────────Warning: Some Statistics may not be shown───────────────────────────────

It has 16GB physical memory and 16GB virtual memory space. This machine is running as Oracle Server 10g and configured with 8G of SGA and 2G of PGA. As recommended by AWR report, the SGA is till undersized and requires to raise from 8GB to 10GB. As the machine is already suffered from high virtual memory faults (as shown above DANGER: 11402/s), it probably worsens the performance. Need to find out what causes such many paging faults.

Listing 1.2


$ vmstat -v
 4194304 memory pages            <= Total RAM = 4194304 * 4KB = 16GB
 3978160 lruable pages
 5122 free pages
 2 memory pools
 669610 pinned pages
 80.0 maxpin percentage
 5.0 minperm percentage      <= system's minperm for permenent storage (approximately File systems of Oracle)
 90.0 maxperm percentage      <= system's minperm for permenent storage (approximately File systems of Oracle)
 17.1 numperm percentage
 682216 file pages              <= actually cached in permenent storage of RAM
 <= (File systems inlcuding JFS, JFS2 and NFS)
 0.0 compressed percentage
 0 compressed pages
 17.1 numclient percentage
 90.0 maxclient percentage
 682216 client pages            <= among total files pages cached(=682216),
 <= the File systems on JFS2 and NFS are client pages (=682216)
 0 remote pageouts scheduled
 1827 pending disk I/Os blocked with no pbuf
 0 paging space I/Os blocked with no psbuf
 2228 filesystem I/Os blocked with no fsbuf
 0 client filesystem I/Os blocked with no fsbuf
 4563 external pager filesystem I/Os blocked with no fsbuf
 0 Virtualized Partition Memory Page Faults
 0.00 Time resolving virtualized partition memory page faults

 


Reference

Overview of AIX page replacement

Values for minperm and maxperm parameters

Running Oracle on AIX

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: