Thursday, June 4, 2009

PCS - Predictive Cache Statistics for PAM cards

This is in continuation of last part where I have covered details of PAM, in this part I will cover PCS which you can use to determine if the PAM module will give you benifite before ordering your one from NetApp

PCS: Determining If PAM Will Improve Performance

To determine whether your storage systems can benefit from added cache, NetApp has developed its Predictive Cache Statistics software, which is currently available in Data ONTAP 7.3 and later releases. PCS allows you to predict the effects of adding the cache equivalent of two, four, and eight times system memory.

Using PCS, you can determine whether PAM will improve performance for your workloads and decide how many modules you will need. PCS also allows you to test the different modes of operation to determine whether the default, metadata, or low-priority mode is best.

To begin using PCS, you enable the feature with the command:

options flexscale.enable pcs

Don’t enable PCS if your storage system is consistently above 80% CPU utilization. Once PCS is enabled, you have to let the simulated cache “warm up” or gather data blocks. Once the cache is warmed up, you can review and analyze the data using the perfstat tool.

This procedure simulates caching using the default caching mode that includes both metadata and normal user data. You can also test the other operating modes.

To enable metadata mode:

options flexscale.normal_data_blocks off

To enable low-priority mode:

options flexscale.normal_data_blocks on
options flexscale.lopri_blocks on

Once you have completed testing, disable PCS:

options flexscale.enable off

With PCS enabled, you can find out what's happening using the following command:

stats show -p flexscale-pcs

Sample output is shown below



Example PCS output.

Use the following guidelines to help you interpret the data:

· If the hit/(invalidate+evict) ratio is small, then a lot of data is being discarded before it is used. The instance (ec0, ec1, ec2) may be too small.

· If the (hit+miss)/invalidate ratio is too small, it might indicate a workload with a large amount of updates; switch to metadata mode and check the hit% again.

· If the usage is stable and there are a small number of invalidates and evictions, then the working set fits well.

· The KB/s served by the cache is approximately equal to the hit/s × 4KB per block.

Note that the three caches simulated in PCS are cascading caches. In the example above, ec0 represents the first cache of size 8GB, ec1 represents the second cache of size 8GB, and ec3 represents the third cache of size 16GB. The hit per second for a 32GB cache is the sum of all the hits per second for all three caches. The key advantage of cascading caches is that in the process of measuring an accurate hit rate for a 32GB cache, we also obtain hit rate estimates of both 8GB and 16GB caches. This gives us three points on the hit rate curve and the ability to estimate hit rates for intermediate cache sizes.

PAM and FlexShare

FlexShare™ is a Data ONTAP option that allows you to set priorities for system resources (processors, memory, and I/O) at a volume level, thereby allocating more resources to workloads on particular volumes when the controller is under significant load. FlexShare is fully compatible with PAM, and settings made in FlexShare apply to the data kept in the PAM cache. With FlexShare, finer-grained control can be applied on top of the global policies you implement with PAM. For example, if an individual volume is given a higher priority with FlexShare, data from that volume will receive a higher priority in the cache.

8 comments:

chrisis said...

afaik the command to switch PCS on is
options flexscale.enable on
(and not options flexscale.enable pcs)

Unknown said...

Chrisis,

To enable PCS which just mimics as PAM card you need to give 'pcs' as parameter whereas if you have PAM card installed in your system you give 'on'.

Nayab said...

Hi Mohit,

Its really good stuff we are using PCS in our environment but i was able to see only one Instance configured for all our filers ( ec0 ).
Could you please help to determine if this is the reason for the performance on our filers else what needs to be done to improve this. Please find the values below

Thanks,
Nayab

Nayab said...

netp3> stats show -p flexscale-pcs
Instance Blocks GB Usage Hit Metadata Miss Hit Evict Invalidate Insert Reads Replaced
% /s /s /s % /s /s /s /s
ec0 134217728 512 97 6016 355 16243 27 360 12500 9594 1405
---
ec0 134217728 512 97 7871 124 18764 29 570 12785 15373 1878
---
ec0 134217728 512 97 7603 611 22548 25 675 18176 16516 1814
---
ec0 134217728 512 97 8399 601 19793 29 394 7104 10568 2952
---
ec0 134217728 512 97 10476 116 19684 34 684 17951 18385 2603
---
ec0 134217728 512 97 10677 369 17745 37 513 12608 14520 2226
---
ec0 134217728 512 97 9079 242 14096 39 431 12690 11981 2834
---
ec0 134217728 512 97 8550 325 16523 34 364 12592 9869 2906

Nayab said...

netp4> stats show -p flexscale-pcs
Instance Blocks GB Usage Hit Metadata Miss Hit Evict Invalidate Insert Reads Replace
% /s /s /s % /s /s /s /
ec0 134217728 512 97 1412 161 452 75 28 0 899 67
---
ec0 134217728 512 97 1172 554 300 79 0 20 0 84
---
ec0 134217728 512 97 1096 89 250 81 30 0 849 65
---
ec0 134217728 512 97 3048 250 384 88 10 0 316 117
---
ec0 134217728 512 97 4917 91 2117 69 131 6678 3578 92
---
ec0 134217728 512 97 1081 67 289 78 0 0 0 62

Nayab said...

netp5> stats show -p flexscale-pcs
Instance Blocks GB Usage Hit Metadata Miss Hit Evict Invalidate Inser t Reads Replaced
% /s /s /s % /s /s / s /s
ec0 134217728 512 97 2697 478 26577 9 544 12618 1323 9 937
---
ec0 134217728 512 97 3050 649 25813 10 256 5506 694 4 1116
---
ec0 134217728 512 97 2185 359 27059 7 357 12711 1184 6 899
---
ec0 134217728 512 97 2401 586 23861 9 215 6883 694 5 1047
---
ec0 134217728 512 97 3139 593 30562 9 331 5705 844 5 1177
---
ec0 134217728 512 97 2188 549 24623 8 374 12594 8688 895
---
ec0 134217728 512 97 2884 476 26480 9 308 6884 7092 878
---
ec0 134217728 512 97 2779 415 27591 9 257 5633 7083 927
---
ec0 134217728 512 97 3154 485 26961 10 260 7006 7062 1064
---
ec0 134217728 512 97 2936 457 26865 9 348 12518 10132 774
---
ec0 134217728 512 97 3490 483 24708 12 250 5920 7629 1163
---
ec0 134217728 512 97 3327 457 21223 13 350 6687 9935 915
---
ec0 134217728 512 97 3684 553 24481 13 397 12590 9692 995
---

Nayab said...

netp6> stats show -p flexscale-pcs
Instance Blocks GB Usage Hit Metadata Miss Hit Evict Invalidate t Reads Replaced
% /s /s /s % /s /s s /s
ec0 134217728 512 97 2508 640 14951 14 406 12286 1 938
---
ec0 134217728 512 97 2403 573 17135 12 224 0 2 907
---
ec0 134217728 512 97 2632 515 18613 12 324 12640 9088 866
---
ec0 134217728 512 97 5794 433 20178 22 251 7090 8390 1040
---
ec0 134217728 512 97 3033 710 12800 19 234 5393 6729 1262
---
ec0 134217728 512 97 2533 381 13410 15 229 7280 6001 1038
---
ec0 134217728 512 97 2757 504 14442 16 226 5299 6100 1060

Unknown said...

Hi Nayab,

I am little confused with only one instance for all of your filers, can you please elaborate?

As designed PCS is only to mimic FlashCache functionality and it help determining if addition of FlashCache will help your system. In no ways it helps your system get higher performance.

Regards,
Mohit