I am regularly contacted by customers about memory sizing when they get alerts that free memory is below a threshold, or they observe that free memory has dropped suddenly. Is there a problem? Will their application stop working because it has run out of memory for running system and application processes? Nearly always, the answer is no, there is nothing to worry about. But that simple answer is usually not enough. What's going on?

Consider the chart below. It is showing the output of the free metric in vmstat.

Problems with Strings

I am accessing IRIS databases with JDBC (or ODBC) using Python. I want to fetch the data into a pandas dataframe to manipulate the data and create charts from it. I ran into a problem with string handling while using JDBC. This post is to help if anyone else has the same issues. Or, if there is an easier way to solve this, let me know in the comments!

I am using OSX, so I am unsure how unique my problem is. I am using Jupyter Notebooks, although the code would generally be the same if you used any other Python program or framework.

I have created some example Ansible modules that add functionality above and beyond simply using the Ansible builtin Command or Shell modules to manage IRIS. You can use these as an inspiration for creating your own modules. My hope is that this will be the start of creating an IRIS community Ansible collection.

• I am assuming you are already familiar with Ansible. For example see the community post: Ansible Community Post
• This post follows up on my 2024 Global Summit presentation.

Most transactional applications have a 70:30 RW profile. However, some special cases have extremely high write IO profiles.

I ran storage IO tests in the ap-southeast-2 (Sydney) AWS region to simulate IRIS database IO patterns and throughput similar to a very high write rate application.

The test aimed to determine whether the EC2 instance types and EBS volume types available in the AWS Australian regions will support the high IO rates and throughput required.

Minimal tuning was done in the operating system or IRIS (see Operating System and IRIS configuration below).

I am often asked to review customers' IRIS application performance data to understand if system resources are under or over-provisioned.

This recent example is interesting because it involves an application that has done a "lift and shift" migration of a large IRIS database application to the Cloud. AWS, in this case.

A key takeaway is that once you move to the Cloud, resources can be right-sized over time as needed. You do not have to buy and provision on-premises infrastructure for many years in the future that you expect to grow into.

Continuous monitoring is required. Your application transaction rate will change as your business changes, the application use or the application itself changes. This will change the system resource requirements. Planners should also consider seasonal peaks in activity. Of course, an advantage of the Cloud is resources can be scaled up or down as needed.

For more background information, there are several in-depth posts on AWS and IRIS in the community. A search for "AWS reference" is an excellent place to start. I have also added some helpful links at the end of this post.

AWS services are like Lego blocks, different sizes and shapes can be combined. I have ignored networking, security, and standing up a VPC for this post. I have focused on two of the Lego block components;

• Compute requirements.
• Storage requirements.

Overview

Predictable storage IO performance with low latency is vital to provide scalability and reliability for your applications. This set of benchmarks is to inform users of IRIS considering deploying applications in AWS about EBS gp3 volume performance.

Summary

• An LVM stripe can increase IOPS and throughput beyond single EBS volume performance limits.
• An LVM stripe lowers read latency.

YASPE is the successor to YAPE (Yet Another pButtons Extractor). YASPE has been written from the ground up with many internal changes to allow easier maintenance and enhancements.

YASPE functions:

• Parse and chart InterSystems Caché pButtons and InterSystems IRIS SystemPerformance files for quick performance analysis of Operating System and IRIS metrics.
• Allow a deeper dive by creating ad-hoc charts and by creating charts combining the Operating System and IRIS metrics with the "Pretty Performance" option.
• The "System Overview" option saves you from searching your SystemPerformance files for system details or common configuration options.

YASPE is written in Python and is available on GitHub as source code or for Docker containers at:

• https://github.com/murrayo/yaspe

When looking at system performance and capacity planning I need to know what processors a server is running.

In ^SystemPerformance Linux systems report Intel processors explicitly, for example:

processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 79
model name	: Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz
stepping	: 1
microcode	: 0xffffffff
cpu MHz	: 2294.

SAM - Hacks and Tips for set up and adding metrics from non-IRIS targets

SAM (System Altering and Monitoring) comes with as a 'batteries included' docker-compose container set that is ready to start monitoring IRIS instances with a default dashboard as soon as it starts up. The initial configuration is good to understand SAM functionality and start basic monitoring of your IRIS systems. However, out of the box, there are some setting s that you will need to change when you start to monitor many systems and collect a lot of metric data.

The following steps show you how to display a sample list of metrics available from the /api/monitor service.

In the last post, I gave an overview of the service that exposes IRIS metrics in Prometheus format. The post shows how to set up and run IRIS preview release 2019.4 in a container and then list the metrics.

This post assumes you have Docker installed. If not, go and do that now for your platform :)

Step 1. Download and run the IRIS preview in docker

Follow the download instructions at Preview Distributions to download the Preview Licence Key and an IRIS Docker image.

Available at:

https://hub.docker.com/r/yape/yape/

$ docker container run --rm -v "$(pwd)":/data yape/yape --version
yape 2.2.6

See the readme at:

https://github.com/murrayo/yape

Changes include:

• Reinstate config file, make some more changes to smarter x and y axis.
• Update line style choices in config file.
• Solve for yyyy dates and yy dates or bail out. Make date string consistent for windows title (drop decimal places), add short day to title.

If a picture is worth a thousand words, what's a video worth? Certainly more than typing a post.

Please check out my "Coding talks" on InterSystems Developers YouTube:

1. Analysing InterSystems IRIS System Performance with Yape. Part 1: Installing Yape

Running Yape in a container.

2. Yape Container SQLite iostat InterSystems

Extracting and plotting pButtons data including timeframes and iostat.

Hi Everyone!

If you are attending VMworld Las Vegas remember to look for two sessions on InterSystems IRIS in the content catalog!

I am often asked by customers, vendors or internal teams to explain CPU capacity planning for large production databases running on VMware vSphere.

This post was originally written in 2017, I am updating the post in February 2026. For context I have kept the original post, but highlighted changes. This post was originally written for ESXi 6.0. The core principles remain valid for vSphere 7.x and 8.x, though there have been improvements to vNUMA handling, CPU scheduling (particularly for AMD EPYC), and CPU Hot Add compatibility with vNUMA in vSphere 8.

I am looking for experience of people running Veeam with Caché databases. 

Tips/Tricks/General questions like; what Veeam features are you using, what your backup cycle looks like, where does your data end up, what recovery/integrity checks you do, what sort of compression/dedupe you get. 

Also what questions _you_ have and what problems you might be trying to solve.

Hi, this post was initially written for Caché. In June 2023, I finally updated it for IRIS. If you are revisiting the post since then, the only real change is substituting Caché for IRIS! I also updated the links for IRIS documentation and fixed a few typos and grammatical errors. Enjoy :)

In this post, I show strategies for backing up InterSystems IRIS using External Backup with examples of integrating with snapshot-based solutions. Most solutions I see today are deployed on Linux on VMware, so a lot of the post shows how solutions integrate VMware snapshot technology as examples.

This post provides guidelines for configuration, system sizing and capacity planning when deploying IRIS and IRIS on a VMware ESXi. This post is based on and replaces the earlier IRIS-era guidance and reflects current VMware and InterSystems recommendations.

Last update Jan 2026. These guidelines are a best effort, remember requirements and capabilities of VMware and IRIS can change.

I jump right in with recommendations assuming you already have an understanding of VMware vSphere virtualization platform.

Index

This is a list of all the posts in the Data Platforms’ capacity planning and performance series in order. Also a general list of my other posts. I will update as new posts in the series are added.

You will notice that I wrote some posts before IRIS was released and refer to Caché. I will revisit the posts over time, but in the meantime, Generally, the advice for configuration is the same for Caché and IRIS. Some command names may have changed; the most obvious example is that anywhere you see the ^pButtons command, you can replace it with ^SystemPerformance.

While some posts are updated to preserve links, others will be marked as strikethrough to indicate that the post is legacy. Generally, I will say, "See: some other post" if it is appropriate.

Capacity Planning and Performance Series

Generally, posts build on previous ones, but you can also just dive into subjects that look interesting.

• Part 1 - Getting started on the Journey, collecting metrics.
• Part 2 - Looking at the metrics we collected.
• Part 3 - Focus on CPU.
• Part 4 - Looking at memory.
• Part 5 - Monitoring with SNMP.
• Part 6 - Caché storage IO profile.
• Part 7 - ECP for performance, scalability and availability.
• Part 8 - Hyper-Converged Infrastructure Capacity and Performance Planning
• Part 9 - Caché VMware Best Practice Guide
• Part 10 - VM Backups and IRIS freeze/thaw scripts
• Part 11 - Virtualizing large databases - VMware cpu capacity planning

Here in Community I use the WYSIWYG Text format control to answer questions and other quick text entries.

But for longer posts when I want formatting or if I am building incrementally over several days I use the Plain text (supports markdown) control because it's quicker and easier to post an article I have written offline. In this post I share my workflow and a set of tools to publish long read posts.

What is markdown?

There are references later in this post with more info on how markdown works.

One of the great availability and scaling features of Caché is Enterprise Cache Protocol (ECP). With consideration during application development distributed processing using ECP allows a scale out architecture for Caché applications. Application processing can scale to very high rates from a single application server to the processing power of up to 255 application servers with no application changes.

ECP was used widely for many years in TrakCare deployments I was involved in. A decade ago a 'big' x86 server from one of the major vendors might only have a total of eight cores.

I saw someone recently refer to ECP as magic. It certainly seems so, and there is a lot of very clever engineering to make it work. But the following sequence of diagrams is a simple view of how data is retrieved and used across a distributed architecture.

For more more on ECP including capacity planning follow this link: Data Platforms and Performance - Part 7 ECP for performance, scalability and availability

To start

• There are three globals on disk ^A, ^B and ^C.
• Global ^B equals "B"
• There is one Data server and two or more Application servers.

Post updated in August 2025 to include links to IRIS.

I have seen customer problems where the use of a virus scanner running over Caché or IRIS databases was causing intermittent application slowdowns and bad user response times.

This is a surprisingly common problem, so this short post is just a reminder to exclude key Caché and IRIS components from your virus scanning.

Generally, virus scanning must exclude the CACHE.DAT or IRIS.DAT database files and the InterSystems binaries. If an anti-virus is scanning *.

This post will guide you through the process of sizing shared memory requirements for database applications running on InterSystems data platforms. It will cover key aspects such as global and routine buffers, gmheap, and locksize, providing you with a comprehensive understanding. Additionally, it will offer performance tips for configuring servers and virtualizing IRIS applications. Please note that when I refer to IRIS, I include all the data platforms (Ensemble, HealthShare, iKnow, Caché, and IRIS).

[A list of other posts in this series is here](https://community.intersystems.

Presenter: Murray Oldfield
Task: Avoid bottlenecks caused by scaling up, before they become a problem
Approach: Discuss what operating system and InterSystems metrics you should look at and how to interpret them
 

Systems behave differently when database activity scales up. In the worst case, bottlenecks appear and users are impacted. This session shows you which operating system and InterSystems metrics you should be looking at and how to interpret them so you can head off bottlenecks before they impact users. This sessions also shows strategies for planning infrastructure taking into consideration InterSystems' data platforms requirements.

Content related to this session, including slides, video and additional learning content can be found here.

Presenter: Murray Oldfield
Task: Deploy applications based on InterSystems’ technology using VMware.
Approach: Provide a checklist of factors to consider, particularly when deploying a production database application that requires high availability
 

Are you ready to deploy your applications on a virtualized architecture? This talk will highlight what you need to plan and do when deploying applications built on ISC data platforms using VMware. Special focus on what you need to know when planning for highly available (HA) production database applications.

Content related to this session, including slides, video and additional learning content can be found here.

Previously I showed you how to run pButtons to start collecting performance metrics that we are looking at in this series of posts.

• Part 1 - Getting started on the Journey, collecting metrics
• Part 2 - Looking at the metrics we collected

##Update: May 2020.

Since this post was written several years ago, we have moved from Caché to IRIS. See the comments for an updated link to the documentation for pButtons (Caché) and SystemPerformance (IRIS). Also, a note on how to update your systems to the latest versions of the performance tools.