InterSystems Data Platform Blog

Syndicate content 30 

InterSystems Data Platform includes utilities and tools for system monitoring and alerting, however System Administrators new to solutions built on the InterSystems Data Platform (a.k.a Caché) need to know where to start and what to configure.

This guide shows the path to a minimum monitoring and alerting solution using references from online documentation and developer community posts to show you how to enable and configure the following;

  1. Caché Monitor: Scans the console log and sends emails alerts.

  2. System Monitor: Monitors system status and resources, generating notifications (alerts and warnings) based on fixed parameters and also tracks overall system health.

  3. Health Monitor: Samples key system and user-defined metrics and compares them to user-configurable parameters and established normal values, generating notifications when samples exceed applicable or learned thresholds.

  4. History Monitor: Maintains a historical database of performance and system usage metrics.

  5. pButtons: Operating system and Caché metrics collection scheduled daily.

Remember this guide is a minimum configuration, the included tools are flexible and extensible so more functionality is available when needed. This guide skips through the documentation to get you up and going. You will need to dive deeper into the documentation to get the most out of the monitoring tools, in the meantime, think of this as a set of cheat sheets to get up and running.

Last comment 7 days ago
3 4
596

views

+ 10

rating

As we all know, Caché is a great database that accomplishes lots of tasks within itself. However, what do you do when you need to access an external database? One way is to use the Caché SQL Gateway via JDBC. In this article, my goal is to answer the following questions to help you familiarize yourself with the technology and debug some common problems

Last comment 28 December 2018
2 4
1712

views

+ 8

rating

In this post I show strategies for backing up Caché using External Backup with examples of integrating with snapshot based solutions. The majority of solutions I see today are deployed on Linux on VMware so a lot of the post shows how solutions integrate VMware snapshot technology as examples.

Caché backup - batteries included?

Caché online backup is included out of the box with a Caché install for uninterrupted backup of Caché databases. But there are more efficient backup solutions you should consider as systems scale up. External Backup integrated with snapshot technologies is the recommended solution for backing up systems including Caché databases.

Are there any special considerations for external backup?

Online documentation for External Backup has all the details. A key consideration is

Last comment 16 November 2018
1 5
3882

views

+ 12

rating

Hi Community!

As you know we launched the InterSystems Open Exchange — the marketplace for solutions and tools on InterSystems Data Platforms!

But how to publish your application on OE?

Before we start, let me answer the few basic questions.

Last comment 24 October 2018
2 3
214

views

+ 4

rating

Created by Daniel Kutac, Sales Engineer, InterSystems

Warning: if you get confused by URLs used: the original series used screens from machine called dk-gs2016. The new screenshots are taken from a different machine. You can safely treat url WIN-U9J96QBJSAG as if it was dk-gs2016.

Part 2. Authorization server, OpenID Connect server

In the previous part of this short series, we have learned about simple use case – acting as an OAUTH[1] client. Now, it’s time to bring our experience to a whole new level. We are going to build much more complex environment, where InterSystems IRIS is going to play all OAUTH roles

Last comment 10 August 2018
0 6
2051

views

+ 7

rating


Hello!

This article is a small overview of a tool that helps to understand classes and their structure inside the Caché DBMS.

In short, it visualizes a class or an entire package, shows the relations between classes and provides all the possible information to developers and team leads without making them go to Caché Studio and examine the code there.

If you are learning InterSystems Caché, reviewing Caché projects a lot or just interested in something new in InterSystems Technology solutions — you are more than welcome to read the overview of Caché Class Explorer!

Last comment 3 July 2018
1 11
3002

views

+ 17

rating

++ Update: August 1, 2018

The use of the InterSystems Virtual IP (VIP) address built-in to Caché database mirroring has certain limitations. In particular, it can only be used when mirror members reside the same network subnet. When multiple data centers are used, network subnets are not often “stretched” beyond the physical data center due to added network complexity (more detailed discussion here). For similar reasons, Virtual IP is often not usable when the database is hosted in the cloud.

Network traffic management appliances such as load balancers (physical or virtual) can be used to achieve the same level of transparency, presenting a single address to the client applications or devices. The network traffic manager automatically redirects clients to the current mirror primary’s real IP address. The automation is intended to meet the needs of both HA failover and DR promotion following a disaster. 

Last comment 14 June 2018
0 7
2076

views

+ 11

rating

This post provides useful links and an overview of best practice configuration for low latency storage IO by creating LVM Physical Extent (PE) stripes for database disks on InterSystems Data Platforms; InterSystems IRIS, Caché, and Ensemble.

Consistent low latency storage is key to getting the best database application performance. For applications running on Linux, Logical Volume Manager (LVM) is often used for database disks, for example because of the ability to grow volumes and filesystems or create snapshots for online backups. For database applications the parallelism of writes using LVM PE striped logical volumes can also help increase performance for large sequential reads and writes by improving the efficiency of the data I/O

Last comment 25 May 2018
1 4
646

views

+ 5

rating

Purpose

This tool is used to generate random read Input/Output (IO) from within the Caché database. The goal of this tool is to drive as many jobs as possible to achieve target IOPS and ensure acceptable disk response times are sustained. Results gathered from the IO tests will vary from configuration to configuration based on the IO sub-system. Before running these tests ensure corresponding operating system and storage level monitoring are configured to capture IO performance metrics for later analysis.

Last comment 15 May 2018
0 9
1401

views

+ 10

rating

In the last post we scheduled 24-hour collections of performance metrics using pButtons. In this post we are going to be looking at a few of the key metrics that are being collected and how they relate to the underlying system hardware. We will also start to explore the relationship between Caché (or any of the InterSystems Data Platforms) metrics and system metrics. And how you can use these metrics to understand the daily beat rate of your systems and diagnose performance problems.

Last comment 24 March 2018
0 16
1965

views

+ 15

rating

Hi, Community!

In 2017 we had 734 different contributors to Developer Community who posted articles and announcements, questions and answers.

This post is a compilation of Top Authors, Top Experts and Top Opinion Makers of InterSystems Developer Community in 2017.

It is a good guide "Who to Follow" in 2018.

And I'm glad to present these people!

0 1
0

comments

205

views

+ 2

rating

0 2
0

comments

210

views

0

rating

A request came from a customer to estimate how long it would take to encrypt a database with cvencrypt utility.

This question is a little bit like how long is a piece of string — it depends. But its an interesting question. The answer primarily depends on the performance of CPU and storage on the target platform the customer is using, so the answer is more about coming up with a simple methodology that can be used to benchmark the CPU and storage while running cvencrypt.

Methodology

  1. Copy a large and representative CACHE.DAT file to target storage
  2. Create a keyfile via System Management Portal (includes a key)
  3. Run the cvencrypt over your sample CACHE.DAT file (as below)

The following shows the process once the test file is in place

0 1
0

comments

316

views

+ 5

rating

I am often asked by customers, vendors or internal teams to explain CPU capacity planning for large production databases running on VMware vSphere.

In summary there are a few simple best practices to follow for sizing CPU for large production databases:

  • Plan for one vCPU per physical CPU core.
  • Consider NUMA and ideally size VMs to keep CPU and memory local to a NUMA node.
  • Right-size virtual machines. Add vCPUs only when needed.

Generally this leads to a couple of common questions

Last comment 8 November 2017
0 3
3133

views

+ 5

rating

This post will show you an approach to size shared memory requirements for database applications running on InterSystems data platforms including global and routine buffers, gmheap, and locksize as well as some performance tips you should consider when configuring servers and when virtualizing Caché applications. As ever when I talk about Caché I mean all the data platform (Ensemble, HealthShare, iKnow and Caché).


A list of other posts in this series is here

Last comment 22 August 2017
2 11
4923

views

+ 18

rating

Hyper-Converged Infrastructure (HCI) solutions have been gaining traction for the last few years with the number of deployments now increasing rapidly. IT decision makers are considering HCI when scoping new deployments or hardware refreshes especially for applications already virtualised on VMware. Reasons for choosing HCI include; dealing with a single vendor, validated interoperability between all hardware and software components, high performance especially IO, simple scalability by addition of hosts, simplified deployment and simplified management

Last comment 16 August 2017
0 6
1775

views

+ 7

rating

It was InterSystems hackathon time and our team, consisting of Artem Viznyuk and me had Arduino board (one) and various parts of it (in overabundance). And so like that our course of action was set - like all other Arduino beginners, we decided to build a weather station. But with data persistent storage in Caché and visualization in DeepSee!

Last comment 8 June 2017
0 7
1082

views

+ 12

rating

This week I am going to look at CPU, one of the primary hardware food groups :) A customer asked me to advise on the following scenario; Their production servers are approaching end of life and its time for a hardware refresh. They are also thinking of consolidating servers by virtualising and want to right-size capacity either bare-metal or virtualized. Today we will look at CPU, in later posts I will explain the approach for right-sizing other key food groups - memory and IO.

So the questions are

Last comment 7 June 2017
0 4
2633

views

+ 11

rating

The Data Platforms department here at InterSystems is gearing up for this year's crop of interns, and I for one am very excited to meet them all next week!

We've got folks from top technical colleges with diverse specialties from hard core engineers to pure computer scientists to mathematicians to business professionals. They come from countries around the world like Vietnam, China, and Finland and they all come with impressive backgrounds. We're sure they will do very well this summer.

Rather than providing a set list of projects this year, Data Platforms interns will be developing their own projects which satisfy a short list of requirements. I can't wait to see what they come up with!

For an idea of what to expect, check out the white paper attached from my group last year who ported Deep Feature Synthesis research done at MIT for SQL platforms to Caché. Sarat Vysyaraju and Ryan St. Pierre did me and all the other mentors who provided guidance proud

0 3
0

comments

280

views

+ 6

rating

Note (Sept 2018): There have been big changes since this post first appeared, I suggest using the Docker Container version, the project and details for running as a container are still in the same place  published on GitHub so you can download, run - and modify if you need to

Last comment 11 May 2017
1 5
864

views

+ 8

rating

Created by Daniel Kutac, Sales Engineer, InterSystems

 

Part 3. Appendix

InterSystems IRIS OAUTH classes explained

In the previous part of our series we have learned about configuring InterSystems IRIS to act as an OAUTH client as well as authorization and authentication server (by means of OpenID Connect). In this final part of our series we are going to describe classes implementing InterSystems IRIS OAuth 2.0 framework. We will also discuss use cases for selected methods of API classes.

The API classes implementing OAuth 2.0 can be separated into three different groups according to their purpose. All classes are implemented in %SYS namespace. Some of them are public (via % package), some not and should not be called by developers directly.

0 2
0

comments

1180

views

+ 5

rating

In this post, I am going to detail how to set up a mirror using SSL, including generating the certificates and keys via the Public Key Infrastructure built in to Caché. The goal of this is to take you from new installations to a working mirror with SSL, including a primary, backup, and DR async member, along with a mirrored database. I will not go into security recommendations or restricting access to the files. This is meant to just simply get a mirror up and running. Example screenshots are taken on a 2016.1 version of Caché, so yours may look slightly different.

Step 1: Configure Certificate Authority (CA) Server

Last comment 29 March 2017
0 0
962

views

+ 4

rating

In previous posts I have shown how it is possible to collect historical performance metrics using pButtons. I go to pButtons first because I know it is installed with every Data Platforms instance (Ensemble, Caché, …). However there are other ways to collect, process and display Caché performance metrics in real time either for simple monitoring or more importantly for much more sophisticated operational analytics and capacity planning. One of the most common methods of data collection is to use SNMP (Simple Network Management Protocol). SNMP a standard way for Caché to provide management and monitoring information to a wide variety of management tools. The Caché online documentation includes details of the interface between Caché and SNMP. While SNMP should 'just work' with Caché there are some configuration tricks and traps.

Last comment 17 March 2017
1 2
2066

views

+ 9

rating

** Revised Feb-12, 2018

While this article is about InterSystems IRIS, it also applies to Caché, Ensemble, and HealthShare distributions.

Introduction

Memory is managed in pages.  The default page size is 4KB on Linux systems.  Red Hat Enterprise Linux 6, SUSE Linux Enterprise Server 11, and Oracle Linux 6 introduced a method to provide an increased page size in 2MB or 1GB sizes depending on system configuration know as HugePages.

At first HugePages required to be assigned at boot time, and if not managed or calculated appropriately could result in wasted resources.  As a result various Linux distributions introduced Transparent HugePages with the 2.6.38 kernel as enabled by default.  This was meant as a means to automate creating, managing, and using HugePages.  Prior kernel versions may have this feature as well however may not be marked as [always] and potentially set to [madvise].  

Transparent Huge Pages (THP) is a Linux memory management system that reduces the overhead of Translation Lookaside Buffer (TLB) lookups on machines with large amounts of memory by using larger memory pages.  However in current Linux releases THP can only map individual process heap and stack space.

Last comment 22 February 2017
1 9
2344

views

+ 4

rating

Ansible helped me solve the problem of quickly deploying Caché and application components for Data Platforms benchmarks. You can use the same tools and methodology for standing up your test labs, training systems, development or other environments. If you deploy applications at customer sites you could automate much of the deployment and ensure that system, Caché and your application are configured to your applications best practice standards.

Last comment 29 December 2016
0 7
1231

views

+ 13

rating

One of the great availability and scaling features of Caché is Enterprise Cache Protocol (ECP). With consideration during application development distributed processing using ECP allows a scale out architecture for Caché applications. Application processing can scale to very high rates from a single application server to the processing power of up to 255 application servers with no application changes.

ECP was used widely for many years in TrakCare deployments I was involved in. A decade ago a 'big' x86 server from one of the major vendors might only have a total of eight cores. For larger deployments ECP was a way to scale out processing on commodity servers rather than a single large and expensive big iron enterprise server. Even the high core count enterprise servers had limits so ECP was also used to scale deployments on them as well

Last comment 8 December 2016
0 6
1191

views

+ 8

rating

This post provides guidelines for configuration, system sizing and capacity planning when deploying Caché 2015 and later on a VMware ESXi 5.5 and later environment.

I jump right in with recommendations assuming you already have an understanding of VMware vSphere virtualization platform. The recommendations in this guide are not specific to any particular hardware or site specific implementation, and are not intended as a fully comprehensive guide to planning and configuring a vSphere deployment -- rather this is a check list of best practice configuration choices you can make. I expect that the recommendations will be evaluated for a specific site by your expert VMware implementation team.


A list of other posts in the InterSystems Data Platforms and performance series is here.

2 2
0

comments

2873

views

+ 8

rating

Myself and the other Technology Architects often have to explain to customers and vendors Caché IO requirements and the way that Caché applications will use storage systems. The following tables are useful when explaining typical Caché IO profile and requirements for a transactional database application with customers and vendors.  The original tables were created by Mark Bolinsky.

In future posts I will be discussing more about storage IO so am also posting these tables now as a reference for those articles. 



A list of other posts in this series is here
 


It is vital to have well set up storage such as a storage array to provide predictable disk IO performance, support high availability features and provide storage redundancy, scalability, and reliability for your applications

Last comment 13 November 2016
1 2
1261

views

+ 8

rating

This is a list of all the posts in the data platforms capacity planning and performance series in order. Also a general list of my other posts. I will update as new posts in the series are added.


Capacity Planning and Performance Series

Generally posts build on previous, but you can also just dive in to subjects that look interesting

0 14
0

comments

2799

views

+ 10

rating

In this post I would like to talk about the syslog table.  I will cover what it is, how you look at it, what the entries really are, and why it may be important to you.  The syslog table can contain important diagnostic information.  If your system is having any problems, it is important to understand how to look at this table and what information is contained there.

Last comment 9 November 2016
0 5
797

views

+ 14

rating