InterSystems IRIS

Syndicate content 42 

This post will show you an approach to size shared memory requirements for database applications running on InterSystems data platforms including global and routine buffers, gmheap, and locksize as well as some performance tips you should consider when configuring servers and when virtualizing Caché applications. As ever when I talk about Caché I mean all the data platform (Ensemble, HealthShare, iKnow and Caché).


A list of other posts in this series is here

Last comment 22 August 2017
+ 21   4 11
5653

views

+ 21

rating

In this post I show strategies for backing up Caché using External Backup with examples of integrating with snapshot based solutions. The majority of solutions I see today are deployed on Linux on VMware so a lot of the post shows how solutions integrate VMware snapshot technology as examples.

Last comment 30 September 2019
+ 13   1 6
4637

views

+ 13

rating

This article, and following two articles of the series, is intended as a user guide for developers or system administrators, who need to work with OAuth 2.0 framework (further referred to as OAUTH for simplicity) in their InterSystems product based applications.

Last comment 27 September 2019
+ 12   5 6
4572

views

+ 12

rating

I am often asked by customers, vendors or internal teams to explain CPU capacity planning for large production databases running on VMware vSphere.

In summary there are a few simple best practices to follow for sizing CPU for large production databases:

  • Plan for one vCPU per physical CPU core.
  • Consider NUMA and ideally size VMs to keep CPU and memory local to a NUMA node.
  • Right-size virtual machines. Add vCPUs only when needed.

Generally this leads to a couple of common questions:

Last comment 8 November 2017
+ 5   0 3
3766

views

+ 5

rating


Hello!

This article is a small overview of a tool that helps to understand classes and their structure inside the InterSystems products: from IRIS to Caché, Ensemble, HealthShare.

In short, it visualizes a class or an entire package, shows the relations between classes and provides all the possible information to developers and team leads without making them go to Studio and examine the code there.

If you are learning InterSystems products, reviewing projects a lot or just interested in something new in InterSystems Technology solutions — you are more than welcome to read the overview of ObjectScript Class Explorer!

Last comment 19 July 2019
+ 17   2 12
3495

views

+ 17

rating

Mirroring 101

Caché mirroring is a reliable, inexpensive, and easy to implement high availability and disaster recovery solution for Caché and Ensemble-based applications. Mirroring provides automatic failover under a broad range of planned and unplanned outage scenarios, with application recovery time typically limited to seconds. Logical data replication eliminates storage as a single point of failure and a source of data corruption. Upgrades can be executed with little or no downtime.

Last comment 8 December 2018
+ 9   2 10
3335

views

+ 9

rating

This post provides guidelines for configuration, system sizing and capacity planning when deploying Caché 2015 and later on a VMware ESXi 5.5 and later environment.

+ 9   3 2
0

comments

3321

views

+ 9

rating

This is a list of all the posts in the data platforms capacity planning and performance series in order. Also a general list of my other posts. I will update as new posts in the series are added.


Capacity Planning and Performance Series

Generally posts build on previous, but you can also just dive in to subjects that look interesting.

+ 11   0 14
0

comments

3251

views

+ 11

rating

This week I am going to look at CPU, one of the primary hardware food groups :) A customer asked me to advise on the following scenario; Their production servers are approaching end of life and its time for a hardware refresh. They are also thinking of consolidating servers by virtualising and want to right-size capacity either bare-metal or virtualized. Today we will look at CPU, in later posts I will explain the approach for right-sizing other key food groups - memory and IO.

So the questions are:

Last comment 30 September 2019
+ 13   2 5
2920

views

+ 13

rating

Hi!

I believe the simplest is (to work with csv delimited by ";"):


set file = ##class(%File).%New( "data.csv" )
    set sc = file.Open( "R" ) 
    if $$$ISERR(sc) quit    ; or do smth

    while 'file.AtEnd {
        set str=file.ReadLine() 
        for i=1:1:$length( str, ";" ) {
            set id=$piece( str, ";" ,i ) 
            write !, id  // or do smth
        }
    }
    do file.Close()

Possible options:

different variants of error handling with sc code.

Embrace while loop into try/catch block.

And what's yours?

Last comment 12 May 2016
+ 1   0 6
2875

views

+ 1

rating

** Revised Feb-12, 2018

While this article is about InterSystems IRIS, it also applies to Caché, Ensemble, and HealthShare distributions.

Introduction

Memory is managed in pages.  The default page size is 4KB on Linux systems.  Red Hat Enterprise Linux 6, SUSE Linux Enterprise Server 11, and Oracle Linux 6 introduced a method to provide an increased page size in 2MB or 1GB sizes depending on system configuration know as HugePages.

At first HugePages required to be assigned at boot time, and if not managed or calculated appropriately could result in wasted resources.  As a result various Linux distributions introduced Transparent HugePages with the 2.6.38 kernel as enabled by default.  This was meant as a means to automate creating, managing, and using HugePages.  Prior kernel versions may have this feature as well however may not be marked as [always] and potentially set to [madvise].  

Transparent Huge Pages (THP) is a Linux memory management system that reduces the overhead of Translation Lookaside Buffer (TLB) lookups on machines with large amounts of memory by using larger memory pages.  However in current Linux releases THP can only map individual process heap and stack space.

Last comment 22 February 2017
+ 5   2 9
2669

views

+ 5

rating

Created by Daniel Kutac, Sales Engineer, InterSystems

Warning: if you get confused by URLs used: the original series used screens from machine called dk-gs2016. The new screenshots are taken from a different machine. You can safely treat url WIN-U9J96QBJSAG as if it was dk-gs2016.

Part 2. Authorization server, OpenID Connect server

Last comment 10 August 2018
+ 8   1 6
2506

views

+ 8

rating

++ Update: August 1, 2018

The use of the InterSystems Virtual IP (VIP) address built-in to Caché database mirroring has certain limitations. In particular, it can only be used when mirror members reside the same network subnet. When multiple data centers are used, network subnets are not often “stretched” beyond the physical data center due to added network complexity (more detailed discussion here). For similar reasons, Virtual IP is often not usable when the database is hosted in the cloud.

Network traffic management appliances such as load balancers (physical or virtual) can be used to achieve the same level of transparency, presenting a single address to the client applications or devices. The network traffic manager automatically redirects clients to the current mirror primary’s real IP address. The automation is intended to meet the needs of both HA failover and DR promotion following a disaster. 

Last comment 14 June 2018
+ 11   0 7
2469

views

+ 11

rating

Your application is deployed and everything is running fine. Great, hi-five! Then out of the blue the phone starts to ring off the hook – it’s users complaining that the application is sometimes ‘slow’. But what does that mean? Sometimes? What tools do you have and what statistics should you be looking at to find and resolve this slowness? Is your system infrastructure up to the task of the user load? What infrastructure design questions should you have asked before you went into production? How can you capacity plan for new hardware with confidence and without over-spec'ing? How can you stop the phone ringing? How could you have stopped it ringing in the first place?

Last comment 30 September 2016
+ 18   4 19
2385

views

+ 18

rating

1.About this article

Just like Caché pattern matching, Regular Expressions can be used in Caché to identify patterns in text data – only with a much higher expressive power. This article provides a brief introduction into Regular Expressions and what you can do with it in Caché. The information provided herein is based on various sources, most notably the book “Mastering Regular Expressions” by Jeffrey Friedl and of course the Caché online documentation. The article is not intended to discuss all the possibilities and details of regular expressions. Please refer to the information sources listed in chapter 5 if you would like to learn more. If you prefer to read off-line you can also download the PDF version of this article.

Last comment 17 June 2019
+ 15   3 9
2360

views

+ 15

rating

In the last post we scheduled 24-hour collections of performance metrics using pButtons. In this post we are going to be looking at a few of the key metrics that are being collected and how they relate to the underlying system hardware. We will also start to explore the relationship between Caché (or any of the InterSystems Data Platforms) metrics and system metrics. And how you can use these metrics to understand the daily beat rate of your systems and diagnose performance problems.

Last comment 24 March 2018
+ 17   2 16
2247

views

+ 17

rating

Hyper-Converged Infrastructure (HCI) solutions have been gaining traction for the last few years with the number of deployments now increasing rapidly. IT decision makers are considering HCI when scoping new deployments or hardware refreshes especially for applications already virtualised on VMware. Reasons for choosing HCI include; dealing with a single vendor, validated interoperability between all hardware and software components, high performance especially IO, simple scalability by addition of hosts, simplified deployment and simplified management.

Last comment 16 August 2017
+ 8   1 6
2003

views

+ 8

rating

Purpose

This tool is used to generate random read Input/Output (IO) from within the database. The goal of this tool is to drive as many jobs as possible to achieve target IOPS and ensure acceptable disk response times are sustained. Results gathered from the IO tests will vary from configuration to configuration based on the IO sub-system. Before running these tests ensure corresponding operating system and storage level monitoring are configured to capture IO performance metrics for later analysis.

Last comment 31 July 2019
+ 10   0 10
1672

views

+ 10

rating

InterSystems is pleased to announce that InterSystems IRIS Data Platform 2018.1.0 is now released.

This press release was issued this morning.

InterSystems IRIS is a complete, unified data platform that makes it faster and easier to build real-time, data-rich applications.  It mirrors our design philosophy that software should be interoperable, reliable, intuitive, and scalable.

Last comment 16 May 2018
+ 3   0 9
1535

views

+ 3

rating

Often InterSystems technology architect team is asked about recommended storage arrays or storage technologies.  To provide this information to a wider audience as reference, a new series is started to provide some of the results we have encountered with various storage technologies.  As a general recommendation, all-flash storage is highly recommended with all InterSystems products to provide the lowest latency and predictable IOPS capabilities.

The first in the series was the most recently tested Netapp AFF A300 storage array.  This is middle-tier type storage array with several higher models above it.  This specific A300 model is capable of supporting a minimal configuration of only a few drives to hundreds of drives per HA pair, and also capable of being clustered with multiple controller pairs for tens of PB's of disk capacity and hundreds of thousands of IOPS or higher. 

+ 3   0 1
0

comments

1529

views

+ 3

rating

Database systems have very specific backup requirements that in enterprise deployments require forethought and planning. For database systems, the operational goal of a backup solution is to create a copy of the data in a state that is equivalent to when application is shut down gracefully.  Application consistent backups meet these requirements and Caché provides a set of APIs that facilitate the integration with external solutions to achieve this level of backup consistency.

Last comment 18 March 2019
+ 1   1 4
1455

views

+ 1

rating

Myself and the other Technology Architects often have to explain to customers and vendors Caché IO requirements and the way that Caché applications will use storage systems. The following tables are useful when explaining typical Caché IO profile and requirements for a transactional database application with customers and vendors.  The original tables were created by Mark Bolinsky.

In future posts I will be discussing more about storage IO so am also posting these tables now as a reference for those articles. 

Last comment 13 November 2016
+ 9   2 2
1440

views

+ 9

rating

Created by Daniel Kutac, Sales Engineer, InterSystems

 

Part 3. Appendix

InterSystems IRIS OAUTH classes explained

In the previous part of our series we have learned about configuring InterSystems IRIS to act as an OAUTH client as well as authorization and authentication server (by means of OpenID Connect). In this final part of our series we are going to describe classes implementing InterSystems IRIS OAuth 2.0 framework. We will also discuss use cases for selected methods of API classes.

The API classes implementing OAuth 2.0 can be separated into three different groups according to their purpose. All classes are implemented in %SYS namespace. Some of them are public (via % package), some not and should not be called by developers directly.

+ 7   1 3
0

comments

1400

views

+ 7

rating

One of the great availability and scaling features of Caché is Enterprise Cache Protocol (ECP). With consideration during application development distributed processing using ECP allows a scale out architecture for Caché applications. Application processing can scale to very high rates from a single application server to the processing power of up to 255 application servers with no application changes.

Last comment 8 December 2016
+ 9   1 6
1385

views

+ 9

rating

Globals, these magic swords for storing data, have been around for a while, but not many people can use them efficiently or know about this super-weapon altogether.

If you use globals for tasks where they truly shine, the results may be amazing, either in terms of increased performance or dramatic simplification of the overall solution (1, 2).

Globals offer a special way of storing and processing data, which is completely different from SQL tables. They were first introduced in 1966 in the M(UMPS) programming language, which was initially used in medical databases. It is still used in the same way, but has also been adopted by some other industries where reliability and high performance are top priorities: finance, trading, etc.

Later M(UMPS) evolved into Caché ObjectScript (COS). COS was developed by InterSystems as a superset of M. The original language is still accepted by developers' community and alive in a few implementations. There are several signs of activity around the web: MUMPS Google group, Mumps User's group), effective ISO Standard, etc.

Modern global based DBMS supports transactions, journaling, replication, partitioning. It means that they can be used for building modern, reliable and fast distributed systems.

Globals do not restrict you to the boundaries of the relational model. They give you the freedom of creating data structures optimized for particular tasks. For many applications reasonable use of globals can be a real silver bullet offering speeds that developers of conventional relational applications can only dream of.

Globals as a method of storing data can be used in many modern programming languages, both high- and low-level. Therefore, this article will focus specifically on globals and not the language they once came from.

Last comment 31 July 2017
+ 12   0 3
1349

views

+ 12

rating

Containers

With the launch of InterSystems IRIS Data Platform, we provide our product even  in a Docker container. But what is a container?

The fundamental container definition is that of a sandbox for a process.  

Containers are software-defined packages that have some similarities to virtual machines (VM) like for example they can be executed. 

Containers provide isolation without a full OS emulation. Containers are therefore much lighter than a VM. 

Last comment 15 November 2018
+ 8   1 8
1337

views

+ 8

rating