New post

#InterSystems Data Platform Blog

11

This group consists of fairly long technical posts regarding InterSystems Technology features and experience.

Every post in this group is a story of a developer or engineer who wants to share his experience related to InterSystems Data Platform with you.

Every post in this group contains either snippets of code or even ready to use projects with sources on Github or with working demos of InterSystems Technology.

Every post is reviewed by InterSystems specialists through a special reviewing procedure.

Subscribe to the group and you'll get weekly posts with essentials of experience on developing, testing, deploying, maintaining solutions that utilize InterSystems Data Platform technology: multimodel DBMS Caché,  Platform for connected applications Ensemble, analytics technology DeepSee and unstructured data management technology  - iKnow.

Yes, you can contribute to this group. If you want to share your experience, promote interesting development tool or a framework to InterSystems Developer Community in the form of a long article reviewed by InterSystems engineers - please contact me.

Evgeny Shvarov,

InterSystems Community Manager

Released with no formal announcement in IRIS preview release 2019.4 is the /api/monitor service exposing IRIS metrics in Prometheus format. Big news for anyone wanting to use IRIS metrics as part of their monitoring and alerting solution. The API is a component of the new IRIS System Alerting and Monitoring (SAM) solution that will be released in an upcoming version of IRIS.

11 2
7 2K

The following steps show you how to display a sample list of metrics available from the /api/monitor service.

In the last post, I gave an overview of the service that exposes IRIS metrics in Prometheus format. The post shows how to set up and run IRIS preview release 2019.4 in a container and then list the metrics.

This post assumes you have Docker installed. If not, go and do that now for your platform :)

18 10
6 1.4K

** Revised Feb-12, 2018

While this article is about InterSystems IRIS, it also applies to Caché, Ensemble, and HealthShare distributions.

Introduction

Memory is managed in pages. The default page size is 4KB on Linux systems. Red Hat Enterprise Linux 6, SUSE Linux Enterprise Server 11, and Oracle Linux 6 introduced a method to provide an increased page size in 2MB or 1GB sizes depending on system configuration know as HugePages.

At first HugePages required to be assigned at boot time, and if not managed or calculated appropriately could result in wasted resources. As a result various Linux distributions introduced Transparent HugePages with the 2.6.38 kernel as enabled by default. This was meant as a means to automate creating, managing, and using HugePages. Prior kernel versions may have this feature as well however may not be marked as [always] and potentially set to [madvise].

Transparent Huge Pages (THP) is a Linux memory management system that reduces the overhead of Translation Lookaside Buffer (TLB) lookups on machines with large amounts of memory by using larger memory pages. However in current Linux releases THP can only map individual process heap and stack space.

6 9
5 5.3K

Your application is deployed and everything is running fine. Great, hi-five! Then out of the blue the phone starts to ring off the hook – it’s users complaining that the application is sometimes ‘slow’. But what does that mean? Sometimes? What tools do you have and what statistics should you be looking at to find and resolve this slowness? Is your system infrastructure up to the task of the user load? What infrastructure design questions should you have asked before you went into production? How can you capacity plan for new hardware with confidence and without over-spec'ing? How can you stop the phone ringing? How could you have stopped it ringing in the first place?

23 13
5 4.5K

Myself and the other Technology Architects often have to explain to customers and vendors Caché IO requirements and the way that Caché applications will use storage systems. The following tables are useful when explaining typical Caché IO profile and requirements for a transactional database application with customers and vendors. The original tables were created by Mark Bolinsky.

In future posts I will be discussing more about storage IO so am also posting these tables now as a reference for those articles.

9 7
2 2.9K

Created by Daniel Kutac, Sales Engineer, InterSystems

Warning: if you get confused by URLs used: the original series used screens from machine called dk-gs2016. The new screenshots are taken from a different machine. You can safely treat url WIN-U9J96QBJSAG as if it was dk-gs2016.

Part 2. Authorization server, OpenID Connect server

9 12
3 5.4K

This post provides useful links and an overview of best practice configuration for low latency storage IO by creating LVM Physical Extent (PE) stripes for database disks on InterSystems Data Platforms; InterSystems IRIS, Caché, and Ensemble.

5 4
1 4.7K

This week I am going to look at CPU, one of the primary hardware food groups :) A customer asked me to advise on the following scenario; Their production servers are approaching end of life and its time for a hardware refresh. They are also thinking of consolidating servers by virtualising and want to right-size capacity either bare-metal or virtualized. Today we will look at CPU, in later posts I will explain the approach for right-sizing other key food groups - memory and IO.

So the questions are:

14 10
2 5K

InterSystems Data Platform includes utilities and tools for system monitoring and alerting, however System Administrators new to solutions built on the InterSystems Data Platform (a.k.a Caché) need to know where to start and what to configure.

This guide shows the path to a minimum monitoring and alerting solution using references from online documentation and developer community posts to show you how to enable and configure the following;

  1. Caché Monitor: Scans the console log and sends emails alerts.

  2. System Monitor: Monitors system status and resources, generating notifications (alerts and warnings) based on fixed parameters and also tracks overall system health.

  3. Health Monitor: Samples key system and user-defined metrics and compares them to user-configurable parameters and established normal values, generating notifications when samples exceed applicable or learned thresholds.

  4. History Monitor: Maintains a historical database of performance and system usage metrics.

  5. pButtons: Operating system and Caché metrics collection scheduled daily.

Remember this guide is a minimum configuration, the included tools are flexible and extensible so more functionality is available when needed. This guide skips through the documentation to get you up and going. You will need to dive deeper into the documentation to get the most out of the monitoring tools, in the meantime, think of this as a set of cheat sheets to get up and running.

13 1
8 2.2K

++ Update: August 1, 2018

The use of the InterSystems Virtual IP (VIP) address built-in to Caché database mirroring has certain limitations. In particular, it can only be used when mirror members reside the same network subnet. When multiple data centers are used, network subnets are not often “stretched” beyond the physical data center due to added network complexity (more detailed discussion here). For similar reasons, Virtual IP is often not usable when the database is hosted in the cloud.

Network traffic management appliances such as load balancers (physical or virtual) can be used to achieve the same level of transparency, presenting a single address to the client applications or devices. The network traffic manager automatically redirects clients to the current mirror primary’s real IP address. The automation is intended to meet the needs of both HA failover and DR promotion following a disaster.

15 12
5 6.4K

In the last post we scheduled 24-hour collections of performance metrics using pButtons. In this post we are going to be looking at a few of the key metrics that are being collected and how they relate to the underlying system hardware. We will also start to explore the relationship between Caché (or any of the InterSystems Data Platforms) metrics and system metrics. And how you can use these metrics to understand the daily beat rate of your systems and diagnose performance problems.

19 10
2 4K

Hi, Community!

Here is the digest of the best questions on InterSystems Data Platform in 2017.

In 2017 1,158 questions and here are Top 20 on most viewed, most voted and most commented.

Most viewed

WebSockets vs Long Polling vs Short Polling?, by Sean Connelly 2639

GitLab CI examples?, by Eduard Lebedyuk 2172

WebMethod - Issue with encoding (special character), by Murillo Braga 755

SHA256 Signing with RSA PSS padding, by Stefan Cronje 752

New Programming Language - Discussion, by Maks Atygaev 672

Managing UTF-8 characters on the database with a REST application, by Amir Samary 602

Decoding Base64 PDF File, by John Flippance 519

ERROR #822: Access Denied, by Natasa Klenovsek Arh 506

Convert to JSON string, by Bapu Hirave 501

XML to Json conversion, by Yuriy Borokhov 497

Authentication with REST, by Pasi Leino 496

Notepad++ Language Pack for COS?, by Benjamin Schlanger 467

Cache DB in Docker Container, by Natasa Klenovsek Arh 448

Globals vs Locals: What Is Faster?, by Evgeny Shvarov 441

Configuring IIS Web Server to Serve RESTful Web Services, by Kenneth Poindexter 432

ERROR #5002: Cache error in Ensemble Soap Service, by Tom Philippi 426

Output 9876543210 without using numbers, by Eduard Lebedyuk 426

SQL with 2 columns in where condition, by Natasa Klenovsek Arh 425

Scripted/Automated Production Deployment?, by Greg Arnold 422

SQL or Cache Function?, by Roger Beeman 421

0 0
0 446

A request came from a customer to estimate how long it would take to encrypt a database with cvencrypt utility.

This question is a little bit like how long is a piece of string — it depends. But its an interesting question. The answer primarily depends on the performance of CPU and storage on the target platform the customer is using, so the answer is more about coming up with a simple methodology that can be used to benchmark the CPU and storage while running cvencrypt.

6 0
1 1K

Hyper-Converged Infrastructure (HCI) solutions have been gaining traction for the last few years with the number of deployments now increasing rapidly. IT decision makers are considering HCI when scoping new deployments or hardware refreshes especially for applications already virtualised on VMware. Reasons for choosing HCI include; dealing with a single vendor, validated interoperability between all hardware and software components, high performance especially IO, simple scalability by addition of hosts, simplified deployment and simplified management.

9 7
1 3.6K
Article
· May 25, 2017 2m read
The Interns are Coming!

The Data Platforms department here at InterSystems is gearing up for this year's crop of interns, and I for one am very excited to meet them all next week!

We've got folks from top technical colleges with diverse specialties from hard core engineers to pure computer scientists to mathematicians to business professionals. They come from countries around the world like Vietnam, China, and Finland and they all come with impressive backgrounds. We're sure they will do very well this summer.

6 0
0 509

Ansible helped me solve the problem of quickly deploying Caché and application components for Data Platforms benchmarks. You can use the same tools and methodology for standing up your test labs, training systems, development or other environments. If you deploy applications at customer sites you could automate much of the deployment and ensure that system, Caché and your application are configured to your applications best practice standards.

13 4
0 2.6K

One of the great availability and scaling features of Caché is Enterprise Cache Protocol (ECP). With consideration during application development distributed processing using ECP allows a scale out architecture for Caché applications. Application processing can scale to very high rates from a single application server to the processing power of up to 255 application servers with no application changes.

9 6
2 3.2K

In this post I would like to talk about the syslog table. I will cover what it is, how you look at it, what the entries really are, and why it may be important to you. The syslog table can contain important diagnostic information. If your system is having any problems, it is important to understand how to look at this table and what information is contained there.

15 1
1 2.6K

A short post for now to answer a question that came up. In post two of this series I included graphs of performance data extracted from pButtons. I was asked off-line if there is a quicker way than cut/paste to extract metrics for mgstat etc from a pButtons .html file for easy charting in Excel.

See: - Part 2 - Looking at the metrics we collected

7 2
0 1.5K

This article contains the tutorial document for a Global Summit academy session on Text Categorization and provides a helpful starting point to learn about Text Categorization and how iKnow can help you to implement Text Categorization models. This document was originally prepared by Kerry Kirkham and Max Vershinin and should work based on the sample data provided in the SAMPLES namespace.

5 0
1 689
Article
· Sep 30, 2016 1m read
ECP Magic

I saw someone recently refer to ECP as magic. It certainly seems so, and there is a lot of very clever engineering to make it work. But the following sequence of diagrams is a simple view of how data is retrieved and used across a distributed architecture.

For more more on ECP including capacity planning follow this link: Data Platforms and Performance - Part 7 ECP for performance, scalability and availability

10 0
0 1.2K

++Update: August 2, 2018

This article provides a reference architecture as a sample for providing robust performing and highly available applications based on InterSystems Technologies that are applicable to Caché, Ensemble, HealthShare, TrakCare, and associated embedded technologies such as DeepSee, iKnow, Zen and Zen Mojo.

Azure has two different deployment models for creating and working with resources: Azure Classic and Azure Resource Manager. The information detailed in this article is based on the Azure Resource Manager model (ARM).

12 4
0 12.1K

Introduction

The field test of Caché 2016.2 has been available for quite some time and I would like to focus on one of the substantial features that is new in this version: the document data model. This model is a natural addition to the multiple ways we support for handling data including Objects, Tables and Multidimensional arrays. It makes the platform more flexible and suitable for even more use cases.

13 12
0 2.7K