InterSystems Data Platforms and performance - Part 5 Monitoring with SNMP

Ansible helped me solve the problem of quickly deploying Caché and application components for Data Platforms benchmarks. You can use the same tools and methodology for standing up your test labs, training systems, development or other environments. If you deploy applications at customer sites you could automate much of the deployment and ensure that system, Caché and your application are configured to your applications best practice standards.

Introduction

The field test of Caché 2016.2 has been available for quite some time and I would like to focus on one of the substantial features that is new in this version: the document data model. This model is a natural addition to the multiple ways we support for handling data including Objects, Tables and Multidimensional arrays. It makes the platform more flexible and suitable for even more use cases.

In this article I'll show you how you can easily convert between many different document formats, these to be precise:

• in: html, odt, doc, docx, rtf, sdw, txt...
• out: html, odt, docx, rtf, pdf, txt, tex, xhtml...

and many more, check out this table. We'll use LibreOffice as a conversion engine.

One of the reasons why I love Cache and Iris is that not only you can do anything you can imagine, also you can do it in a lot of different ways!!.

Imagine that you have an integration running with IRIS connected by ODBC you probably only run SQL queries but you can also create stored procedures and inside write the code to do everything you can imagine.

I'm going to give you some examples but the limit is your imagination!!

In this series of articles, I'd like to present and discuss several possible approaches toward software development with InterSystems technologies and GitLab. I will cover such topics as:

By now it's a commonplace how to implement a basic REST API in Caché and there is good documentation about it here: REST in Caché

A question that comes up from time to time is:

How can I make a parameter in my REST url optional?

Simply put, is it possible to create a URL map in Caché that maps a URL like this:

or "So you just got yelled at by your boss, for sending him an unformatted Hello World webpage"

Our previous lesson ended with us serving a Message value obtained from a Caché REST service to the client, using Angular as a runtime. While there is a lot of moving parts involved in this process, the page is not especially exciting at the moment. Before we can start adding new features, we should take a step back and review our tools.

I bet that not everyone familiar with InterSystems Caché knows about Studio extensions for working with the source code. You can actually use the Studio to create your own type of source code, compile it into interpretable (INT) and object code, and sometimes even add code completion support. That is, theoretically, you can make the Studio support any programming language that will be executed by the DBMS just as well as Caché ObjectScript. In this article, I will give you a simple example of writing programs in Caché Studio using a language that resembles JavaScript. If you are interested, please read along.

YASPE is the successor to YAPE (Yet Another pButtons Extractor). YASPE has been written from the ground up with many internal changes to allow easier maintenance and enhancements.

YASPE functions:

• Parse and chart InterSystems Caché pButtons and InterSystems IRIS SystemPerformance files for quick performance analysis of Operating System and IRIS metrics.
• Allow a deeper dive by creating ad-hoc charts and by creating charts combining the Operating System and IRIS metrics with the "Pretty Performance" option.
• The "System Overview" option saves you from searching your SystemPerformance files for system details or common configuration options.

YASPE is written in Python and is available on GitHub as source code or for Docker containers at:

• https://github.com/murrayo/yaspe

As we all well know, InterSystems IRIS has an extensive range of tools for improving the scalability of application systems. In particular, much has been done to facilitate the parallel processing of data, including the use of parallelism in SQL query processing and the most attention-grabbing feature of IRIS: sharding. However, many mature developments that started back in Caché and have been carried over into IRIS actively use the multi-model features of this DBMS, which are understood as allowing the coexistence of different data models within a single database. For example, the HIS qMS database contains both semantic relational (electronic medical records) as well as traditional relational (interaction with PACS) and hierarchical data models (laboratory data and integration with other systems). Most of the listed models are implemented using SP.ARM's qWORD tool (a mini-DBMS that is based on direct access to globals). Therefore, unfortunately, it is not possible to use the new capabilities of parallel query processing for scaling, since these queries do not use IRIS SQL access.

Meanwhile, as the size of the database grows, most of the problems inherent to large relational databases become right for non-relational ones. So, this is a major reason why we are interested in parallel data processing as one of the tools that can be used for scaling.

In this article, I would like to discuss those aspects of parallel data processing that I have been dealing with over the years when solving tasks that are rarely mentioned in discussions of Big Data. I am going to be focusing on the technological transformation of databases, or, rather, technologies for transforming databases.

Healthcare interoperability is instrumental in improving patient care, decreasing healthcare provider costs, and providing a more accurate picture to providers. However, with so many different systems, data is formatted in many different ways. There are many standards that have been created to try to solve this problem, including HL7v2, HL7v3, and CDA but each one has its drawbacks.

InterSystems Data Platform includes utilities and tools for system monitoring and alerting, however System Administrators new to solutions built on the InterSystems Data Platform (a.k.a Caché) need to know where to start and what to configure.

This guide shows the path to a minimum monitoring and alerting solution using references from online documentation and developer community posts to show you how to enable and configure the following;

1. Caché Monitor: Scans the console log and sends emails alerts.

2. System Monitor: Monitors system status and resources, generating notifications (alerts and warnings) based on fixed parameters and also tracks overall system health.

3. Health Monitor: Samples key system and user-defined metrics and compares them to user-configurable parameters and established normal values, generating notifications when samples exceed applicable or learned thresholds.

4. History Monitor: Maintains a historical database of performance and system usage metrics.

5. pButtons: Operating system and Caché metrics collection scheduled daily.

Remember this guide is a minimum configuration, the included tools are flexible and extensible so more functionality is available when needed. This guide skips through the documentation to get you up and going. You will need to dive deeper into the documentation to get the most out of the monitoring tools, in the meantime, think of this as a set of cheat sheets to get up and running.

The object and relational data models of the Caché database support three types of indexes, which are standard, bitmap, and bitslice. In addition to these three native types, developers can declare their own custom types of indexes and use them in any classes since version 2013.1. For example, iFind text indexes use that mechanism.