In this article we are going to show the results of the comparision between IRIS and Postgress when handling Astronomy data.

Introduction

Since the earliest days of human civilization we have been fascinated by the sky at night. There are so many stars! Everybody has dreamed about them and fantasized about life in other planets.

Purpose

This tool is used to generate random read Input/Output (IO) from within the database. The goal of this tool is to drive as many jobs as possible to achieve target IOPS and ensure acceptable disk response times are sustained. Results gathered from the IO tests will vary from configuration to configuration based on the IO sub-system. Before running these tests ensure corresponding operating system and storage level monitoring are configured to capture IO performance metrics for later analysis.

The InterSystems IRIS has functions that allows create DIWK digital services. A few products have the ability to transform data into wisdom, according to the following pyramid.

Sometimes you need quickly and easily import data into IRIS. For this, an IRIS import manager has been developed.

This application allows you to import JSON data and also provides a really simple interface for transferring data from MongoDB collections to IRIS globals. It has never been so easy.

Let's look at examples.

Import JSON

A More Industrial-Looking Global Storage Scheme

In the first article in this series, we looked at the entity–attribute–value (EAV) model in relational databases, and took a look at the pros and cons of storing those entities, attributes and values in tables. We learned that, despite the benefits of this approach in terms of flexibility, there are some real disadvantages, in particular a basic mismatch between the logical structure of the data and its physical storage, which causes various difficulties.

IRIS and Ensemble are designed to act as an ESB/EAI. This mean they are build to process lots of small messages.

But some times, in real life we have to use them as ETL. The down side is not that they can't do so, but it can take a long time to process millions of row at once.

To improve performance, I have created a new SQLOutboundAdaptor who only works with JDBC.

BatchSqlOutboundAdapter

Extend EnsLib.SQL.OutboundAdapter to add batch batch and fetch support on JDBC connection.

InterSystems IRIS Deployment Guide for AWS using CloudFormation template 

Introduction

In many projects I was faced with storing hierarchical data (tree) in classes.
By tree, I mean such data, where each node has a parent node — an object of the same class.
Many examples of such data can be given. For example, a catalog in the online store. Suppose that this online store sells books, in this case, the category tree might look like this:

This is a self contained class that can be run from the Intersystems Task Scheduler which records peak usage details for databases and licenses built up throughout the day and retaining 30 days history.

To schedule the task to run every hour:  

d ##class(Metrics.Task).Schedule()

You can also specify your own start time, stop time, and run interval:

d ##class(Metrics.Task).Schedule(startTime, stopTime, intervalMins)

Metrics are stored in ^Metrics in the namespace that the class resides in/is run from.

InterSystems Caché globals provide very convenient features for developers. But why are globals so fast and efficient?

Theory

Basically, the Caché database is a catalog having the same name as the database and containing the CACHE.DAT file. On Unix systems, the database can also be an ordinary disk partition.

Astronomers’ tools

5 years ago, on December 19, 2013, the ESA launched an orbital telescope called Gaia. Learn more about the Gaia mission on the official website of the European Space Agency or in the article by Vitaly Egorov (Billion pixels for a billion stars).

However, few people know what technology the agency chose for storing and processing the data collected by Gaia. Two years before the launch, in 2011, the developers were considering a number of candidates (see “Astrostatistics and Data Mining” by Luis Manuel Sarro, Laurent Eyer, William O’Mullane, Joris De Ridder, pp. 111-112):

• IBM DB2,
• PostgreSQL,
• Apache Hadoop,
• Apache Cassandra and
• InterSystems Caché (to be more precise, the Caché eXTreme Event Persistence technology).

Comparing the technologies side-by-side produced the following results (source):

The first four will probably sound familiar even to schoolchildren. But what is Caché XEP?

The following post concludes the series with a list of all databases seen in the example for the fully flexible architecture.