NewBie's Corner Session 27 Traversing A Global with $Order Part 1

Welcome to NewBie's Corner, a weekly or biweekly post covering basic Caché Material.

Traversing A Global

Perhaps the most difficult concept in Caché/MUMPS is its Global Structure. This session and several that follow it deals with the Global Structure. However, just presenting the material will not guarantee your understanding of it. You must experiment with the data and concepts that are presented.

1 20
0 759

InterSystems IRIS currently limits classes to 999 properties.

But what to do if you need to store more data per object?

This article would answer this question (with the additional cameo of Community Python Gateway and how you can transfer wide datasets into Python).

The answer is very simple actually - InterSystems IRIS currently limits classes to 999 properties, but not to 999 primitives. The property in InterSystems IRIS can be an object with 999 properties and so on - the limit can be easily disregarded.

5 13
1 594

Hi developers!

As you probably noticed in IRIS 2021 the names of globals are random.

And if you create IRIS classes with DDL and want to be sure what global was created you probably would want to provide a name.

And indeed you can do it.

Use WITH %CLASSPARAMETER DEFAULTGLOBAL='^GLobalName' in CREATE Table to make it work. Documentation. See the example below:

3 11
0 483

Globals, these magic swords for storing data, have been around for a while, but not many people can use them efficiently or know about this super-weapon altogether.

If you use globals for tasks where they truly shine, the results may be amazing, either in terms of increased performance or dramatic simplification of the overall solution (1, 2).

Globals offer a special way of storing and processing data, which is completely different from SQL tables. They were first introduced in 1966 in the M(UMPS) programming language, which was initially used in medical databases. It is still used in the same way, but has also been adopted by some other industries where reliability and high performance are top priorities: finance, trading, etc.

Later M(UMPS) evolved into Caché ObjectScript (COS). COS was developed by InterSystems as a superset of M. The original language is still accepted by developers' community and alive in a few implementations. There are several signs of activity around the web: MUMPS Google group, Mumps User's group), effective ISO Standard, etc.

Modern global based DBMS supports transactions, journaling, replication, partitioning. It means that they can be used for building modern, reliable and fast distributed systems.

Globals do not restrict you to the boundaries of the relational model. They give you the freedom of creating data structures optimized for particular tasks. For many applications reasonable use of globals can be a real silver bullet offering speeds that developers of conventional relational applications can only dream of.

Globals as a method of storing data can be used in many modern programming languages, both high- and low-level. Therefore, this article will focus specifically on globals and not the language they once came from.

14 10
0 2.3K
Article
· Nov 23, 2017 12m read
Where is my global stored?

It's well-known that namespace global mapping helps us to write code independent on database storage details (Caché instance name, directory path). But sometimes we can face problems accessing an unsubscripted global which has subscript level mapping (SLM) defined. Most of such cases are evident and associated with administrative tasks that should be done on database level, but some of them can confuse even an experienced developer. Just to start:

4 8
0 1.3K
Article
· May 20, 2016 12m read
Collations in Caché

Order is a necessity for everyone, but not everyone understands it in the same way
(Fausto Cercignani)

Disclaimer: This article uses Russian language and Cyrillic alphabet as examples, but is relevant for anyone who uses Caché in a non-English locale.
Please note that this article refers mostly to NLS collations, which are different than SQL collations. SQL collations (such as SQLUPPER, SQLSTRING, EXACT which means no collation, TRUNCATE, etc.) are actual functions that are explicitly applied to some values, and whose results are sometimes explicitly stored in the global subscripts. When stored in subscripts, these values would naturally follow the NLS collation in effect (“SQL and NLS Collations”).

9 7
1 2.7K
Article
· Nov 11, 2019 11m read
Transactions in Global InterSystems IRIS

InterSystems IRIS supports a unique data structure, called globals, for information storage. Essentially, globals are persistent arrays with multi-level indices, having several extra capabilities—transactions, quick traversal of tree structures, and a programming language known as ObjectScript.

I'd note that for the remainder of the article, or at least the code samples, we'll assume you have familiarised yourself with the basics of globals:

6 7
1 818

The Art of Mapping Globals to Classes (4 of 3)

The forth in the trilogy, anyone a Hitchhikers Guide to the Galaxy fan?

If you are looking to breathe new life into an old MUMPS application follow these steps to map your globals to classes and expose all that beautiful data to Objects and SQL.

If the above does not sound familiar to you please start at the beginning with the following:

The Art of Mapping Globals to Classes (1 of 3)

5 7
0 2K

For some years I missed being able to offer, to everybody interested in ObjectScript, a tutorial more or less complete, to start with ObjectScript. Something that could help more and make things easier to those new developers that come to our technology... something intermediate, halfway between the common "Hello World!", that doesn't really get you further, and the "Advanced Training", that is unaffordable because of lack of time,etc.

If there were something truly helpful not only as an introduction to the ecosystem, but as a starting point, as a boost, to really start to walk into ObjectScript and move forward by yourself... wouldn't that be awesome?

7 7
0 595

In this article you will have access to the curated base of articles from the InterSystems Developer Community of the most relevant topics to learning InterSystems IRIS. Find top published articles ranked by Machine Learning, Embedded Python, JSON, API and REST Applications, Manage and Configure InterSystems Environments, Docker and Cloud, VSCode, SQL, Analytics/BI, Globals, Security, DevOps, Interoperability, Native API. Learn and Enjoy!

10 6
7 882
Article
· Dec 20, 2021 1m read
Holiday Reading: What Lies Beneath!

For those of you who might be new to IRIS, and even those who have used Cache or IRIS for some time but want to explore beyond its usually-assumed boundaries and practices, you might want to dive into this detailed exploration of the database engine that is at its heart, and discover just what you can really do with it, going way beyond what InterSystems have done with it for you.

10 5
4 437

Spoilers: Daily Integrity Checks are not only a best practice, but they also provide a snapshot of global sizes and density.

Update:
Many of the below utilities now offer a mode to estimate the size with <2% error on average with orders of magnitude improvements in performance and IO requirements. I continue to urge regular Integrity Checks, however there are situations where more urgent answers are needed.

8 5
5 1.3K
Article
· Jan 24, 2022 1m read
GlobalToJSON-Compact

I have created a package that offers a utility to load a Global into JSON object and reverse
to create a Global from this type of JSON object. Compact refers to the structure created.
Globals nodes are included with data for a fast data load.
But also the related code is quite compact.

5 3
0 320
Article
· Aug 8, 2017 1m read
Outperforming PostgreSQL and MySQL

In a previous exercise, I was able to show the power of Caché.
A medium-designed set of interdependent tables with some GB of data.
URLs cross reference over some million pages resulting in ~3 billion records

Competition was between

  • Caché
  • PostgreSQL
  • MySQL

Criteria were Speed + Storage consumption
I composed a customized loader fed over a "raw" TCP connection
Mapping the "objects" into the final table by directly writing to Global Storage.,

17 3
0 739

In the previous parts (1, 2) we talked about globals as trees. In this article, we will look at them as sparse arrays.

A sparse array - is a type of array where most values assume an identical value.

In practice, you will often see sparse arrays so huge that there is no point in occupying memory with identical elements. Therefore, it makes sense to organize sparse arrays in such a way that memory is not wasted on storing duplicate values.

In some programming languages, sparse arrays are part of the language - for example, in J, MATLAB. In other languages, there are special libraries that let you use them. For C++, those would be Eigen and the like.

Globals are good candidates for implementing sparse arrays for the following reasons:

8 3
1 1.3K
Article
· Jan 24, 2022 2m read
GlobalToJSON-Efficient

I have created a package that offers a utility to load a Global into JSON object and reverse
to create a Global from this type of JSON object. Efficient refers to the structure created.
Only Globals nodes containing data are presented in the generated JSON object.

4 2
0 311

Process-private Globals can be used as a data global in storage definition. That way, each process can have its own objects for the class with ppg storage. For example lets define a pool, which can:

  • add elements to a pool (ignoring duplicates)
  • check if an element exists in the pool

Here's the class:

4 2
0 888
Article
· Feb 7, 2022 1m read
GlobalToJSON-embeddedPython

I have created a package to export a Global into JSON object file and to re-create it by reloading from this file
embeddedPython refers to the new available technologies. It should be understood as a learning exercise of
how to handle the language interfaces. Only Global nodes containing data are presented in the generated JSON file.

1 2
0 336