The Art of Mapping Globals to Classes 1 of 3

Looking to breathe new life into an old MUMPS application?  Follow these steps to map your existing globals to classes and expose all that beautiful data to Objects and SQL.

By following the simple steps in this article and the next two you will be able to map all but the craziest globals to Caché classes.  For the crazy ones I will put up a zip file of different mappings I have collected over the years.  This is NOT for new data; if you don’t already have existing global please just use the default storage.

Introduction

In the first article in this series, we’ll take a look at the entity–attribute–value (EAV) model in relational databases to see how it’s used and what it’s good for. Then we'll compare the EAV model concepts to globals.

Sometimes you have objects with an unknown number of fields, or perhaps hierarchically nested fields, for which, as a rule, you need to search.

Take, for example, an online store with diverse groups of products. Each product group has its own unique set of properties and has common properties as well.

Order is a necessity for everyone, but not everyone understands it in the same way (Fausto Cercignani)

Disclaimer: This article uses Russian language and Cyrillic alphabet as examples, but is relevant for anyone who uses Caché in a non-English locale. Please note that this article refers mostly to NLS collations, which are different than SQL collations. SQL collations (such as SQLUPPER, SQLSTRING, EXACT which means no collation, TRUNCATE, etc.) are actual functions that are explicitly applied to some values, and whose results are sometimes explicitly stored in the global subscripts. When stored in subscripts, these values would naturally follow the NLS collation in effect (“SQL and NLS Collations”).

The Art of Mapping Globals to Classes  (2 of 3)

If you are looking to breathe new life into an old MUMPS application follow these steps to map your globals to classes and expose all that beautiful data to Objects and SQL.

This example is going to cram in 4 or 5 different things beyond what was covered in Part 1 

All that is left after this is the Parent Child mapping example and then you will be on your way.

Same disclaimer:  If you can’t make heads or tails of your globals after reviewing these articles please contact the WRC and we will try to help you out:  Support@InterSystems.

Globals, these magic swords for storing data, have been around for a while, but not many people can use them efficiently or know about this super-weapon altogether.

If you use globals for tasks where they truly shine, the results may be amazing, either in terms of increased performance or dramatic simplification of the overall solution (1, 2).

Globals offer a special way of storing and processing data, which is completely different from SQL tables. They were first introduced in 1966 in the M(UMPS) programming language, which was initially used in medical databases. It is still used in the same way, but has also been adopted by some other industries where reliability and high performance are top priorities: finance, trading, etc.

Later M(UMPS) evolved into Caché ObjectScript (COS). COS was developed by InterSystems as a superset of M. The original language is still accepted by developers' community and alive in a few implementations. There are several signs of activity around the web: MUMPS Google group, Mumps User's group), effective ISO Standard, etc.

Modern global based DBMS supports transactions, journaling, replication, partitioning. It means that they can be used for building modern, reliable and fast distributed systems.

Globals do not restrict you to the boundaries of the relational model. They give you the freedom of creating data structures optimized for particular tasks. For many applications reasonable use of globals can be a real silver bullet offering speeds that developers of conventional relational applications can only dream of.

Globals as a method of storing data can be used in many modern programming languages, both high- and low-level. Therefore, this article will focus specifically on globals and not the language they once came from.

Hi:

I'm looking for some help with JSON notation for a global structure.  I need a way to represent a global structure in JSON where the global structure both has a value at a given node as well as sub-nodes.

Is there a way to pull a user name and password from the Credentials list that is kept in Ensemble? Right now I have a LDAP user that I have hard coded into my ZAUTHENTICATE, which I would like to get away from. I am not to familiar with settings Global, or calling them at least.

Thanks

Scott

Motivation

This project was thought of when I was thinking of how to let Python code deal naturally with the scalable storage and efficient retrieving mechanism given by IRIS globals, through Embedded Python.

My initial idea was to create a kind of Python dictionary implementation using globals, but soon I realized that I should deal with object abstraction first.

So, I started creating some Python classes that could wrap Python objects, storing and retrieving their data in globals, i.e., serializing and deserializing Python objects in IRIS globals.

How does it work?

Spoilers: Daily Integrity Checks are not only a best practice, but they also provide a snapshot of global sizes and density. 
Update 2024-04-16:  As of IRIS 2024.1, Many of the below utilities now offer a mode to estimate the size with <2% error on average with orders of magnitude improvements in performance and IO requirements. I continue to urge regular Integrity Checks, however there are situations where more urgent answers are needed.

• EstimatedSize^%GSIZE- Runs %GSIZE in estimation mode.   
• ##class(%Library.GlobalEdit).GetGlobalSize(directory, globalname, .allocated, .used.

Usually I recommend to set the global buffer (GB) as large as possible in order to maintain in memory more globals as possible. I don't know if this is a standard recommendation, but I want to discuss what should be the best buffer size for a Ensemble instance that only process messages. 

I mean, if the instance only runs productions that pass messages then the 90% of the global access is used for persist the messages.

The Art of Mapping Globals to Classes (4 of 3)

The forth in the trilogy, anyone a Hitchhikers Guide to the Galaxy fan?

If you are looking to breathe new life into an old MUMPS application follow these steps to map your globals to classes and expose all that beautiful data to Objects and SQL.

If the above does not sound familiar to you please start at the beginning with the following:

The Art of Mapping Globals to Classes (1 of 3)

The Art of Mapping Globals to Classes (2 of 3)

The Art of Mapping Globals to Classes (3 of 3)

This one is for you Joel!

The Art of Mapping Globals to Classes (3 of 3)

If you are looking to breathe new life into an old MUMPS application follow these steps to map your globals to classes and expose all that beautiful data to Objects and SQL.

If the above does not sound familiar to you please start at the beginning with the following:

The Art of Mapping Globals 1

The Art of Mapping Globals 2

This example is going to show you how to map a classic parent-child structure.

If a global node contains special characters, (eg : a line returns), it will be displayed like this in Portal ("System > Globals > View Global Data" panel) :

^A(1) = "this is"_$c(13,10)_"a test"

I would like to export global data to a txt file using a similar format.

I already wrote the main code (that loops on all nodes and dump them to file), the problem is how to handle special characters.
For the moment I replace them manually one by one. It works, but it's far from perfect :

set text = .

Some of our interfaces use globals for lookups, but we are currently looking at putting a groups of (document) interfaces in a separate production with a shared ‘Default Database for Routines’ to reduce code duplication.

Beginning - see Part 1.

3. Variants of structures when using globals

A structure, such as an ordered tree, has various special cases. Let's take a look at those that have practical value for working with globals.

3.1 Special case 1. One node without branches

When I describe InterSystems IRIS to more technically-minded people, I always start with how it is a multimodel DBMS at its core.

In my opinion that is its main advantage (on the DBMS side). And the data is stored only once. You just choose the access API you want to use.

• You want some sort of summary for your data? Use SQL!
• Do you want to work extensively with one record? Use objects!
• Want to access or set one value and you know the key? Use globals!

Hi,

I use cache 5.0 and access cache through OpenVMS.

The disk-space under a particular namespace has grown up in size due to cache.dat. How to recreate cache.dat to bring up more space back on disk?

In the previous parts (1, 2) we talked about globals as trees. In this article, we will look at them as sparse arrays.

A sparse array - is a type of array where most values assume an identical value.

In practice, you will often see sparse arrays so huge that there is no point in occupying memory with identical elements. Therefore, it makes sense to organize sparse arrays in such a way that memory is not wasted on storing duplicate values.

In some programming languages, sparse arrays are part of the language - for example, in J, MATLAB. In other languages, there are special libraries that let you use them. For C++, those would be Eigen and the like.

Globals are good candidates for implementing sparse arrays for the following reasons:

Hey Community,

Ready for the next contest? We are glad to invite you all to the 19th InterSystems online programming competition: 

🏆 InterSystems Globals Contest 🏆

Duration: March 21 - April 10, 2022

In prizes: $10,000

Hi, colleagues!

Consider you need to put some (less  than 1GB) data to an indexed array and do some data manipulations and calculations with it.

Should I prefer global or local for it?

Is there any general rule when local is faster than global and vice-versa?

Hi.

The message is received a few times a day.

On documentation, this ^ISCSOAP^is log to service SOAP, but why send to cconsole.log?

04/04/18-01:00:00:597 (10608) 2 ^ISCSOAP in Namespace %SYS has been active for 348 day(s).
04/04/18-01:00:00:598 (10608) 2 ^ISCSOAP in Namespace X has been active for 165 day(s).

Help-me.

Tks

It's well-known that namespace global mapping helps us to write code independent on database storage details (Caché instance name, directory path). But sometimes we can face problems accessing an unsubscripted global which has subscript level mapping (SLM) defined. Most of such cases are evident and associated with administrative tasks that should be done on database level, but some of them can confuse even an experienced developer. Just to start:

• We can't export such a global using Caché Block format (a.k.a. GOF).
• We can't kill such a global.

Maybe you can continue this list.

This post is the direct result of working with an InterSystems customer who came to me with the following problem:

SELECT COUNT(*) FROM MyCustomTable

Takes 0.005 seconds, total 2300 rows.  However:

SELECT * FROM MyCustomTable

Took minutes.  The reason for this is subtle and interesting enough for me to write a post about.  This post is lengthy, but if you scroll to the bottom I'll write a quick summary, so if you've gotten this far and think you've already read enough, scroll to the end to get the main point.  Check for the sentence in bold.

My major interest is  Working with Globals in Embedded Python.
So I checked the available official documentation.

#1 Introduction to Globals
an attempt of a generic description of what a global is. Pointing to

#2 A Closer Look at ObjectScript
But where is Embedded Python ?
Way down you see

#3 Embedded Python

3.1 Embedded Python Overview
3.1.1 Work with Globals
Great if you have never seen a Global before
Otherwise a shocking primitive example
3.2 Using Embedded Python
Last hope: >>> but there is just NOTHING visible.
This is more than just disappointing! Even IRIS Native API for Python is more detailed.

Good day, I have this issue. I have global like: ^myglobal(varX) = $LB(var1,var2). I need to delete all globals that have variable var1 in that list same without a need to iterate through all globals and checking their value.

Exact example:

- in database exist following:

^myglobal("Marek") = $LB("code2",1400)
^myglobal("Ondrej") = $LB("code1",12)
^myglobal("Nikita") = $LB("code2",32)
^myglobal("Evgeny") = $LB("code1",76)
^myglobal("Maks") = $LB("code2",99)
^myglobal("Eduard") = $LB("code1",14)
Now! How to delete all globals that have in list in the first place value "code2" please?

Hi, I work on my application and I  want to develop rules for extracting values from my database but i've any idea:

I want to order my records by the most recent timestamps values with profile_id and biometric type as keys, after comparing them with my personal program date (when the timestamp > at my fixed date : an alert message is sent to the patient)

you find here the attached screenshots for my to databases. thank's

Let's imagine I have a global like this:

^Users(12, "SETTINGS", "IsAllowed") = 1

^Users(41, "SETTINGS", "IsAllowed") = 0

^Users(52, "SETTINGS", "IsAllowed") = 1

Now I would like to check for each user whether they are allowed, therefore I'd need to iterate through the Global. How to do that? It seems that I can't use $Order here like such:

S FF = ""
For {
 S FF=$O(^Users(FF,"SETTINGS","isAllowed"))
 Q:$L(FF)
 W "User ",FF," is allowed",!
}

Is there any other way of doing this? 

Currently, we have an application running in one namespace ("Database B") that has globals and routines mapped to another database ("Database A"). After enforcing clean up on Database A, we found that 90% of the disk is free. We would like to compact Database A and release the unused space. However, we are running OpenVMS, which seems to be the issue.

For databases consisting of only globals, we are able to use ^GBLOCKCOPY; however, we need to ensure that the routines and mappings are also copied.

What would be the best recommended way to do this?

Good Morning,

I have this question for a long time. When I make an insert by SQL from an external application in the fields of type %String, if they are empty, it writes the character $c(0) in the global.

Is there a way so that if an Insert is received in a field of type% String with null value instead of the $ c (0) it leaves it empty?:

I have the class defined like this:

Class User.MiClase Extends (% Persistent,% XML.Adaptor)

and the property like this:

Property myProperty As% String;

I have tried creating a trigger or setting default values but it doesn't work.