Using %DynamicObject and %DynamicArray as general purpose data structures

Question

Question

Arcady Goldmint... · Aug 14, 2018

Traditional Caché Objectscript has the multi-dimensional array as its main form of complex data structure and the $order command as the main means of traversing said data structures. But newer versions of Caché ObjectScript also have data structures that are direct parallels of what languages such as JavaScript provide, in the form of %DynamicObject and %DynamicArray. These have an easy to use iterator feature via the %GetIterator method, and even a handy built-in literal syntax for constructing new objects.

Given all these nice features, are there any disadvantages to using these fancy new features? I imagine performance might not be as fast and memory usage might be a bit higher compred to plain old arrays. And of course, this makes the code not backwards compatible to older versions of Caché which did not have these features. Are there any other caveats I should be aware of?

Discussion (4)3

Log in or sign up to continue

Ben Spead · Aug 15, 2018

Eduard - I think Arcady was asking from a purely data-processing perspective. I.e. if you are already have a bunch of in-memory data and need to process/iterate on it, is it better to stick it in traditional structures and use $Order, or is it equivalent to stick it into the new structures for iterating and processing? I think it's a good question and I would also like to hear what people think :)

0 0

score 2 · Answer 1 · 2018-08-15T05:44:22-04:00

In my opinion schema-less data (NoSQL, dynamic objects, globals) does not particularly exist. All that happens when you create schema-less data structures is that data validation and enforcing schema becomes someone else problem. Usually that means application programmer. That's why if possible, consider using strict schemas - the more assumptions about the data you can guarantee, the less validation client application need to do. Also process of data cleansing, reporting and so on become much easier.

That said there are some use cases where using schema-less data is the way to go:

For new applications/mock-up/PoC, when schema is unknown
When schema is very extensive and changes often
When speed is very important and data is retrieved by key (no complex queries)

To sum up, don't use schema if creating it and maintaining it would be a considerably more time-consuming affair than creating and maintaining application/reporting level data validation.

That said I see dynamic objects available in Caché mainly as a means to convert data from/to JSON.

With InterSystems IRIS we introduced DocDB - document database, based on dynamic objects, check it out.

Also mentioning @Stefan Wittmann.

score 0 · Answer 2 · 2018-08-16T05:24:29-04:00

I agree with Benjamin - the question is more about globals/locals vs DynamicObject/DynamicArray.

@Arcady Goldmints-Orlov could you share the examples of code with globals/locals and Dynamic structures to illustrate two approaches more clear?

score 0 · Answer 3 · 2024-08-13T16:02:15-04:00

The %DynamicAbstractObject (%DAO) subclasses, including the %DynamicArray and %DynamicObject classes, provide the best way to read/send JSON representation into/out-of IRIS databases.

The %FromJSON method can create a tree of %DAO objects from external JSON representation. Then elements of %DAO data can be read or accessed using ObjectScript property syntax and using methods like %Get and %GetIterator.

Data inside an IRIS database can be used to build a tree of %DAO objects by using the ObjectScript JSON constructor syntax, property assignment statements and using methods like %Set and %Push. If all you do is store values into previously undefined elements of a %DAO object then the performance is usually quite good. Then that %DAO tree can be turned into JSON representation using the %ToJSON method.

It is not recommended that data in a tree of %DAO objects be extensively and repetitively modified/updated by the %Set method or by other ways that can modify data in a %DAO. This is especially true if you are repetitively changing the size of data elements. In such cases, the data in %DAO objects should be extracted and converted to some other optimized database structure. Often this can be Relational DataBase access if a mix of rectangular arrays provides the desirable access. It can be a %Stream if sequential access is needed. If a number of data elements are to be searched then columnar vectors can be used. IRIS can supply all these other access methods by layering them on top of ObjectScript multi-dimensional arrays. The IRIS multi-dimensional arrays (both global and local) provide the best tradeoffs across a large variety of differently ordered read/write/update/modification operations.

If you have data in a %DAO tree and you need to make small or infrequent modifications of the data then it is acceptable to leave it in %DAO representation. You can also make repetitive reads of %DAO data without needing to convert to some other representation. But otherwise, %DAO objects are best limited to moving data into and out-of JSON representation.