Question
· Dec 19, 2023

Transitioning from %𝗣𝗲𝗿𝘀𝗶𝘀𝘁𝗲𝗻𝘁 to %SerialObject: Data Purge and Global Usage Concerns

Good morning,

First of all thanks for your help and time.

We find ourselves at a crossroads in our development journey, and we're reaching out to the community for insights and guidance regarding a critical aspect of our data management strategy.

Our current challenge revolves around the persistence of data in Globals, specifically when it comes to purging. The issue at hand is that, despite purging, data stored in Globals under the current %Persistent classes remains intact. To address this, we are contemplating a shift from %Persistent to %SerialObject for each relevant class.

Our primary concern is whether the Global continues to be in use even after making this transition. We have delved into the InterSystems documentation, particularly the section on class storage updates [1], which states that the storage definition is established during the initial compilation. However, it leaves us with uncertainties about the behavior post-compilation.

As per the documentation [2], it appears that deleting the storage definition during development and recompiling the class is suggested for a clean storage structure. Our specific question is: Will changing the class's Extends from %Persistent to %SerialObject, deleting the storage definition, and recompiling suffice to ensure that upon purging, data is automatically removed, and the Global is no longer in use?

We've also sought information from the InterSystems documentation on %Library.SerialObject [3] and a topic related created by Eduard Lebedyuk in the forum [4], but we believe your experiences and insights could provide valuable clarity on this matter.

Your expertise is highly appreciated, and we look forward to your thoughts and suggestions.

Thank you,

Thanks for your replies, time, help and support.

Yone

Product version: IRIS 2020.1
Discussion (3)3
Log in or sign up to continue

If you want to clear all old data you have to use  ..%DeleteExtent(..) that you inherit from %Persistent.

Changing from %Persistent  to %SerialObject is a compiler-related action.
The compiler doesn't take care of your storage.
In this case only the access methods change, but not the underlying storage and its content 

Hi,

When you say "purging" do you mean that you are deleting the class definition? Or do you simply mean that you are deleting the data - perhaps by invoking %DeleteExtent or %KillExtent?

IRIS has three types of Classes - datatypes (not instantiable), registered (instantiable), and dynamic (not registered but instantiable). Your question seems to involve only registered types. While any data can be serialized/deserialized, we define an interface for doing so - the so-called Storage Interface. Storage Definition is a class member that can be included in just about any class but we only recognize and process it when the class is either a serial or persistent class. A serial class is a class that extends %SerialObject (meaning that %SerialObject is in its primary superclass hierarchy) and a persistent class is a class that extends %Persistent (again, %Persistent is in its primary superclass hierarchy). For these classes, the class compiler will ensure there is a fully defined Storage Definition. As with all class members, the Storage Definition has a type class. This type class is often referred to as the Storage Class - it is a class that implements the Storage Interface. The Storage Definition defines the format of the serialized object. For serial classes, this serialized object value is not directly stored anywhere. Rather, the serialized value is expected to be stored as a value embedded inside of another object. For persistent classes, an object is serialized, perhaps into multiple nodes, and stored in a global at a location defined by the IDKEY index. Persistent classes can also use globals for storing other things such as indexes, counters, metadata and so on. The set of all globals used by a persistent class is referred to as the Extent Set which is, by default, registered with the Extent Manager.

When you remove the Storage Definition from a serial or persistent class, then it is possible that any existing data serialized using the removed Storage Definition will not be able to be deserialized as it might not be compatible with a newly defined Storage Definition. We only recommend removing a Storage Definition from a class if there are no serialized/stored instances of the class present.

When you delete a persistent class, you can also delete the data and the Extent Set by adding the /deleteextent qualifier (or the 'e' flag) to the command. You can also delete the data by invoking %DeleteExtent (logical delete) or %KillExtent (physical delete - no constraints are enforced). That only deletes the data and leaves the Extent Set registered with the Extent Manager. You can delete the Extent Set from the Extent Registry by using utilities we provide. Start here: https://docs.intersystems.com/irislatest/csp/documatic/%25CSP.Documatic....

I am happy to help if you wish (I wrote much of the core serialization/deserialization/storage code) but I will need to know more about the types of classes you are using and the issues you have.

-Dan

I edited this post and made some corrections below.

I think the Community needs more information in order to help you. Data for %Persistent classes is stored in globals. Data for %SerialObject classes is also stored in globals. Properties of %Persistent classes can reference %SerialObject classes, and when the %Persistent data is saved, the referenced %SerialObject data is also stored in globals.

The decision about whether to create %Persistent or %SerialObject classes for storing data is an object modeling decision, unrelated to the need to purge data on some schedule. Changing classes from %Persistent to %SerialObject (or vice versa) for an existing application is a major design change which would probably require plenty of code rewrites. I don't think anyone would do that work just for the sake of purging. There must be something else going on.

Semi-educated guess: Maybe when you say "purging" you're talking about purging messages in interoperability productions. These messages have message headers, message bodies, and the bodies may have properties that reference %Persistent or %SerialObject objects. When you purge those messages, you have the option to purge message bodies, but if the bodies reference %Persistent objects, the referenced objects are not purged. Maybe that's what you're seeing, and maybe that's why you're thinking of switching storage definitions. There is certainly a way to make sure that any referenced objects get purged when messages are purged, documented (briefly) here. The WRC can also offer assistance.

Please tell us if my semi-educated guess is correct, or if not, please give us more information. Happy New Year!