Article
· Jan 12, 2017 5m read

DeleteHelper - A Class to Help with Deleting Referenced Persistent Classes

Following up on the topic of making sure that referenced instances are deleted when the referrer is deleted –

This topic was discussed in this "Ensemble Orphaned Messages" post, and was also touched in my post with a utility that can help verify all related data was purged.

What is the DeleteHelper

In the "Ensemble Orphaned Messages" one, Suriya suggests to use an %OnDelete() method in your related classes (also as documented), and provides a sample for one.

In this post I want to share a class that could help with this.

You can find it here on GitHub.

This is based on some code that evolved through being used by several sites, and then adapted for this public share. You are of course welcome to fix any issues you find, or add any enhancements.

The class includes a Code Generator %OnDelete method, which generates code at compile-time that should take care of deleting persistent objects referenced by the instance being deleted.

[Note in this case the %OnDelete approach was taken, but in general there are various ways of "telling" a class to delete "other objects" when an instance is deleted – there are some "cascading" mechanisms, for Relationships and for Foreign Keys, as well as using an SQL Trigger (which nowadays also will be called upon object events as well)].

What you'd need to do is add this class as another Super class for your class.

There is also an accompanying class that should help with adding this as a Super class for several classes at once (useful for example for classes that were generated by the SOAP Wizard, which might be quite a few).

Note the use-case in which this class is brought here is Ensemble Message (and Business Process Context) purging, but this is relevant also for "pure" Caché scenarios.

Example

There is more information in the class reference of the 2 classes involved, including usage examples and sample cases. From there:

Here is an example of a class and what it's generated %OnDelete will look like -

A snippet from the Class Definition:

Property Name As %String(MAXLEN = "", XMLNAME = "Name") [ Required ];

Property SSN As %String(MAXLEN = "", XMLNAME = "SSN") [ Required ];

Property DOB As %Date(XMLNAME = "DOB");

Property Home As ITest.Proxy.s0.Address(XMLNAME = "Home");

Property Office As ITest.Proxy.s0.Address(XMLNAME = "Office");

Property Spouse As ITest.Proxy.s0.Person(XMLNAME = "Spouse");

(Assuming the Address and Person classes are Persistent)

And this is the generated %OnDelete (from the generated INT routine):

%OnDelete(oid) public { 
Set status = 1 
Set obj = ..%Open(oid,,.status) 
If ('status) { 
 Set errorCode = $System.Status.GetErrorCodes(status) 
  If errorCode [ 5809 { 
   Quit 1 
  } Else { 
   Quit status 
  } 
} 
If $IsObject(obj.Home) { 
 Set delStatus = obj.Home.%DeleteId(obj.Home.%Id()) 
  If ('delStatus) { 
   Set errorCode = $System.Status.GetErrorCodes(delStatus) 
   If errorCode [ 5810 { 
  } Else { 
   Set status=$select(+status:delStatus,1:$$AppendStatus^%occSystem(status,delStatus)) 
  } 
 } 
} 
If $IsObject(obj.Office) { 
 Set delStatus = obj.Office.%DeleteId(obj.Office.%Id()) 
 If ('delStatus) { 
  Set errorCode = $System.Status.GetErrorCodes(delStatus) 
  If errorCode [ 5810 { 
  } Else { 
   Set status=$select(+status:delStatus,1:$$AppendStatus^%occSystem(status,delStatus)) 
  } 
 } 
} 
If $IsObject(obj.Spouse) { 
 Set delStatus = obj.Spouse.%DeleteId(obj.Spouse.%Id()) 
 If ('delStatus) { 
  Set errorCode = $System.Status.GetErrorCodes(delStatus) 
  If errorCode [ 5810 { 
  } Else { 
   Set status=$select(+status:delStatus,1:$$AppendStatus^%occSystem(status,delStatus)) 
  } 
 } 
} 
Quit status }

(The code uses some macros, primarily for Status handling, that are displayed above, as this is INT code, in their translated manner)

Here is another sample, this time with a list collection -

The class definition snippet:

Property GetListByNameResult As list Of ITest.Proxy.s0.PersonIdentification;

And the generated code snippet:

If $IsObject(obj.GetListByNameResult) { 
 Set key="" 
 Set item = obj.GetListByNameResult.GetNext(.key) 
 While key'="" { 
  If $IsObject(item) { 
   Set delStatus = item.%DeleteId(item.%Id()) 
   If ('delStatus) { 
    Set errorCode = $System.Status.GetErrorCodes(delStatus) 
    If errorCode [ 5810 { 
    } Else { 
     Set status=$select(+status:delStatus,1:$$AppendStatus^%occSystem(status,delStatus)) 
    } 
   } 
  } 
 Set item = obj.GetListByNameResult.GetNext(.key) 
} 

 

Upcoming New 2017.1 Planned Related Feature

Version 2017.1 (available now as a Field Test) includes a new feature (internal reference – MXT2018) that allows you to choose, as part of the XML Schema Wizard, or the SOAP Wizard, that your generated classes will include %OnDelete methods to do exactly what was discussed above:

And this is a sample method that was created in the Wizard-generated class (not via the DeleteHelper):

/// The %OnDelete method is generated in order to cascade deletes of an XML tree.
/// If this class is modified, then the %OnDelete method will need to change to accommodate changed persistent object references.
ClassMethod %OnDelete(deleteOid As %ObjectIdentity) As %Status [ Private, ProcedureBlock = 1, ServerOnly = 1 ]
{
 Set oref=..%Open(deleteOid,,.sc) If $$$ISERR(sc) Quit sc
 Set oid=oref.HomeGetObject()
 If oid'="" Set sc=$$$ADDSC(sc,##class(WSTest.Proxy.s0.Address).%Delete(oid))
 Set oid=oref.OfficeGetObject()
 If oid'="" Set sc=$$$ADDSC(sc,##class(WSTest.Proxy.s0.Address).%Delete(oid))
 Set oid=oref.SpouseGetObject()
 If oid'="" Set sc=$$$ADDSC(sc,##class(WSTest.Proxy.s0.Person).%Delete(oid))
 Quit sc
}

[The method generated by the DeleteHelper is similar, yet slightly different. See sample in Class Reference and above]

So when 2017.1 is released you can definitely use this approach.

A few notes though:

  • See the comment in the generated method: "If this class is modified then the %OnDelete method will need to change to accommodate changed persistent object references", with the DeleteHelper as it is an inherited generator-method and changes should be reflected automatically in the re-generated method. Of course if you run the Wizard (XML Schema or SOAP) then the classes will also be regenerated with "new" appropriate methods.
  • This will add this to the Wizard XML/WSDL-generated data-related classes, but not for example for the Message classes (even if the Request/Response classes were created by the same Wizard, at least as it is now). The DeleteHelper could help with that.
  • And obviously this new feature will not cover manually developed classes, but the DeleteHelper could.

A quick disclaimer regarding the availability of this upcoming feature - as is always true with regards to Field Tests and planned features - changes to the contents of the 2017.1 version, or to it's release itself, could occur (possibly without notice). So eventually you might not find this feature in 2017.1, or it might behave different than mentioned above.

Discussion (10)1
Log in or sign up to continue

Hello Netanel, 

Nice framework you've provided, thanks.

I am wondering if it would be of any help to my case: I have used the Studio SOAP wizard to generate some webservice client classes and amongst those classes:

Class RMH.SOAP.s0.Output Extends (%Persistent, %XML.Adaptor) [ ProcedureBlock, SqlTableName = _Output ] {
...
Property Value As %GlobalCharacterStream(XMLNAME = "Value");

And the caller looks like

Class RMH.SOAP.SoapTreeSoap Extends %SOAP.WebClient [ ProcedureBlock ] {
...
Method Run...(Tree As %String, Inputs As %String, Debug As %Integer) As RMH.SOAP.s0.Output 

We do run a daily purge in all of our productions, but I noticed that some globals aren't being taken into consideration. The global related to the class above is occupying 8GB worth of data and its size isn't reduced after the daily purge take place (it uses the ensemble base class Ens.Util.Tasks.PurgeActivityData)

When I checked the global contents, I could definitely spot very old data... so that reinforces my assumption. Could the delete helper take care of this scenario? Or should I develop custom classes for the purge process?

Any help is appreciated.

Many thanks!

Hi Murillo,

Happy you found interest in this utility.

The situation you described is in fact the classic scenario I had in mind when I built this utility, so it could definitely help.

Here are a few clarifications though –

  1. Ensemble Version

As I mentioned in my original post above, in “relatively” newer versions of Ensemble (since 2017.1), and definitely in InterSystems IRIS, within the SOAP Wizard there is a checkbox that if you check, the auto-generated classes will include a similar %OnDelete method to the one generated by my utility. And therefore the built-in Purge Task will take care of deleting this data (going ahead that is… see more about this in the next comment).

So in these versions, for this specific scenario (SOAP Wizard generated classes, as well the XML Schema Wizard) you don’t have to use my utility. For other cases, where you have custom Persistent classes with inter-relations, my utility would still be helpful.

 

  1. Looking Ahead vs. Looking Back

This %OnDelete method (generated by my utility, or by checking the checkbox in the Wizard mentioned above) takes care of deleting these objects when the Message Body is deleted. But if the Message Body has already been deleted (which seems like your case), this would not help (directly) retroactively.

It would still be recommended to add it, for Purges happening going ahead (and for another consideration mentioned soon), but just by adding this, the old data accumulated will not get auto-magically deleted.

If indeed you have this old data (of objects pointed to by older, previously purged Message Bodies) accumulated then you’ll have to take care of deleting it programmatically.

I won’t get into too many details about how to do this, but here are a few general words of background –

The general structure of Ensemble Messages (in this context) is:

Message Header -> Message Body [ -> Possible other Persistent objects referenced at; and potentially other levels of more object hierarchy]

The built-in Purge Task will delete the Message Header along with the Message Bodies (assuming you checked the “Include Bodies” checkbox in the Purge Task definition, which in 99.99% cases I’ve seen should be checked). But it will not delete other Persistent objects the Message Body was referencing, at least not just “out-of-the-box” (that’s where the %OnDelete method comes into play).

So if your Message Headers and Bodies referring to the “other objects” were already deleted, in order to delete the “not referenced anymore” objects, you’d need to find the last ID of these objects (at the top most level of the object hierarchy) that still has a reference, and delete those under that ID (assuming an accumulative running integer ID for these objects).

I believe the WRC (InterSystems Support – Worldwide Response Center) have a utility built for these kinds of cases (at least scenarios), so I recommend reaching out to the WRC for their assistance, if this is your situation.

Note that if you added my %OnDelete to your classes, then finding just the top-level object in the hierarchy, the one(s) that the Message Body referenced, would be enough to delete, since the %OnDelete will take care of deleting the rest of “the tree” of objects. That’s why I said above that it won’t help the situation of this “lingering” data all by itself, but it could help.

 

  1. Future Proofing

As you can see finding yourself in the situation of having accumulated old data that is not too straightforward to delete, is something you’d very much want to avoid. That is why I also built another utility that helps in validating, during development/testing stages, that there are no such “Purge leaks” in your interfaces.

In addition this tool also helps at providing an estimate as to how much diskspace (for database growth and journaling) your interface will require.

You can check this out here.

 

Hope this helps.

Important Note -

Some classes - for example the built-in Business Process class (Ens.BusinessProcessBPL) - have already specific implementation of an %OnDelete method, and hence adding the DeleteHelper as a SuperClass would override the "default" %OnDelete that class might have (depending on the order of Inheritance defined in the class).

So for example a BPL-based Business Process would have this %OnDelete -

ClassMethod %OnDelete(oid As %ObjectIdentity) As %Status
{
    Set tId=$$$oidPrimary(oid)
    &sql(SELECT %Context INTO :tContext FROM Ens.BusinessProcessBPL WHERE %ID = :tId)
    If 'SQLCODE {
        &sql(DELETE from Ens_BP.Context where %ID = :tContext)
        &sql(DELETE from Ens_BP.Thread where %Process = :tId)
    }
    Quit $$$OK
}

Note this takes care of deleting the related Context and Thread data when deleting the Business Process instance. This is the desired behavior. Without this lots of "left-over" data could remain.

Note also that in fact the "AddHelper" class that assists in adding the DeleteHelper as a Super Class to various classes - does not add the DeleteHelper to Business Process classes -

// Will ignore classes that are Business Hosts (i.e. Business Service, Business Process or Business Operation)
// As they do not require this handling
If  $ClassMethod(className,"%IsA","Ens.Host")||$ClassMethod(className,"%IsA","Ens.Rule.Definition")  {
Continue
}


But if you add the DeleteHelper manually to your classes please take caution to see if the class already has an %OnDelete() method and how you'd like to handle that.

A good option is to simply add a call to the original %OnDelete in the newly generated %OnDelete() method calling ##super().

Thanks to @Suriya Narayanan Suriya Narayanan Vadivel Murugan for helping a customer diagnose this situation!

I raised a different question but go no reply. 

I don't understand how you actually use this? 

Have a class like 

Class Messages.NonHL7.PMEP.Outbound.PathologyResult Extends (Ens.Request, SRFT.Utility.DeleteHelper.OnDeleteSuper)

contains 


Property freeTextLine As list Of Messages.NonHL7.PMEP.Outbound.FreeTextLine;

How do you actually get the ondelete to work or generate? What should you see?

i added  

ClassMethod %OnDelete(oid As %ObjectIdentity) As %Status
{
        Do ##super()
}

but did not delete the id 2211269686 listed in the list. 

Can an implementation example be provided to this? The only step really to use this yourself is "add this class as another Super class for your class." but i don't grasp what we should see if we do this and if indeed this is correct

Tried with just Class Messages.NonHL7.PMEP.Outbound.PathologyResult Extends (Ens.Request, SRFT.Utility.DeleteHelper.OnDeleteSuper) without defining an %onDelete as the guide would suggest but again have id 2211269800 which did not delete 

id 2211269834 of  Messages.NonHL7.PMEP.Outbound.Location also did not delete

Hi Mark,

Indeed all you need to do with your class/es is extend it/them from the DeleteHelper class. No need to add your own %OnDelete() method (in fact that might interfere with the method generation).

From your comment I cannot tell clearly enough what exactly did not get deleted (or of course why).

Per your question, in general what you can expect to see, as per the example I provided, is that the %OnDelete method (label) gets generated in the generated routine (INT).

If you want you can elaborate a little further about the full structure of your classes (you mention PathologyResult and FreeTextLine, but then later also Location which is unclear how it is related), and data (you just mention some ID numbers, but not really who's who), including message header info, and what you see when you purge the data (messages).

(if you want you can even share the actual classes, if you would prefer, also privately).

Maybe then I can be of more assistance.

Thanks I now understand. 

I was not looking at the .int 

Also was testing using SQL trigger delete i.e. from sql Delete from (field) where ID = id. This delete helper uses the objectscript deleteID which is perfectly good as this is what the main Ens.Util.Tasks.Purge will use. 

Great tool as this was for one message object with 10 embedded xml objects and 2 lists of xml objects, added some manually on before and it is the case this with using this plus doing a test on a single object first is a lot faster than the risks of manual typos for doing this manually. The SQL trigger on delete will not be required in my usecases