Message data global

Question

Question

Mary George · Nov 13, 2023

#Globals #HL7 #HealthShare #InterSystems IRIS

Hello,

We noticed from the integrity logs that some of the namespaces in our integration engine are using large amount of space ( ~380GB ) for the global ^EnsHL7.Segment.

We have culling implemented for large globals like MessageHeader and MessageBody.

Is it a standard process to implement culling for ^EnsHL7.Segment global and any idea about what will be the impact of adding culling to this global?

Not sure if it is possible to implement normal culling task on this Global as I noticed this global content is not accessible via SQL functionality in Management page but only via the Globals in Management portal.

Thank you for your help.

Product version: IRIS 2021.1

Discussion (7)2

Log in or sign up to continue

score 0 · Answer 1 · 2023-11-14T13:19:41-05:00

Out of curiosity, was the system upgraded from a Cahcé based product? (Ensemble, HealthShare etc.)

Enrico

score 0 · Answer 2 · 2023-11-16T04:06:13-05:00

Mary George · Nov 16, 2023

No We have been using HealthShare 2021.1 from last year.

0 0

score 0 · Answer 3 · 2024-12-10T11:06:22-05:00

Hello!

I realized as well that this global seems to keep increasing and the purge standard utility doesn't seem to do anything. How did you solve it?

SOLVED: I found out that using this it can be purged:
d ##class(EnsLib.HL7.Message).Purge(.deleted, 7)

Many thanks,

score 0 · Answer 4 · 2024-12-12T07:04:33-05:00

Hey David.

Your solution has caught my attention. What led you to this?

I have just run this against a non-production namespace that had a 30 day retention period and was sat at around 6GB. Running this command with the "pDaysToKeep" parameter set to 30 days has managed to free up 5GB and deleted 92158 messages, where I would have expected to have not deleted anything given the retention period for the purge task and the running of this method were the same.

score 0 · Answer 5 · 2024-12-12T08:05:38-05:00

When you purge that namespace with 30 day retention period, what value of "Purge only completed sessions" (was Keep Integrity in older versions) do you use?

Have you ever run the purge without including message bodies?

score 0 · Answer 6 · 2024-12-12T08:47:42-05:00

Hey Enrico.

The value of KeepIntegrity is set to True. I don't believe we have ever run the purge without including message bodies, but it's a non-production environment so stranger things have happened.

However, I did also just try running ##class(EnsLib.HL7.Message).Purge() in our production environment with the settings matching the retention period there, and it ran for a period, freed up around 5gb and then failed over to the other mirror as running the method consumed all of the ram on the box 😅

This namespace has been around for a long time (started life at around caché/ensemble 2014) and has likely accumulated a lot of baggage. I had a similar problem when trying to run Ensemble Orphaned Messages Purge Routine

score 0 · Answer 7 · 2024-12-12T10:06:33-05:00

If/when you set KeepIntegrity to True, then ALL messages and message headers belonging to a session containing ANY message with status Created or Queued or Delivered or Suspended or Deferred are NOT deleted/purged, same goes for the headers (Ens.MessageHeader).

Since your is a test environment, it's (very?) likely having "messed" sessions that are not purged using KeepIntegrity = True.
Sometimes even production systems do have sessions that are not purged using KeepIntegrity = True.

While setting using KeepIntegrity = True is "best practice" and prevent purging "uncompleted sessions", I suggest to run a purge with KeepIntegrity = False with a (much?) longer DaysToKeep parameter, unless you do need/want to keep uncompleted sessions and messages for.....ever and ever.

For example, if you purge with KeepIntegrity = True and DaysToKeep=30, you may run a second purge with KeepIntegrity = false and DaysToKeep=90 (or longer) to purge really old messages belonging to "uncompleted" sessions.