Avoiding STORE memory errors when setting values in very large HL7 messages

Hi

We have a use case whereby very large HL7 MDM messages, which contain many OBX segments, need to be processed by a custom business process. Part of the process is to iterate over each OBX segment and set a value in the same field for each segment.

Testing with a message that is ~500MB and contains over 8000 OBX segments I've encountered <STORE> errors, which surprisingly seems to be due to using SetValueAt. It appears to get through a few thousand segments before it falls over with the memory error, which I assume is because the SetValueAt method keeps in memory each preceding OBX segment?

I've found a way to avoid this problem by having the message reference killed and re-opened after every number of iterations, as though "batched", however this does seem hacky to me and given larger messages may not hold up.

I'd greatly appreciate any alternative solutions to this. It would be great if there were a way to easily set a field value across a given number of segments, using a virtual property path like the following would be great, which I tried to no avail:

do request.SetValueAt("a value","OBX(*):ObservationValue.ContentType")

Here's the logic in question:

ClassMethod FilterOBXSegments(ByRef meshRequest As Ens.Request) As %Status
{
set obxSegmentsTotal = meshRequest.GetValueAt("OBX(*)")
set contentType = ..GetContentType(meshRequest)
$$$LOGINFO("Content type: "_contentType) if '..IsContentTypeValid(contentType) return $$$ERROR($$$GeneralError, "Did not find mapped content type")

do meshRequest.%Save()
set objectId = meshRequest.%Id()

set batchSize = 100
$$$LOGINFO("Object id is: "_objectId)
for obxLoopIter = 1:1:obxSegmentsTotal
{
    $$$LOGINFO("OBX iteration of: "_obxLoopIter)    
    do meshRequest.SetValueAt(contentType,"OBX("_obxLoopIter_"):ObservationValue.ContentType")
    if obxLoopIter = batchSize
    {
        $$$LOGINFO("Reached memory batch size, saving and re-swizzling")
        do meshRequest.%Save()
         kill meshRequest
         set meshRequest = ##class(EnsLib.HL7.Message).%OpenId(objectId)
         set batchSize = batchSize + batchSize
    }
}
quit $$$OK
}