New post
Article
· Jun 8, 2016 2m read

How to avoid truncation in HL7 messages with very large fields (32K+)

#Code Snippet #DTL #HL7 #Ensemble

DTL Transformations and GetValueAt/SetValueAt calls on HL7 messages will truncate any fields longer than 32K.  To avoid this, the methods GetFieldStreamRaw and StoreFieldStreamRaw must be used when dealing with fields that might be larger than 32K.  OBX:5 is a frequent example.  These methods have some subtleties and must be used carefully.

This can't be done by simply dragging from left to right in a DTL.  It must be done with a code action.  Also, the StoreFieldStreamRaw call must be the last edit made to the segment because the segment becomes immutable after that.

StoreFieldStreamRaw takes 3 arguments: The stream to put in the field, the VDoc path of the field in which to store the stream, and a pRemainder argument.  If the pRemainder argument isn't specified, then all fields after the field being stored are blown away.  If pRemainder is specified, then all fields after the one being stored are replaced with what's in pRemainder.

Example:
OBX|1|2|3|4|5|6|7
after StoreFieldStreamRaw(stream, "OBX:5")
OBX|1|2|3|4|<stream>
(notice fields 6 and 7 are gone).
after StoreFieldStreamRaw(stream, "OBX:5", "|six|seven"):
OBX|1|2|3|4|<stream>|six|seven

Likewise, GetFieldStreamRaw takes 3 arguments: the stream output argument, the VDoc path of the field, and the pRemainder output argument.  pRemainder will be filled with all fields after the one being extracted.

Example:
OBX|1|2|3|4|5|6|7
after GetFieldStreamRaw(stream, "OBX:5", .rem)
rem contains "|6|7"

So, to put it altogether, we want to extract the field from the source and store it in the target.  But we don't want to take the remainder from the source and store that in the target because it will revert any edits we already made to fields after the long field in the target.  Therefore, what we want to do is take the stream field from the source, but the remainder from the target, and store both of those in the target like this:

    do source.GetFieldStreamRaw(.stream, "OBX:5")
    do target.GetFieldStreamRaw(.dummy, "OBX:5", .rem)
    do target.StoreFieldStreamRaw(stream, "OBX:5", rem)

Doing it this way prevents the previous edits made to fields after OBX:5 in the target from being reverted or deleted.

Discussion (6)1
Log in or sign up to continue
James Fitzpatrick · Jun 8, 2016

Helpful post! 

Just wanted to clarify that if you attempt to access any OBX fields after OBX:5 downstream in some code (not necessarily in a DTL) you'll need to use the remainder of the GetFieldStreamRaw() method. GetValueAt() will not work in this case because of the large stream object in OBX:5.

 set tSC = tMessage.GetFieldStreamRaw(.tObx5Stream,"OBX:5",.tObxRemainder)
 set tObx11Value = $piece(tObxRemainder,"|",7)
1 0
Paul Riker · Jun 9, 2017

If I perform a transformation on my target before my code step, the code removes my transformation.

 

<assign value='"F"' property='target.{OBX:ObservationResultStatus}' action='set' />
<code>
<annotation>Convert to stream and copy to work around 32k character limitation</annotation>
<![CDATA[ do source.GetFieldStreamRaw(.stream, "OBX:5.5")
  do target.GetFieldStreamRaw(.dummy, "OBX:5.5", .rem)
  do target.StoreFieldStreamRaw(.stream, "OBX:5.5", rem)]]></code>

0 0
Jeff Morgan · Jun 9, 2017

Paul,

You are setting a target property and then calling GetFieldStreamRaw on the source.  Any changes you make to the target message before you call GetFieldStreamRaw and then the subsequent StoreFieldStreamRaw will be overwritten by the StoreFieldStreamRaw method.

Essentially your DTL example is doing the following:

Set source = "OldValue"

Set target = "NewValue"

Set target = source

"NewValue" is overwritten.

Simply add your DTL actions to manipulate the target message AFTER the StoreFieldStreamRaw calls and you should be all set.

0 0