Question
Elisha Gould · May 26, 2020

How to truncate hl7 fields according to schema

Hi,

We are using IRIS, and have some systems that we send data to that cannot accept messages with certain fields larger than a particular size.

Is there any way to utilize the schema to truncate any fields that are larger than the entered data?

I've tried just setting the max lengths and doing the transform, but it puts in the entire string, and can't find a topic for it.

There are a large number of fields that need to be truncated, so it would be ideal to get it to respect the hl7 schema maximum lengths.

 

ie for the XPN name fields, each name may not be larger than 20 characters.  

so the name:

12345678901234567890123^PERSON^^MR

would become

12345678901234567890^PERSON^^MR
 

 

10
1 5 192 1

Replies

Hi Elisha,

The general method for handling incoming too-long fields would be the validation setting, which can send a message to the bad message handler to be treated as needed:

Validation

Validating Fields

"Test that field sizes in the message conform to the schema."

In your case if you know what fields need to be truncated it should be really easy to use a DTL to truncate those fields.

Hope that helps!

Hi,

It's a few particular down stream applications that have strict limits on its field sizes. I can't just drop the message if it doesn't conform as they still require the data, and it is valid output from the source system.

I know I could update the DTL to go through every field and truncate as needed, however that would need every field to have the limit put on it, which gets a bit excessive with the number of fields that need the limit placed on.

I was mainly trying to see if there was something already existing that was able to just use the schema before I went down that rabbit hole.

Unfortunately, I'm not aware of anything built-in that can enforce the schema - I think it would be difficult to have this as a preset option because it wouldn't be able to match all use cases. Of course, the ideal would be for the upstream to conform, but that isn't always an option.

I think this will have to be a custom solution, though maybe somebody else can weigh in with some option I've missed. Perhaps somebody has come up with something clever to leverage the schema.

For those interested. We copied part of the validation check, and updated with some options to truncate as per the schema.

It can be called with:

set tSC = ##class(User.Hl7RangeCheck).RangeChk(.pRequest)

Include EnsHL7

Class User.Hl7RangeCheck
{ ClassMethod RangeChk(ByRef pMsg As EnsLib.HL7.Message) As %Status
{
#dim pIndex as %Integer = 1
Set tSC=$$$OK , tSC2=$$$OK, tSC1=$$$OK
for 
{
set pSeg = pMsg.GetSegmentAt(pIndex)
quit:($ZSTRIP(##class(%String).LogicalToDisplay(pSeg), "<>W" ) = "") quit:(pSeg.Name = "") //$$$ERROR($$$EnsErrGeneral, "Cannot validate segment schemas without a segment name. Unable to validate segment "_$get(pSegNum)_".")
set hasSegNum = 1 - ($get(pSegNum)="")
quit:(pSeg.DocTypeCategory = "") //$$$ERROR($$$EnsErrGeneral, "Cannot validate segment schemas without a category. Unable to validate segment "_$S(hasSegNum:pSegNum_":",1:"")_pSeg.Name_".")
quit:($get($$$vaSchemaGbl(pSeg.DocTypeCategory, "SS", pSeg.Name)) = "") //$$$ERROR($$$EnsErrGeneral, "Unable to retrieve the schema to validate segment against. Unable to validate segment "_$S(hasSegNum:pSegNum_":",1:"")_pSeg.Name_".")
set schema = $$$vaSchemaGbl(pSeg.DocTypeCategory, "SS", pSeg.Name)
for fieldNum=1:1:$listlength(schema)
{
quit:(fieldNum > pSeg.Count)
set fieldDat = pSeg.GetValueAt(fieldNum, pSeg.Separators, .tSC2)
if ($$$ISERR(tSC2))
{
set tSC = $$$ADDSC(tSC, $$$ERROR($$$EnsErrGeneral, "Unable to retrieve field "_fieldNum_" from segment "_$S(hasSegNum:pSegNum_":",1:"")_pSeg.Name_"."))
quit ; can only continue with this field if no error in getting it
}
set fieldSchema = $listget(schema, fieldNum)
if '(((pSeg.Name = "MSH") || (pSeg.Name = "BHS") || (pSeg.Name = "FHS")) && ((fieldNum = 1) || (fieldNum = 2)))
{
set dat = $tr(fieldDat, pSeg.Separators, "") ; remove separators from field
}
else 
{
set dat = fieldDat
}
set data = (dat '= "") ; is anything left?
if (data)
{
//if not data structures, components, subcomponents, datatypes, and/or code tables
if ($listget(fieldSchema, 1) = "")
{ set len = $length(fieldDat)
if (($listget(fieldSchema, 4) '= "") && (len > $listget(fieldSchema, 4))) ; over length restriction
{
if '(((pSeg.Name = "MSH") || (pSeg.Name = "BHS") || (pSeg.Name = "FHS")) && (fieldNum = 2))
{
for rep=1:1:($length(fieldDat, pSeg.RS))
{
if ($length($p(fieldDat, pSeg.RS, rep)) > $listget(fieldSchema, 4))

set tSC = pMsg.SetValueAt(..Truncate(pMsg.GetValueAt(pSeg.Name_":"_fieldNum_"("_rep_")"),$listget(fieldSchema, 4)) , pSeg.Name_":"_fieldNum_"("_rep_")")
}
}
} }
}
else 
{
info
set info("SegName") = pSeg.Name , info("fieldNum") = fieldNum, info("sep") = pSeg.Separators
set (info("rep"), info("comp"), info("subComp")) = 1
set:(hasSegNum) info("SegNum") = pSegNum
set info("hasSegNum") = hasSegNum if ($listget(fieldSchema, 1) '= "") ; there is a data structure for the field
{
set dataStruct = $listget(fieldSchema, 1)
set tSC = ..ValidateFieldDataStructure(.pMsg,fieldDat, dataStruct, .info)
}
}
}
} set pIndex = pIndex+1
}
return tSC
} ClassMethod ValidateFieldDataStructure(ByRef pMsg As EnsLib.HL7.Message, field As %String, struct As %String, ByRef info) As %Status [ Internal ]
{
set tSC = $$$OK, error = 0
set category = $p(struct, ":", 1)
set dataStruct = $p(struct, ":", 2)
quit:((category = "") || (dataStruct = "")) $$$ERROR($$$EnsErrGeneral, "Cannot find schema for data structure "_struct_". Unable to validate segment "_$S($get(info("SegNum"))'="":info("SegNum")_":",1:"")_info("SegName")_" field "_info("fieldNum")_" against this structure.")
set structure = $get($$$vaSchemaGbl(category, "DT", dataStruct))
quit:('$data(structure)) $$$ERROR($$$EnsErrGeneral, "Cannot find schema for data structure "_struct_". Unable to validate segment "_$S(info("hasSegNum"):info("SegNum")_":",1:"")_info("SegName")_" field "_info("fieldNum")_" against this structure.") if '(((info("SegName") = "MSH") || (info("SegName") = "BHS") || (info("SegName") = "FHS")) && (info("fieldNum") = 2))
{
for info("rep") = 1:1:$length(field, $$$RSSEP(info("sep")))
 {
set repeat = $piece(field, $$$RSSEP(info("sep")), info("rep"))
for info("comp")=1:1:$length(repeat, $$$CSSEP(info("sep")))
{
quit:(info("comp") > $listlength(structure))
set component = $piece(repeat, $$$CSSEP(info("sep")), info("comp"))
set dat = $tr(component, $$$SSSEP(info("sep"))) ; remove separators from field
set data = (dat '= "") ; is anything left?
if (data)
{
set len = $length(component)
if (($listget($listget(structure, info("comp")),3)'="") && (len > $listget($listget(structure, info("comp")),3))) ; over length restriction

set tSC = pMsg.SetValueAt(..Truncate(component,$listget($listget(structure, info("comp")),3)), info("SegName")_":"_info("fieldNum")_"("_info("rep")_")"_"."_info("comp"))
}
}
}
}
}
Return tSC
} ClassMethod Truncate(pString As %String, pNum As %Integer) As %String
{
#dim tString As %String = pString If ($LENGTH(pString) > pNum)
{
Set tString = $EXTRACT(pString,1,pNum)
} Return tString
} }

Glad you were able to get this to work, and thanks for sharing!