Unable to Parse CSV from Business Service to Business Process

Question

Question

Stuart Byrne · May 23, 2017

Dear All,

I’m having trouble creating the following business service. It’s intention is to pick up an CSV via FTP and pass it to a Business process that transforms it to a HL7 message.

I have created a record map for the CSV file, which I am trying to call in the business service and parse it into a new message class, which can be transformed in the business process.

Please could you advise:

Business Service:

/// Collects CSV from Epic FTP, looking for ENH patient
/// antiobitic sensitivities to be passed on to ENH Infection control
/// Patch 176
Class CUH.Serv.EpicFTPFilePickup Extends (Ens.BusinessService, CUH.Recordmap.ENHSens2HL7)
{

Parameter ADAPTER = "EnsLib.FTP.InboundAdapter";

// Adds Option to state which Business process is the target of the Business service

Parameter SETTINGS As %String = "TargetConfigName";

// Declares that the TargetConfigName as a %String

Property TargetConfigName As %String(MAXLEN = 999);

// previous pInput: pInput As %RegisteredObject & Output pOutput As %RegisteredObject

Method OnProcessInput(pInput As %IO.DeviceStream, Output pOutput As %RegisteredObject) As %Status
{
                //Create Request Message
                //set trequest = ##class(CUH.Recordmap.ENHSens2HL7)
               
                do ##class(CUH.Recordmap.PutObject).%New()
                //Send Request to CUH.Proc.ICENETROUTER
               
                set tgo = ..SendRequestAsync(..TargetConfigName,trequest)

                Quit $$$ERROR($$$NotImplemented)
}

}

Record map (using CSV mapping tool).

Class CUH.Recordmap.ENHSens2HL7 Extends EnsLib.RecordMap.RecordMap
{

Parameter OBJECTNAME = "CUH.Recordmap.ENHSens2HL7.Record";

/// Method to write a record to a stream or device. This method will be generated based on the
/// structure of the RecordMap. For testing purposes, a file name may be passed in the pStream argument.
ClassMethod PutObject(pStream As %IO.DeviceStream, pObject As %RegisteredObject, pFlush As %Boolean = 1, ByRef pPadArray As %String) As %Status
{
                Try {
                                Set tStatus = $$$OK
                                If '$IsObject(pStream) {
                                                Set tFilename = pStream
                                                Set pStream = ##class(%IO.FileStream).%New()
                                                Do pStream.Open(tFilename,"WAE",,"UTF-8", .tStatus)
                                                If $$$ISERR(tStatus) Quit
                                }
                                Set tHasTopFields = $parameter(pObject,"INCLUDETOPFIELDS")
                                Set pStream.CharEncoding = "UTF-8"

                                Set tStatus = ..PutRecord(pStream, pObject, tHasTopFields, .pPadArray)
                                If $$$ISERR(tStatus) Quit
                                If pFlush Do pStream.Flush(.tStatus)
                                If $$$ISERR(tStatus) Quit
                }
                Catch ex {
                                Set tStatus = ex.AsStatus()
                }
                Quit tStatus
}

/// Method to retrieve a record from a stream or device. This method will be generated based on the
/// structure of the RecordMap. For testing purposes, a file name may be passed in the pStream argument.
ClassMethod GetObject(pStream As %IO.DeviceStream, Output pObject As %RegisteredObject, pTimeout As %Numeric = -1, ByRef pLookAhead As %String = "", pLabelFound As %Boolean = 0) As %Status
{
                Try {
                                Set tStatus = $$$OK
                                Set pObject = $$$NULLOREF
                                If '$IsObject(pStream) {
                                                Set tFilename = pStream
                                                Set pStream = ##class(%IO.FileStream).%New()
                                                Do pStream.Open(tFilename,,pTimeout,"UTF-8", .tStatus)
                                                If $$$ISERR(tStatus) Quit
                                }
                                Set tObjectName = $parameter($this,"OBJECTNAME")
                                Set tObject = $classmethod(tObjectName,"%New")
                                If '$IsObject(tObject) {
                                                Set tStatus = $$$ERROR($$$CannotCreateObject,tObjectName)
                                                Quit
                                }
                                Set tObject.%Source = pStream.Name
                                Set tGetTopFields = $parameter(tObject,"INCLUDETOPFIELDS")
                                If pStream.AtEnd {
                                                Set tStatus = $$$ERROR($$$EnsRecordMapErrStreamAtEnd, pStream.Name)
                                                Quit
                                }
                                Set pStream.CharEncoding = "UTF-8"
                               
                                do pStream.WriteLine(1)

                                #; Parse incoming stream
                                Set tStatus = ..GetRecord(pStream, tObject, tGetTopFields, pTimeout, pLookAhead, pLabelFound)
                                If $$$ISERR(tStatus) Quit
                }
                Catch ex {
                                Set tStatus = ex.AsStatus()
                }
                If $$$ISERR(tStatus) { Set pObject = $$$NULLOREF }
                Else { Set pObject = tObject }
                Quit tStatus
}

/// Internal helper method to output object to a stream
ClassMethod PutRecord(pStream As %IO.DeviceStream, pObject As %RegisteredObject, pHasTopFields As %Boolean, ByRef pPadArray As %String) As %Status [ Internal, Private ]
{
                Set tStatus = $$$OK
                Do pStream.Write($select(pHasTopFields: ..Escape(##class(CUH.Recordmap.ENHSens2HL7.Record).SpecimenIDLogicalToDisplay(pObject.SpecimenID), $lb($char(44)),"quote",$char(34)), 1: ""),0)
                Do pStream.Write($char(44) _ $select(pHasTopFields: ..Escape(##class(CUH.Recordmap.ENHSens2HL7.Record).PatientSurnameLogicalToDisplay(pObject.PatientSurname), $lb($char(44)),"quote",$char(34)), 1: ""),0)
                Do pStream.Write($char(44) _ $select(pHasTopFields: ..Escape(##class(CUH.Recordmap.ENHSens2HL7.Record).PatientFirstNameLogicalToDisplay(pObject.PatientFirstName), $lb($char(44)),"quote",$char(34)), 1: ""),0)
                Do pStream.Write($char(44) _ $select(pHasTopFields: ..Escape(##class(CUH.Recordmap.ENHSens2HL7.Record).DOBLogicalToDisplay(pObject.DOB), $lb($char(44)),"quote",$char(34)), 1: ""),0)
                Do pStream.Write($char(44) _ $select(pHasTopFields: ..Escape(##class(CUH.Recordmap.ENHSens2HL7.Record).PatientNHSNumberLogicalToDisplay(pObject.PatientNHSNumber), $lb($char(44)),"quote",$char(34)), 1: ""),0)
                Do pStream.Write($char(44) _ $select(pHasTopFields: ..Escape(##class(CUH.Recordmap.ENHSens2HL7.Record).MRNLogicalToDisplay(pObject.MRN), $lb($char(44)),"quote",$char(34)), 1: ""),0)
                Do pStream.Write($char(44) _ $select(pHasTopFields: ..Escape(##class(CUH.Recordmap.ENHSens2HL7.Record).CollectedLogicalToDisplay(pObject.Collected), $lb($char(44)),"quote",$char(34)), 1: ""),0)
                Do pStream.Write($char(44) _ $select(pHasTopFields: ..Escape(##class(CUH.Recordmap.ENHSens2HL7.Record).ReceivedLogicalToDisplay(pObject.Received), $lb($char(44)),"quote",$char(34)), 1: ""),0)
                Do pStream.Write($char(44) _ $select(pHasTopFields: ..Escape(##class(CUH.Recordmap.ENHSens2HL7.Record).ResultedLogicalToDisplay(pObject.Resulted), $lb($char(44)),"quote",$char(34)), 1: ""),0)
                Do pStream.Write($char(44) _ $select(pHasTopFields: ..Escape(##class(CUH.Recordmap.ENHSens2HL7.Record).VerifiedLogicalToDisplay(pObject.Verified), $lb($char(44)),"quote",$char(34)), 1: ""),0)
                Do pStream.Write($char(44) _ $select(pHasTopFields: ..Escape(##class(CUH.Recordmap.ENHSens2HL7.Record).SpecimenTypeLogicalToDisplay(pObject.SpecimenType), $lb($char(44)),"quote",$char(34)), 1: ""),0)
                Do pStream.Write($char(44) _ $select(pHasTopFields: ..Escape(##class(CUH.Recordmap.ENHSens2HL7.Record).SourceLogicalToDisplay(pObject.Source), $lb($char(44)),"quote",$char(34)), 1: ""),0)
                Do pStream.Write($char(44) _ $select(pHasTopFields: ..Escape(##class(CUH.Recordmap.ENHSens2HL7.Record).TestLogicalToDisplay(pObject.Test), $lb($char(44)),"quote",$char(34)), 1: ""),0)
                Do pStream.Write($char(44) _ $select(pHasTopFields: ..Escape(##class(CUH.Recordmap.ENHSens2HL7.Record).OrganismLogicalToDisplay(pObject.Organism), $lb($char(44)),"quote",$char(34)), 1: ""),0)
                Do pStream.Write($char(44) _ $select(pHasTopFields: ..Escape(##class(CUH.Recordmap.ENHSens2HL7.Record).AntibioticLogicalToDisplay(pObject.Antibiotic), $lb($char(44)),"quote",$char(34)), 1: ""),0)
                Do pStream.Write($char(44) _ $select(pHasTopFields: ..Escape(##class(CUH.Recordmap.ENHSens2HL7.Record).AntibioticInterpretationLogicalToDisplay(pObject.AntibioticInterpretation), $lb($char(44)),"quote",$char(34)), 1: ""),0)
                Do pStream.Write($char(44) _ $select(pHasTopFields: ..Escape(##class(CUH.Recordmap.ENHSens2HL7.Record).SubmitterLogicalToDisplay(pObject.Submitter), $lb($char(44)),"quote",$char(34)), 1: ""),0)
                Do pStream.Write($char(44) _ $select(pHasTopFields: ..Escape(##class(CUH.Recordmap.ENHSens2HL7.Record).ReqExtPatientIDLogicalToDisplay(pObject.ReqExtPatientID), $lb($char(44)),"quote",$char(34)), 1: ""),0)
                Do pStream.Write($char(13,10), 0, .tStatus)
                If $$$ISERR(tStatus) Quit tStatus
                Quit tStatus
}

/// Internal helper method to perform parse of incoming data
ClassMethod GetRecord(pStream As %IO.DeviceStream, pObject As %RegisteredObject, pGetTopFields As %Boolean, pTimeout As %Numeric, ByRef pLookAhead As %String = "", pLabelFound As %Boolean = 0) As %Status [ Internal, Private ]
{
                Set tStatus = $$$OK
                If pLabelFound { Set tRecLabel = "" } 
                Else { Set tRecLabel = "" } 
                Set tStatus = ..chunkRecord(pStream, .tRecordData, .tCharCount, "", pTimeout, $char(13,10), tRecLabel, pLookAhead, $char(34))
                If $$$ISERR(tStatus) Quit tStatus
                Set tMaxLocal = $$$MaxLocalLength
                Set tRecordData("charcount") = +$get(tCharCount)
                Set tCurrChunk = 1
                Set tTopPiece = 1
                Set tCurrSep = 1
                Set tStrings = 1
                Set tSeps = $listbuild($char(44))
                If '$data(tRecordData(tCurrChunk, "block")) {
                                Set tCurrString = $piece(tRecordData(tCurrChunk), $char(44), tTopPiece)
                }
                Else {
                                Set tCurrString = tRecordData(tCurrChunk)
                                Set tCurrPiece = 1
                }
                Do ..checkMerge(.tCurrChunk, .tRecordData, .tCurrString, $char(44), .tTopPiece)
                If pGetTopFields {
                                Set pObject.SpecimenID = tCurrString
                }
                If '$data(tRecordData(tCurrChunk, "block")) {
                                Set tCurrString = $piece(tRecordData(tCurrChunk), $char(44), tTopPiece)
                }
                Do ..checkMerge(.tCurrChunk, .tRecordData, .tCurrString, $char(44), .tTopPiece)
                If pGetTopFields {
                                Set pObject.PatientSurname = tCurrString
                }
                If '$data(tRecordData(tCurrChunk, "block")) {
                                Set tCurrString = $piece(tRecordData(tCurrChunk), $char(44), tTopPiece)
                }
                Do ..checkMerge(.tCurrChunk, .tRecordData, .tCurrString, $char(44), .tTopPiece)
                If pGetTopFields {
                                Set pObject.PatientFirstName = tCurrString
                }
                If '$data(tRecordData(tCurrChunk, "block")) {
                                Set tCurrString = $piece(tRecordData(tCurrChunk), $char(44), tTopPiece)
                }
                Do ..checkMerge(.tCurrChunk, .tRecordData, .tCurrString, $char(44), .tTopPiece)
                If pGetTopFields {
                                Set pObject.DOB = tCurrString
                }
                If '$data(tRecordData(tCurrChunk, "block")) {
                                Set tCurrString = $piece(tRecordData(tCurrChunk), $char(44), tTopPiece)
                }
                Do ..checkMerge(.tCurrChunk, .tRecordData, .tCurrString, $char(44), .tTopPiece)
                If pGetTopFields {
                                Set pObject.PatientNHSNumber = tCurrString
                }
                If '$data(tRecordData(tCurrChunk, "block")) {
                                Set tCurrString = $piece(tRecordData(tCurrChunk), $char(44), tTopPiece)
                }
                Do ..checkMerge(.tCurrChunk, .tRecordData, .tCurrString, $char(44), .tTopPiece)
                If pGetTopFields {
                                Set pObject.MRN = tCurrString
                }
                If '$data(tRecordData(tCurrChunk, "block")) {
                                Set tCurrString = $piece(tRecordData(tCurrChunk), $char(44), tTopPiece)
                }
                Do ..checkMerge(.tCurrChunk, .tRecordData, .tCurrString, $char(44), .tTopPiece)
                If pGetTopFields {
                                Set pObject.Collected = tCurrString
                }
                If '$data(tRecordData(tCurrChunk, "block")) {
                                Set tCurrString = $piece(tRecordData(tCurrChunk), $char(44), tTopPiece)
                }
                Do ..checkMerge(.tCurrChunk, .tRecordData, .tCurrString, $char(44), .tTopPiece)
                If pGetTopFields {
                                Set pObject.Received = tCurrString
                }
                If '$data(tRecordData(tCurrChunk, "block")) {
                                Set tCurrString = $piece(tRecordData(tCurrChunk), $char(44), tTopPiece)
                }
                Do ..checkMerge(.tCurrChunk, .tRecordData, .tCurrString, $char(44), .tTopPiece)
                If pGetTopFields {
                                Set pObject.Resulted = tCurrString
                }
                If '$data(tRecordData(tCurrChunk, "block")) {
                                Set tCurrString = $piece(tRecordData(tCurrChunk), $char(44), tTopPiece)
                }
                Do ..checkMerge(.tCurrChunk, .tRecordData, .tCurrString, $char(44), .tTopPiece)
                If pGetTopFields {
                                Set pObject.Verified = tCurrString
                }
                If '$data(tRecordData(tCurrChunk, "block")) {
                                Set tCurrString = $piece(tRecordData(tCurrChunk), $char(44), tTopPiece)
                }
                Do ..checkMerge(.tCurrChunk, .tRecordData, .tCurrString, $char(44), .tTopPiece)
                If pGetTopFields {
                                Set pObject.SpecimenType = tCurrString
                }
                If '$data(tRecordData(tCurrChunk, "block")) {
                                Set tCurrString = $piece(tRecordData(tCurrChunk), $char(44), tTopPiece)
                }
                Do ..checkMerge(.tCurrChunk, .tRecordData, .tCurrString, $char(44), .tTopPiece)
                If pGetTopFields {
                                Set pObject.Source = tCurrString
                }
                If '$data(tRecordData(tCurrChunk, "block")) {
                                Set tCurrString = $piece(tRecordData(tCurrChunk), $char(44), tTopPiece)
                }
                Do ..checkMerge(.tCurrChunk, .tRecordData, .tCurrString, $char(44), .tTopPiece)
                If pGetTopFields {
                                Set pObject.Test = tCurrString
                }
                If '$data(tRecordData(tCurrChunk, "block")) {
                                Set tCurrString = $piece(tRecordData(tCurrChunk), $char(44), tTopPiece)
                }
                Do ..checkMerge(.tCurrChunk, .tRecordData, .tCurrString, $char(44), .tTopPiece)
                If pGetTopFields {
                                Set pObject.Organism = tCurrString
                }
                If '$data(tRecordData(tCurrChunk, "block")) {
                                Set tCurrString = $piece(tRecordData(tCurrChunk), $char(44), tTopPiece)
                }
                Do ..checkMerge(.tCurrChunk, .tRecordData, .tCurrString, $char(44), .tTopPiece)
                If pGetTopFields {
                                Set pObject.Antibiotic = tCurrString
                }
                If '$data(tRecordData(tCurrChunk, "block")) {
                                Set tCurrString = $piece(tRecordData(tCurrChunk), $char(44), tTopPiece)
                }
                Do ..checkMerge(.tCurrChunk, .tRecordData, .tCurrString, $char(44), .tTopPiece)
                If pGetTopFields {
                                Set pObject.AntibioticInterpretation = tCurrString
                }
                If '$data(tRecordData(tCurrChunk, "block")) {
                                Set tCurrString = $piece(tRecordData(tCurrChunk), $char(44), tTopPiece)
                }
                Do ..checkMerge(.tCurrChunk, .tRecordData, .tCurrString, $char(44), .tTopPiece)
                If pGetTopFields {
                                Set pObject.Submitter = tCurrString
                }
                If '$data(tRecordData(tCurrChunk, "block")) {
                                Set tCurrString = $piece(tRecordData(tCurrChunk), $char(44), tTopPiece)
                }
                Do ..checkMerge(.tCurrChunk, .tRecordData, .tCurrString, $char(44), .tTopPiece)
                If pGetTopFields {
                                Set pObject.ReqExtPatientID = tCurrString
                }
                Quit tStatus
}

XData RecordMap [ XMLNamespace = "http://www.intersystems.com/Ensemble/RecordMap" ]
{
<Record xmlns="http://www.intersystems.com/Ensemble/RecordMap" name="CUH.Recordmap.ENHSens2HL7" type="delimited" char_encoding="UTF-8" targetClassname="CUH.Recordmap.ENHSens2HL7.Record" padFromLeft="0" recordTerminator="\x0d\x0a" escaping="quote" escapeSequence="&quot;">
  <Separators>
    <Separator>,</Separator>
  </Separators>
  <Field name="SpecimenID" required="0" datatype="%String"></Field>
  <Field name="PatientSurname" required="0" datatype="%String"></Field>
  <Field name="PatientFirstName" required="0" datatype="%String"></Field>
  <Field name="DOB" required="0" datatype="%String"></Field>
  <Field name="PatientNHSNumber" required="0" datatype="%String"></Field>
  <Field name="MRN" required="0" datatype="%String"></Field>
  <Field name="Collected" required="0" datatype="%String"></Field>
  <Field name="Received" required="0" datatype="%String"></Field>
  <Field name="Resulted" required="0" datatype="%String"></Field>
  <Field name="Verified" required="0" datatype="%String"></Field>
  <Field name="SpecimenType" required="0" datatype="%String"></Field>
  <Field name="Source" required="0" datatype="%String"></Field>
  <Field name="Test" required="0" datatype="%String"></Field>
  <Field name="Organism" required="0" datatype="%String"></Field>
  <Field name="Antibiotic" required="0" datatype="%String"></Field>
  <Field name="AntibioticInterpretation" required="0" datatype="%String"></Field>
  <Field name="Submitter" required="0" datatype="%String"></Field>
  <Field name="ReqExtPatientID" required="0" datatype="%String"></Field>
</Record>
}

}

Kind Regards

Stuart

Stuart Byrne | eHospital Senior Epic Interface & Environment Systems Analyst |

Cambridge University Hospitals NHS Foundation Trust | Box 117 |
Ext: 58365 Direct Dial: 01223 348365
This email is confidential, see www.cuh.org.uk/email_disclaimer.html

Annual Leave: 26/05 (Half Day), 02/06, 05/06, 12/06, 12/07 – 28/07

Discussion (2)0

Log in or sign up to continue

David E Nelson · May 25, 2017

Hi Stuart,

Are you writing your own business service?

Have you looked at EnsLib.RecordMap.Service.FTPService , which Ensemble includes out of the box? Maybe it does everything you need.

Best Regards,

David

1 0

score 0 · Answer 1 · 2017-06-08T12:09:27-04:00

Thanks David. This worked really well and I was able to call my record map and pass to a BP router.

In the meantime I got my stream to work, but the EnsLib.RecordMap.Service.FTPService is more straightforward and will be supported in future upgrades.