Custom file pass through Operation writing out unreadable pdf files

Question

Question

Thembelani Mlalazi · Jun 22, 2018

#Beginner #Business Operation #Object Data Model #Ensemble

I have a service that takes a file and pass it through to the production .While I am passing the file through I get the file stream and set it to a variable within my message and the variable is of type %Ens.StreamContainer. But after all processing and I need to write out my file to a pdf format The file gets written but is a corrupt file since I can not read it I have tried this with asimple pass through everything is fine .But here I do not know what I am doing wrong here is the operation code

 set pInput=pRequest.FileStream
     ;the variable to hold the status for the method
 #dim status as %Status=$$$OK
 ;clear the pResponse
  kill pResponse
     set pResponse=$$$NULLOREF
 ;set the file name to the sequence number
  set ..Filename=pRequest.NewFileName
 
  ;the filepath set on the settings of this OPERATION
   set origDirectory = ..Adapter.FilePath
  ;the file directory to drop the file
  set ..Adapter.FilePath = ..Adapter.FilePath_"\"_..StubDirectory
  
 ;start writing out file data to the file in filename
;this bit works fine
    set:$$$ISOK(status) status= ..Adapter.PutLine(..Filename_"."_..stubExtension,
  $CHAR(34)_ pRequest.ClientID_$CHAR(34)_$CHAR(44)_$CHAR(34)_pRequest.UserId_$CHAR(34)_$CHAR(44)_$CHAR(34)_
  pRequest.DocumentType _$CHAR(34)_$CHAR(44)_$CHAR(34)_pRequest.Title_$CHAR(34)_$CHAR(44)_$CHAR(34)_pRequest.Description_$CHAR(34)_$CHAR(44)_$CHAR(34)_pRequest.Author _$CHAR(34)_$CHAR(44)_$CHAR(34)_pRequest.DocumentDate_$CHAR(34)_$CHAR(44)_$CHAR(34)_pRequest.FinalRevision_$CHAR(34))
 
 if ($$$ISOK(status))
 {
  ;set back to the operation settings
  set ..Adapter.FilePath = origDirectory
    ;set the file name  to write out to
  set ..Filename=pRequest.NewFileName_"."_..DocExtension
  ;set the filepath on the production settings to this variable
  set origDirectory = ..Adapter.FilePath
  ;set the new filepath
  set ..Adapter.FilePath = ..Adapter.FilePath_"\"_..DocumentDirectory
     
 
  //here I turn to write out but file is corrupt
  
   Quit:'$IsObject(pInput.Stream) $$$ERROR($$$EnsErrGeneral,"No Stream contained in StreamContainer Request")
 Set tFilename=..Adapter.CreateTimestamp(##class(%File).GetFilename(pInput.OriginalFilename),..Filename)
 Set status=..Adapter.PutStream(..Filename, pInput.Stream)
 Do pInput.%Save() ; re-save in case PutStream() optimization changed the Stream filename
 
 
 
 
   set ..Adapter.FilePath = origDirectory
  
   if ($$$ISOK(status))
 {
  set pResponse=##class(BSMHFT.DocumentUpload.GenericRESP).%New()
  set pResponse.Process="RiOFileOPRN_files Written to their respective directories"
  set pResponse.Status=status
  set status=pResponse.%Save()
 }
  
 }
 
 return status
}

Discussion (11)2

Log in or sign up to continue

Marc Mundt · Jun 27, 2018

I see an issue in the first piece of code you posted.

At the top of the method you do this:
set pInput=pRequest.FileStream

But when you write the stream to the file, you do this:
set status=..Adapter.PutStream(..Filename, pInput.Stream)

I don't know for certain what type of object pRequest is or pRequest.FileStream, but in the service you posted it looks like pRequest.FileStream is a %Stream.Object.

Method OnProcessInput(pInput As %Stream.Object, Output pOutput As %RegisteredObject) As %Status
...
set pt.filestream=pInput

If pRequest.FileStream is a %Stream.Object, then it won't have a Stream property and passing pInput.Stream to PutStream should fail. Try changing this to:
set status=..Adapter.PutStream(..Filename, pInput)

0 0

score 0 · Answer 1 · 2018-06-22T10:37:36-04:00

Try setting Charset to Binary in your BO, and check that file is Binary in your source BH.

I'd try sending txt file through the pipeline and compare the results.

score 0 · Answer 2 · 2018-06-27T03:08:49-04:00

@Eduard Lebedyuk I have tried your suggestions and still get the error when opening the file with adobe reader. If I try on a simple pass through operation like this below everything works fine.

ERROR: Adobe acrobat reader could not open file.pdf because it is either not a supported file type or because the file has been damaged.

the service

Method OnProcessInput(pInput As %Stream.Object, Output pOutput As %RegisteredObject) As %Status
{
#dim pt as TestingEnvironment.ECGTrace.TEST.FSMREQ=##class(TestingEnvironment.ECGTrace.TEST.FSMREQ).%New()
  Set tSource=pInput.Attributes("Filename"), pInput=$zobjclassmethod(..#CONTAINERCLASS,"%New",pInput)
 Set tSC=..resolveAndIndex(pInput) Quit:$$$ISERR(tSC) tSC
 set pt.filestream=pInput
 set pt.path=tSource
 Set tWorkArchive=(""'=..Adapter.ArchivePath)&&(..Adapter.ArchivePath=..Adapter.WorkPath || (""=..Adapter.WorkPath && (..Adapter.ArchivePath=..Adapter.FilePath)))
 $$$SyncCommitSet(tSyncCommit)
 For iTarget=1:1:$L(..TargetConfigNames, ",") { Set tOneTarget=$ZStrip($P(..TargetConfigNames,",",iTarget),"<>W")  Continue:""=tOneTarget
  $$$sysTRACE("Sending input Stream "_pInput.Stream_"("_pInput.Stream.Size_")"_$S(tWorkArchive:" Async",1:" Sync")_" from '"_tSource_"' to '"_tOneTarget_"'")
  If tWorkArchive {
   Set tSC1=..SendRequestAsync(tOneTarget,pt)  Set:$$$ISERR(tSC1) tSC=$$$ADDSC(tSC,tSC1)
   //Set tSC1=..SendRequestAsync(tOneTarget,pInput)  Set:$$$ISERR(tSC1) tSC=$$$ADDSC(tSC,tSC1)
  } Else {
   #; If not archiving send Sync to avoid Adapter deleting file before Operation gets it
   //Set tSC1=..SendRequestSync(tOneTarget,pInput)  Set:$$$ISERR(tSC1) tSC=$$$ADDSC(tSC,tSC1)
   Set tSC1=..SendRequestSync(tOneTarget,pt)  Set:$$$ISERR(tSC1) tSC=$$$ADDSC(tSC,tSC1)
  }
 }
 $$$SyncCommitClear(tSyncCommit)
 Quit tSC
}

the operation

Method OnMessage(pREs As TestingEnvironment.ECGTrace.TEST.FSMREQ, pRequest As Ens.StreamContainer, Output pResponse As %Persistent) As %Status
{
 set pRequest=pREs.filestream
 Quit:'$IsObject(pRequest.Stream) $$$ERROR($$$EnsErrGeneral,"No Stream contained in StreamContainer Request")
 Set tFilename=..Adapter.CreateTimestamp(##class(%File).GetFilename(pRequest.OriginalFilename),..Filename)
 Set tSC=..Adapter.PutStream(tFilename, pRequest.Stream)
 Do pRequest.%Save() ; re-save in case PutStream() optimization changed the Stream filename
 Quit tSC
}

my code the service

Method OnProcessInput(pInput As %Stream.Object, Output pOutput As %RegisteredObject) As %Status
{
 #dim meta as DocumentUpload.GenericUploadMREQ=##class(DocumentUpload.GenericUploadMREQ).%New()
 
   ;get the filepath from the request 
   ;;wrap the stream object into container for easy transpotation
    Set tFileName=pInput.Attributes("Filename") , pInput=$zobjclassmethod(..#CONTAINERCLASS,"%New",pInput)
 $$$TRACE(tFileName) 
 ;get the file name
    set dataPiece=##class(%File).GetFilename(tFileName)
 $$$TRACE(dataPiece)
      Set tSC=..resolveAndIndex(pInput) Quit:$$$ISERR(tSC) tSC
      ;check if the file path data is populated
 if (dataPiece'="")
    {
    ;build the ECG Message
          set meta.ClientID =$Piece(dataPiece,"_",1) 
          set meta.LastName=$Piece(dataPiece,"_",2)
          set meta.FirstName=$Piece(dataPiece,"_",3)
          set meta.DateOfBirth=$Piece(dataPiece,"_",4)
          set meta.Directory =$Piece(tFileName,"\",*-1)
          set meta.OGFileName  =dataPiece
          set meta.Fullpath  =tFileName
          ;get the date to testing
          set DateOfTest=$Piece(dataPiece,"_",5)
          ;get the time of testing
          set mtim=$Piece(dataPiece,"_",6)
          ;separate the extension of the file path and the time
          set TimeOfTest=$Piece(mtim,".",1)
          set meta.FileTimeStamp =DateOfTest_""_TimeOfTest
          set meta.recordAdded =$ZDT($ZTIMESTAMP,3,1,3)
          set meta.TargetConfig ="RIO.DocumentUpload.RiOFileOPRN"
          set meta.payLoad=pInput
          set meta.DocumentType=..DocumentType
       set meta.Description=..Description
       set meta.Title=..Title
       set meta.FinalRevision=..Revesion
       set meta.Author=..Author_""_$Piece(tFileName,"\",*-1)
       set messagetype=$PIECE(..Author," ",1)
       set meta.UserId=..UserID
          set meta.sourceConfig =tFileName
          set meta.TypeMes=messagetype
         
         
          Set tWorkArchive=(""'=..Adapter.ArchivePath)&&(..Adapter.ArchivePath=..Adapter.WorkPath || (""=..Adapter.WorkPath && (..Adapter.ArchivePath=..Adapter.FilePath)))
 $$$SyncCommitSet(tSyncCommit)
 For iTarget=1:1:$L(..TargetConfigNames, ",") { Set tOneTarget=$ZStrip($P(..TargetConfigNames,",",iTarget),"<>W")  Continue:""=tOneTarget
  $$$sysTRACE("Sending input Stream "_pInput.Stream_"("_pInput.Stream.Size_")"_$S(tWorkArchive:" Async",1:" Sync")_" from '"_tFileName_"' to '"_tOneTarget_"'")
  If tWorkArchive {
   Set tSC1=..SendRequestAsync(tOneTarget,meta)  Set:$$$ISERR(tSC1) tSC=$$$ADDSC(tSC,tSC1)
   
  } Else {
   #; If not archiving send Sync to avoid Adapter deleting file before Operation gets it
   Set tSC1=..SendRequestSync(tOneTarget,meta)  Set:$$$ISERR(tSC1) tSC=$$$ADDSC(tSC,tSC1)
  }
 }
 $$$SyncCommitClear(tSyncCommit)
         
         
       }
            quit tSC
}

the operation please note I have a Route in between which simple transformers the message to the message expected by the operation its just a simple mapping scenario

Method WriteOutFiles(pRequest As DocumentUpload.FileMREQ, pInput As Ens.StreamContainer, Output pResponse As DocumentUpload.GenericRESP) As %Status
{
 set pInput=pRequest.FileStream
     ;the variable to hold the status for the method
 #dim status as %Status=$$$OK
 ;clear the pResponse
  kill pResponse
     set pResponse=$$$NULLOREF
 ;set the file name to the sequence number
  set ..Filename=pRequest.NewFileName
 
  ;the filepath set on the settings of this OPERATION
   set origDirectory = ..Adapter.FilePath
  ;the file directory to drop the file
  set ..Adapter.FilePath = ..Adapter.FilePath_"\"_..StubDirectory
  
 ;start writing out file data to the stub file
    set:$$$ISOK(status) status= ..Adapter.PutLine(..Filename_"."_..stubExtension,
  $CHAR(34)_ pRequest.ClientID_$CHAR(34)_$CHAR(44)_$CHAR(34)_pRequest.UserId_$CHAR(34)_$CHAR(44)_$CHAR(34)_
  pRequest.DocumentType _$CHAR(34)_$CHAR(44)_$CHAR(34)_pRequest.Title_$CHAR(34)_$CHAR(44)_$CHAR(34)_pRequest.Description_$CHAR(34)_$CHAR(44)_$CHAR(34)_pRequest.Author _$CHAR(34)_$CHAR(44)_$CHAR(34)_pRequest.DocumentDate_$CHAR(34)_$CHAR(44)_$CHAR(34)_pRequest.FinalRevision_$CHAR(34))
 
 if ($$$ISOK(status))
 {
  ;set back to the operation settings
  set ..Adapter.FilePath = origDirectory
    ;set the file name  to write out to
  set ..Filename=pRequest.NewFileName_"."_..DocExtension
  ;set the filepath on the production settings to this variable
  set origDirectory = ..Adapter.FilePath
  ;set the new filepath
  set ..Adapter.FilePath = ..Adapter.FilePath_"\"_..DocumentDirectory
     
 
 // set:$$$ISOK(status) status=  ..Adapter.PutStream(..Filename, pInput.Stream) //error here to file
  set:$$$ISOK(status) status=..OriginalFileOut(pInput,pResponse,..Filename)
 
      ;set adapter to its original file path
   set ..Adapter.FilePath = origDirectory
  
   ;check writing out file worked
   if ($$$ISOK(status))
 {
  set pResponse=##class(DocumentUpload.GenericRESP).%New()
  set pResponse.Process="FileOPRN_files Written to their respective directories"
  set pResponse.Status=status
  set status=pResponse.%Save()
 }
  
 }
 
 ;return status
 return status
}

// passthrough original method to write out file

Method OriginalFileOut(pRequest As Ens.StreamContainer, Output pResponse As %Persistent, filenamess) As %Status
{
 
 Quit:'$IsObject(pRequest.Stream) $$$ERROR($$$EnsErrGeneral,"No Stream contained in StreamContainer Request")
 Set tFilename=..Adapter.CreateTimestamp(##class(%File).GetFilename(pRequest.OriginalFilename),filenamess)
 Set tSC=..Adapter.PutStream(tFilename, pRequest.Stream)
 Do pRequest.%Save() ; re-save in case PutStream() optimization changed the Stream filename
 Quit tSC
}

score 0 · Answer 3 · 2018-06-29T04:55:44-04:00

thanks the problem was the pdf is more like an image file so using a container to pass the stream around is a good idea only if the message leaves the service to the operation but in my case I had a process which was going to forward that message to another process so the first process had all the contents of a stream but when it forwards it to the next process the contents are empty so in my case I changed the stream object to a %Streambinary and I am not using the container anymore everything is working fine.

score 0 · Answer 4 · 2018-06-25T05:22:00-04:00

Eduard Lebedyuk · Jun 25, 2018

Please try to send 1.txt through the pipeline with contents like 123.

Are the hexdumps the same for that case?

0 0

score 0 · Answer 5 · 2018-06-25T10:25:26-04:00

Thembelani Mlalazi · Jun 25, 2018

@Eduard Lebedyuk I have tried changing the file to a txt file and the file is being created with nothing in it

0 0

score 0 · Answer 6 · 2018-06-25T10:46:54-04:00

Eduard Lebedyuk · Jun 25, 2018

And what about pdf?

Is it empty too?

0 0

score 0 · Answer 7 · 2018-06-27T03:04:50-04:00

Thembelani Mlalazi · Jun 27, 2018

@Eduard Lebedyuk its not readable I cannot open it

0 0

score 0 · Answer 8 · 2018-06-27T04:28:05-04:00

I recently had a case where PDF and even Excel did not work in the Cache and the reason was the lack of Java on one of my computers.

score 0 · Answer 9 · 2018-06-27T14:32:54-04:00

Eduard Lebedyuk · Jun 27, 2018

What about output file size?

0 0

score 0 · Answer 10 · 2018-06-27T16:09:11-04:00

The method signature for OnMessage usually only has two parameters: the inbound request and a response to return.

In your method you have three parameters and two of them seem to be inbound requests. Can you clarify?

Method OnMessage(pREs As TestingEnvironment.ECGTrace.TEST.FSMREQ, pRequest As Ens.StreamContainer, Output pResponse As %Persistent) As %Status