Decoding Base64 PDF File

Question

Question

John Flippance · Jun 2, 2017

#Caché #Business Process (BPL) #Ensemble

Hello, I am writing some cache code that will pick up a PDF file, Base64 encode the contents and then send on to a third party system within a Long String (via their API). I have been testing this and discovered that the PDFs do not open within the supplier system (I get an error saying that it hasn't been decoded correctly). I wanted to prove that the issue does not lie with the way that I have Base64 encoded it within Ensemble, and therefore as a test wanted to Encode the PDF stream, then decode the stream and write out to a new file. Unfortunately after my code has created the file locally, when I go to open the file in Adobe, I get the same error message. Therefore I am trying to work out if I have not Encoded it properly, or have not Decoded it properly (or both!). Below is a code snippet of how I am performing this test within a BPL code block; ~~~ //context.streamPDF is a %StreamGlobalBinary containing the PDF stream. //context.streamPDFbase64 is a %Stream.GlobalBinary set stream2=##class(%Stream.GlobalBinary).%New() do context.streamPDF.Rewind() while 'context.streamPDF.AtEnd { set buffer = context.streamPDF.Read(4000) do context.streamPDFbase64.Write($System.Encryption.Base64Encode(buffer)) } do context.streamPDFbase64.Rewind() while 'context.streamPDFbase64.AtEnd { set temp=context.streamPDFbase64.Read(4000) set temp=$system.Encryption.Base64Decode(temp) do stream2.Write(temp) } //Output decoded to pdf file do stream2.Rewind() Set file=##class(%File).%New("C:\ANewPDF.pdf") Do file.Open("WSN") // The W means Write, S means put the file in stream mode, and N means create if not there already Set sc=file.CopyFrom(stream2) Do file.%Save() Do file.Flush() Do file.Close() ~~~ Any help greatly appreciated!

Discussion (13)2

Log in or sign up to continue

Rubens Silva · Jun 2, 2017

By default, Base64Decode and Base64Encode are functions used to decode and encode datatypes, or best saying... STRING.

Since you want to encode a stream the Decoder must understand that it should continue from the last chunk position instead of assuming a new string, otherwise you'll get a corrupted result.

Here's how XML Writer outputs an encoded binary.

/// <method>WriteBase64</method> encodes the specified binary bytes as base64 and writes out the resulting text.
/// This method is used to write element content.<br>
/// Argument:<br>
/// - <var>binary</var> The binary data to output. Type of %Binary or %BinaryStream.
Method WriteBase64(binary) As %Status
{
  If '..InRootElement Quit $$$ERROR($$$XMLNotInRootElement)

  If ..OutputDestination'="device" {
    Set io=$io
    Use ..OutputFilename:(/NOXY)
  }

  If ..InTag Write ">" Set ..InTag=0

  If $isObject(binary) {
    Do binary.Rewind() Set len=12000
    While 'binary.AtEnd {
      Write $system.Encryption.Base64Encode(binary.Read(.len),'..Base64LineBreaks)
    }
  } Else {
    Write $system.Encryption.Base64Encode(binary,'..Base64LineBreaks)
  }

  If ..OutputDestination'="device" {
    Use io
  }

  Set ..IndentNext=0

  Quit $$$OK
}

3 0

John Flippance · Jun 2, 2017

Thanks Rubens/Sean/Dmitry,

I decided to switch to the long strings route as the documents are small. I couldn't get this working either but it was because I wasn't reading the stream in a number divisible by 3 (I was just doing a .ReadLine() ). Have performed a Read(12000) and it now works a treat! PDF opening up in the supplier's system.

do context.streamPDF.Rewind()
while 'context.streamPDF.AtEnd
{
  set line=context.streamPDF.Read(12000)
  set context.strDocument =context.strDocument_line
}
set context.strDocumentEncoded = $system.Encryption.Base64Encode(context.strDocument)

3 0

Vitaliy Serdtsev · Jun 2, 2017

See still of the source code for the following methods:

##class(%Net.MIMEWriter).EncodeStreamBase64()
##class(%Net.SMTP).EncodeStreamBase64()
##class(%Atelier.v1.Utils.General).Base64FromStream()
##class(%XML.Writer).WriteBase64()

2 0

score 1 · Answer 1 · 2017-06-02T07:40:25-04:00

Great answer Rubens.

The class documentation makes no mention of the second parameter and I was not aware that it existed.

Fortunately I've only had to deal with documents under the large string size to date and did wonder how I would might need to work around that limitation at some point.

Question, the length the XML writer uses is set to 12000. Would this solution work for 12001 or does the size have to be divisible by 3? I'm wondering because 3 characters are represented by 4 characters in base64.

Sean.

score 2 · Answer 2 · 2017-06-02T07:42:53-04:00

Dmitry Maslennikov · Jun 2, 2017

Yes, should be divisible by 3.

2 0

score 0 · Answer 3 · 2017-06-02T08:14:53-04:00

Fabian Haupt · Jun 2, 2017

Which is also why your original code doesn't work (-> 4000)

0 0

score 0 · Answer 4 · 2017-06-02T08:38:07-04:00

Older Caché versions don't have that second parameter.
I also discovered about this on the hardest way: breaking the build.

score 0 · Answer 5 · 2022-09-14T15:35:45-04:00

I would be interested in a method that would simple encode a stream object into a base64 string but I don't see this method in the doc e.g.

score 0 · Answer 6 · 2022-09-14T17:49:00-04:00

This method mentioned above by Vitaliy takes a stream as input and outputs a base64 encoded stream:

https://cedocs.intersystems.com/ens20171/csp/documatic/%25CSP.Documatic....

score 0 · Answer 7 · 2022-09-14T18:31:28-04:00

If you can't find the methods Vitaliy mentioned because (for example) your Cache version is too old, you can always reinvent the wheel and write your own encoder/decoder method. Where is the problem? This snippet could be a starting point

Parameter Base64 = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";

ClassMethod Base64Enc(x)
{
    s f=-$l(x)#3, x=x_$e($c(0,0),1,f), y=..#Base64, z="" zt:$ziswide(x) "WIDE"
    f i=1:3:$l(x) s a=$a(x,i)*256+$a(x,i+1)*256+$a(x,i+2), c=262144 f j=1:1:4 s z=z_$e(y,a\c+1), a=a#c, c=c\64
    s:f z=$e(z,1,$l(z)-f)_$e("==",1,f) q z
}

score 0 · Answer 8 · 2022-09-16T15:00:14-04:00

Well we are in IRIS 2021 and that's the documentation I was looking in. So I'm not sure what's going on.

Here's where I landed:

Given a %Stream.FileBinary I calculate the size to read as such

s readSize=($J((stream.SizeGet()/12000),"",0)+1)*12000

And then 'cast' my stream to a %xsd.base64Binary datatype (ODBC type VARBINARY) as such

s base64string=##class(%xsd.base64Binary).LogicalToJSON(stream.Read(readSize,.sc))

In my command line testing I'm able to decode this base64string, write it to a file stream and save and I have a very much in tact PDF. This is new to me however, so I hope I'm not tricking myself into thinking this is working correctly.

When I run w ##class(%xsd.base64Binary).IsValid(myVarBinaryData) I get 1 so I think it's working correctly!

Wondering however about the reading of 12,000 at a time. Just as long as the read len is divisible by four it should work?

score 0 · Answer 9 · 2022-10-07T02:15:14-04:00

There are no such methods, because the size of the stream (file) can exceed the maximum length of the string, which at the moment is 3641144, so the type of string (%String, %VarString, %xsd.base64Binary, etc.) will not always be able to hold all the data.

But also it is not difficult to read the stream into a line:

ClassMethod StreamToStr(ByRef stream As %Stream.Object) As %String
{
  s s=""
  d stream.Rewind()
  while 'stream.AtEnd {
    s s=s_stream.Read($$$MaxLocalLength)
  }
  q s
}

PS: above I have given methods, some of which are available in older versions of Caché.

score 0 · Answer 10 · 2023-02-17T04:40:05-05:00

you can test this code，it will work

s B64S=##class(%GlobalCharacterStream).%New()
i '$IsObject(file) d
.s fileName=file
.s file=##class(%FileBinaryStream).%New()
.d file.LinkToFile(fileName)
.d file.Rewind()
s len=513 ; text must be divisible by 3 and 57
While 'file.AtEnd {
s text=file.Read(.len)
q:len'>0
d B64S.Write($tr($system.Encryption.Base64Encode(text),$c(10,13),""))
}
q B64S