Large Stream to base64Encoded
I have below method , which receives a PDF ; this method throws error for a PDF file of 3MB size
<MAXSTRING>zBase64Encode+9 -- logged as '-' number - @' set encodedData = $system.Encryption.Base64Encode(content)'
How do I fix this.
----------------
ClassMethod Base64Encode(pStream As %Stream) As %Stream
{
set tSC = $$$OK
set tSC = pStream.Rewind()
s pEncoded=##class(%Stream.GlobalCharacter).%New()
while 'pStream.AtEnd {
Set tLen = pStream.Size
set content = pStream.Read(.tLen)
set encodedData = $system.Encryption.Base64Encode(content)
set encodedData=$translate(encodedData, $c(13,10))
s tSC=pEncoded.Write(encodedData)
q:$$$ISERR(tSC)
}
//q:$$$ISERR(tSC)
do pEncoded.Rewind()
Quit pEncoded
}
Comments
There is a limit to string length, as explained in the documentation.
To convert a binary stream to a base64 encoded character stream, you have to call the function in a loop using a buffer. Due the way the function works, the buffer length must be a multiple of 57.
Here a sample implementation, from ks.lib.stream.Utils class in the ks-iris-lib package available on Open Exchange.
/// encode stream data into encoded using bufferSize (due the implementation of $system.Encryption.Base64Encode, this must be a multiple of 57)
ClassMethod Base64Encode(stream As %Stream.Object, Output encoded As %Stream.Object, bufferSize As %Integer = 5700) As %Status
{
#Dim sc as %Status
#Dim ex as %Exception.AbstractException
#Dim len As %Integer
s sc = $$$OK
try {
throw:((bufferSize#57)'=0) ##class(%Exception.General).%New("buffer size must be a multiple of 57")
$$$TOE(sc,stream.Rewind())
s:'$d(encoded) encoded = ##class(%Stream.TmpCharacter).%New()
s len=bufferSize
while 'stream.AtEnd {
$$$TOE(sc,encoded.Write($system.Encryption.Base64Encode(stream.Read(.len),1)))
s len = bufferSize
}
}
catch (ex) {
s sc = ex.AsStatus()
}
return sc
}See the example class below
Class DC.Encoding Extends %RegisteredObject
{
/// Take an raw stream (i.e. unencoded) and
/// output a new, Base64 encoded stream.
///
ClassMethod ToBase64(str)
{
// Base64 encoding means:
// you take 3*N characters from the source
// and put 4*N characters into the destination.
// If the size of the source is not a multiple of 3 then
// the last one or two bytes will be padded.
//
// If you take an N such that 4*N less or equal 32767
// (the max size of a short string) then Cache or IRIS
// can work with short strings, which perform (usually)
// better than long strings
//
// N is integer.
//
// A good value for N is 8190,
// so you read 24570 bytes from the source and write 32760 to the destination
//
// Of course, you can take whatever number up to 910286
// (3 * 910286 = 2730858, 4 * 910286 = 3641144)
//
set len=8190*3
set flg=1 // this flag instructs $system.Encryption.Base64Encode
// not to insert linebreaks at every 76 characters
set new=##class(%Stream.GlobalCharacter).%New()
do str.Rewind()
while 'str.AtEnd {
do new.Write($system.Encryption.Base64Encode(str.Read(len),flg))
}
quit new
}
/// Take a Base64 encoded stream
/// and decode it to a new stream
///
/// The method itself has no information about the decoded data
/// hence it assumens binary data, but you, the caller (hopefully)
/// knows more about your data and can provide the correct stream
/// type for the decoder.
/// For exaple a character stream instead of binary.
ClassMethod FromBase64(str, new = 0)
{
// Base64 decoding means:
// you take 4*N characters from the source
// and put 3*N characters into the destination
//
// If you take an N such that 4*N less or equal 32767
// (the max size of a short string) then Cache or IRIS
// can work with short strings, which perform (usually)
// better than long strings
//
// N is integer.
//
// A good value for N is 8190,
// so you read 24570 bytes from the source and write 32760 to the destination
//
// Of course, you can take whatever number up to 910286
// (3 * 910286 = 2730858, 4 * 910286 = 3641144)
//
set len=8190*4
set:'new new=##class(%Stream.GlobalBinary).%New()
do str.Rewind()
while 'str.AtEnd {
do new.Write($system.Encryption.Base64Decode(str.Read(len)))
}
quit new
}
ClassMethod Test(file)
{
set str=##class(%Stream.FileBinary).%New()
do str.LinkToFile(file)
write str.Size,!
set enc=..ToBase64(str)
write enc.Size,!
set dec=..FromBase64(enc)
write dec.Size,!
}
}