How to resolve proDOCMAN BPL Error: 5002 : Cache error: <MAXSTRING>zS2+72^pacDOCMAN.proDOCMAN.Thread1.1

Question

Question

Adrian Izadpanah · Jun 6, 2022

Hi All,

I am taking a look at some legacy coding as it seems to be generating an error when processing the messaging. The message being process contains a PDF which we are transforming into a stream and sending to another system.

I have noticed this issue only occurs with PDFS that are roughly 2800kbs in size or above. All other PDFS process fine.

The error that it is generating is proDOCMAN BPL Error: 5002 : Cache error: <MAXSTRING>zS2+72^pacDOCMAN.proDOCMAN.Thread1.1

proDOCMAN being the name of the process.

Having taken a look at the process and adding TRACE to see where the issue is, I have narrowed it down to this line.

set context.strDocumentEncoded = $system.Encryption.Base64Encode(context.strDocument)

strDocumentEncoded is set to %String(MAXLEN="")

strDocument is set to %String(MAXLEN="")

Any help would be appreciated.

Discussion (8)1

Log in or sign up to continue

Julius Kavay · Jun 7, 2022

you miss the object reference!

set context.strDocumentEncoded = B64EncodeStream(request.streamPDF)
// ---------------------------^^^^^^^ this should be something
set context.strDocumentEncoded = ##(your.class).B64EncodeStream(request.streamPDF)

but you have another problems too: your context.strDocument and context.strDocumentEncoded are currently STRING properties (according to the operation you try to do), which are limitet to a maxlength of 3.47MB!

You have to change both to a STREAM, so you can handle PDFs larger then ca. 2.6MB (because 2.6 * 4 / 3 ==> 3.46MB, the limit for a string variable).

After you change context.strDocument and context.strDocumentEncoded to a stream properties, you could use this code:

Class DC.Someclass Extends %RegisteredObject
{

Parameter CHUNKSIZE = 2097144;

ClassMethod ToBase64(src As %Stream.Object, dst As %Stream.Object) As %Status
{
	i ..#CHUNKSIZE#3=0, src.Rewind(), dst.Rewind() {
		set sts=$$$OK
		while 'src.AtEnd,sts {
			do dst.Write($system.Encryption.Base64Encode(src.Read(..#CHUNKSIZE,.sts),1))
		}
	} else { set sts=$$Error^%apiOBJ(5001,"Chunksize or src/dst-problem") }
	
	quit sts
}

}

if ##class(your.class).ToBase64(context.strDocument,context.strDocumentEncoded) write "OK"

It seems, this task will take some time... you have to check, how context.strDocument is populated and how context.strDocumentEncoded is later in code used. Good luck.

2 0

score 1 · Answer 1 · 2022-06-06T05:59:11-04:00

It seems, you read the whole PDF into one string thenconverting this to Base64... So there are two "chances to get an error":

- the PDF is longer then the MAXSTRING value (which is 3641144 chars) and

- converting a string to Base64 increases the length of the string by a factor of 1.333 (you get your maxstr here).

The solution is: you read your PDF in chunks, convert this chunks into Base64 and output the converted data into a stream. Due to the nature of Base64, the length of the chunks you read in MUST BE a multiple of 3 (3 input bytes will be converted into 4 output bytes).

score 1 · Answer 2 · 2022-06-06T10:31:47-04:00

Here's a sample class to Base64 encode a stream:

Class Example.B64.Util Extends %RegisteredObject
{

/// Be cautious if changing CHUNKSIZE. Incorrect values could cause the resulting encoded data to be invalid.
/// It should always be a multiple of 57 and needs to be less than ~2.4MB when MAXSTRING is 3641144
Parameter CHUNKSIZE = 2097144;

ClassMethod B64EncodeStream(pStream As %Stream.Object, pAddCRLF As %Boolean = 0) As %Stream.Object
{
    set tEncodedStream=##class(%Stream.GlobalCharacter).%New()
    
    do pStream.Rewind()
    
    while ('pStream.AtEnd) {
        set tReadLen=..#CHUNKSIZE
        set tChunk=pStream.Read(.tReadLen)
        
        do tEncodedStream.Write($System.Encryption.Base64Encode(tChunk,'pAddCRLF))
        if (pAddCRLF && 'pStream.AtEnd) {
            do tEncodedStream.Write($c(13,10))
        }
    }
    
    do tEncodedStream.Rewind()
    
    quit tEncodedStream
}
}

score 0 · Answer 3 · 2022-06-07T05:47:33-04:00

Thanks both for your response.

Marc I have tried the above but seem to now be getting another error

So this is what I currently have.

Parameter CHUNKSIZE = 2097144;

ClassMethod B64EncodeStream(pStream As %Stream.Object, pAddCRLF As %Boolean = 0) As %Stream.Object
{ set tEncodedStream=##class(%Stream.GlobalCharacter).%New()

    do pStream.Rewind()

    while ('pStream.AtEnd) {
        set tReadLen=..#CHUNKSIZE
        set tChunk=pStream.Read(.tReadLen)

        do tEncodedStream.Write($System.Encryption.Base64Encode(tChunk,'pAddCRLF))
        if (pAddCRLF && 'pStream.AtEnd) {
            do tEncodedStream.Write($c(13,10))
        }
    }

    do tEncodedStream.Rewind()
quit tEncodedStream
}

And then in my main piece of code:

set context.strDocumentEncoded = B64EncodeStream(request.streamPDF)

However, I am now getting an error:

proDOCMAN BPL Error: 5002 : Cache error: <UNDEFINED>zS2+61^pacDOCMAN.proDOCMAN.Thread1.1 *B64EncodeStream(24@%Stream.GlobalBinary) (alert request ID=6809)

score 0 · Answer 4 · 2022-06-13T04:28:00-04:00

Hi Julius, thanks for the advice.

Can I just ask a final question. When I trace the stream, all I see is the following: 8@%Stream.GlobalBinary

Is this correct or has something gone wrong?

Thanks

score 1 · Answer 5 · 2022-06-13T05:48:26-04:00

If you have a (whatever) class with an property like:

Property propName As %Stream.GlobalCharacter;

and you have an instance of that object in a variable obj then a command like the below

write obj.propName --> nn@%Stream.GlobalCharacter

shows you the object reference, which is (formal) an integer number followed by an '@' symbol followed by the name of the class. With other words, what you see is correct.

score 0 · Answer 6 · 2022-06-14T03:01:48-04:00

Is there anyway for me to see the actual stream or to confirm that the encoding is working/successful. I say this because I am sending the stream (PDF) to another system, but on the other system the PDF doesn't open, it errors...

I am not sure what has gone wrong.

score 0 · Answer 7 · 2022-06-14T17:11:07-04:00

Sure, you can check some key points:
- the size of your PDF-file (in bytes) must be the same as the size of the context.strDocument
- the size of the encoded stream must be 1.33 times of the unencoded stream (see below)
- the second parameter of the Base64Encode() method must be set to 1, else you get a stream with line breaks!

set docSize = context.strDocument.Size
set encSize = context.strDocumentEncoded.Size

if -docSize#3+docSize*4/3-encSize { write "Base64 stream has wrong size" }

Your "old" version sent a string, the new version should send a stream - is there everything OK? Just double check all the recent changes.