Reading large stream from %CSP.Request

Question

Question

Chris Pursey · Jan 31, 2023

#CSP #ObjectScript #REST API #InterSystems IRIS #InterSystems IRIS for Health

We have created a number of REST adapters that extend %CSP.REST and these have been working fine. However, we are now facing a problem with a new one where very large XML documents are being posted. This is the pattern we are following:

Set inMsg = %request.Content.Read()

Set tSC = ##class(Ens.Director).CreateBusinessService("Epro_ItkHttp",.tBusinessService)

Set tSC = tBusinessService.ProcessInput(inMsg, .tBusinessServiceOutput)

However, when the %request is particularly long, it is being truncated, and working with %XML.Reader then fails in the business service class.

I believe the call to Read() is the problem. If I call it with some arbitrarily long value - such as Read(500000) - then it works.

What is the correct approach here? Is there any harm in putting in a massive number into the len parameter of the Read() method? Or is there a way of looping through the stream and constructing the object to pass to the business service?

Any advice much appreciated.

Product version: IRIS 2022.1

$ZV: IRIS for Windows (x86-64) 2022.1 (Build 209U) Tue May 31 2022 12:16:40 EDT [HealthConnect:3.5.0] [HealthConnect:3.5.0]

Discussion (4)2

Log in or sign up to continue

Vitaliy Serdtsev · Jan 31, 2023

Is there any harm in putting in a massive number into the len parameter of the Read() method?

You cannot read more than 3641144 characters from a stream to a string, as this is the limit for long strings: String Length Limit

From the documentation for the Read() method:

If no len is passed in, ie. 'Read()' then it is up to the Read implementation as to
how much data to return. Some stream classes use this to optimize the amount of
data returned to align this with the underlying storage of the stream.

So you can write like this:
Set inMsg = %request.Content.Read($$$MaxLocalLength)

Macros are defined in %msql.inc:

#define MaxStringLength 3641144
#define MaxLocalLength $zutil(96,39)

3 0

score 1 · Answer 1 · 2023-01-31T07:49:49-05:00

I believe there is a default length for the Read. If you pass in a number large enough or larger, it will read up to the length you specify. I see no harm in having a very large number.

score 0 · Answer 2 · 2023-02-03T03:55:05-05:00

Thanks for your reply, makes complete sense. I wasn't aware of $$$MaxLocalLength.

On looking into this further, it turns out I don't need to Read() at all. Just passing the %request.Content on like this:

Set tSC = tBusinessService.ProcessInput(%request.Content, .tBusinessServiceOutput)

Does the trick. Then in the business service class, using the OpenStream() method of %XML.Reader behaves as expected when deserialising to the class objects.

score 1 · Answer 3 · 2023-01-31T10:37:52-05:00

You can check for the size of the %request content reading "%request.Content.Size" and read the exact amount of bytes but keep in mind that , as Vitaliy already said, as long as you read a stream into a string you have to deal with the string Max Length.