Json with field exceeding maximum string length

Question

Question

Jiri Svoboda · Nov 9, 2018

Hi all,

I have an incoming JSON message with a string field exceeding in length the 'caché long-string limit' of 3641144 characters.

Using {}.%FromJSON(instream) I am able to create %DynamicObject properly. However, I am unable to access the long property, as in every assignment, I get the <MAXSTRING> error.

Is there any way for me to obtain contents of the 'too long' string field as a stream?

Thanks

Jiri

Discussion (3)2

Log in or sign up to continue

Jiri Svoboda · Nov 9, 2018

Thanks for the reply. I eventually solved my task via $ZF(-1) and the 'jq' tool.

0 0

score 0 · Answer 1 · 2018-11-09T15:33:45-05:00

This came up in an earlier post:
https://community.intersystems.com/post/fromjson-doesnt-support-stream-p...

I believe this will be addressed in a future release.

score 0 · Answer 2 · 2018-11-28T19:45:58-05:00

Let us assume that you are using the %DynamicAbstractObject class and its subclasses, which include %DynamicArray and %DynamicObject (instead of using the legacy support for JSON in the %ZEN package.)

If you have an element of a JSON array or JSON object that contains more characters than are supported by an ObjectScript string then the way to manipulate that element is to send it to a %File or a %Stream. In the present releases of Caché and IRIS Data Platform the only way to do that is to use the %ToJSON(outstream) method call. If the element in question is in a nested sequence of JSON elements then you can first use a sequence of %Get method calls to access the closest containing %DynamicArray or %DynamicObject. You can then use %ToJSON(outstream) method call to create a %Stream containing the textual JSON of that entire object. Then process the contents of that %Stream using the Read(len,...)/Readline(len,...) methods, with the 'len' parameter chosen to provide blocking that will prevent exceeding ObjectScript string length limits. Some JSON parsing will be necessary to find the beginning and the ending of the JSON formatted string containing the element in question so that the required element can be copied as the only contents of a new %Stream.

If you can control how the JSON is formatted then you can choose to enclose the long JSON string element inside a JSON array containing just that one element--the long string value requiring additional manipulation. Parsing such an %DynamicArray element would be easy--send the Array to a %Stream using %ToJSON and then ignore the first two characters, [", and the last two characters, "], of the %Stream. What remains is contents of the string element with a few special characters encoded using \-sequences. If you cannot control how the JSON is formatted then it still might be possible to use knowledge of the application that is using JSON representation to short cut the parsing issues.

In a future InterSystems product release there will certainly be an extension to the %Get method that will allow any %DynamicAbstractObject element (including string and numeric elements) to be sent to a %Stream using either Raw String representation or using JSON representation. You will also be able to copy such an element from one JSON array/object into another JSON array/object without needing to copy the characters to a %Stream that is using an external representation.