Question
Jerry Petrole · Mar 22, 2021

Unknown error thrown with large file conversion to dynamic object.

Hello

I am creating an import tool to convert a client's JSON data into IRIS classes.   The sample file is over half a gig.   I am copying the data into an instance of %Stream.FileCharacter;.    My first few attempts worked fine.   However I started getting an error thrown when I try to create a DynamicAbstractObject using the  %FromJSON method.     See the code below.   The error code given is not in the documentation, at least not the doc that I was searching.   

       "READ error while reading input stream, Line 5121169 Offset 23, error code 10"

Can anyone shed light on this error code?

I assume this has something to do with the file size but I'm not sure since I had success on the first few runs.     Any help is much appreciated!

Thank you!

 

 

// Import the stream data into a dynamic object and then into the object properties w "Creating Dynamic Object",!
set jsonError=""

try {

    set DynObj={}.%FromJSON(StreamObj.DataStream)

}

catch errobj 

{

    write errobj.Name_", "_errobj.Location_", error code "_errobj.Code,!
    set jsonError=1
    quit

quit:jsonError

Product version: IRIS 2019.1
$ZV: IRIS for Windows (x86-64) 2019.1.1 (Build 612_0_19971U) Mon Jun 1 2020 18:51:40 EDT
00
2 0 3 89
Log in or sign up to continue

Replies

Try

set DynObj={}.%FromJSON(filename)

Mr. Petrole,

Per the documentation here: https://docs.intersystems.com/latest/csp/documatic/%25CSP.Documatic.cls?&LIBRARY=%25SYS&PRIVATE=1&CLASSNAME=%25Library.DynamicAbstractObject#%25FromJSON

is it possible that at some point your data stream is not encoded as UTF-8? That seems to be a requirement of the %FromJSON method, and if your data string isn't UTF-8 you may need to employ a call to $ZCONVERT, or for a "non-compliant" stream you may need to set the TranslateTable attribute of the stream to "UTF8."

Hope this helps!

Since you've had successful "runs", I wonder if it's a memory issue.  You could increase your partition size or

Set $zstorage=$zstorage*10  // temporarily increase partition size

And see if that helps (ie changes the number of lines processed)