New post
Question
· Jun 23

Long JSON Error {}.%FromJSONFile(file)

#JSON #InterSystems IRIS

I am working on JSON and want to be prepared to handle large Objects. I try this code:

ClassMethod MaxLen() As %Status

{

    set longStr=""

    for i=1:1:$SYSTEM.SYS.MaxLocalLength() { set longStr = longStr_"x" }

    write "Maximum string length = "_$LENGTH(longStr)

    ;

    set longObject = {"a":(longStr),"b":(longStr)}

    set file=##class(%File).%New("/tmp/longObjectFile.txt")

    do file.Open("WSN")

    do longObject.%ToJSON(file)

    do file.Close()

    ;

    do file.Open("RS")

    set newObject = {}.%FromJSONFile(file)

    write !,"Property newObject.a is "_$LENGTH(newObject.a)_" characters long."

    quit 1

}

I get error

Maximum string length = 3641144
<THROW>%FromJSONFile+18^%Library.DynamicAbstractObject.1 *%Exception.General Premature end of data 12 Line 1 Offset 0 

Product version: IRIS 2022.3
$ZV: IRIS for UNIX (Ubuntu Server LTS for x86-64 Containers) 2022.3 (Build 606U) Mon Jan 30 2023 09:27:12 EST
Discussion (5)3
Log in or sign up to continue
Julius Kavay · Jun 23 
Class DC.BigJSON Extends %RegisteredObject
{

ClassMethod Test(filename)
{
	if ..SaveToFile(..MakeJSON(), filename) {
		write "Save OK",!
		write "Size ",##class(%File).GetFileSize(filename),!
		
		set input=##class(%File).%New(filename)
		set sts=input.Open("RS")
		if sts {
			set json={}.%FromJSON(input)
			set iter=json.%GetIterator()
			while iter.%GetNext(.key, .val) {
				write "key=",key," size=",$l(val)," data=",$e(val,1,10)_"...",!
			}
		} else  { write $system.Status.GetOneErrorText(sts),! }
	}
}

ClassMethod MakeJSON()
{
	set obj={}
	set obj.text1=$tr($j("",3600000)," ","a")
	set obj.text2=$tr($j("",3600000)," ","b")
	set obj.text3=$tr($j("",3600000)," ","c")
	quit obj
}

ClassMethod SaveToFile(obj, filename)
{
	set file=##class(%File).%New(filename)
	set sts=file.Open("wnu")
	if sts {
		do obj.%ToJSON(file)
		do file.Rewind()
		use file.Name 
		do file.OutputToDevice()
		do file.Close()
		quit 1
	} else { quit sts }
}

}

The size shouldn't be a problem


USER>do ##class(DC.BigJSON).Test("/tmp/test1.txt")
Save OK
Size 10800034
key=text1 size=3600000 data=aaaaaaaaaa...
key=text2 size=3600000 data=bbbbbbbbbb...
key=text3 size=3600000 data=cccccccccc...
1 0
Enrico Parisi · Jun 23

I'm a little puzzled, according to documentation the method %FromJSONFile() it's not implemented in version 2022.3, it has been implemented in 2024.1, so I don't understand why you get that error, I'd expect a <METHOD DOES NOT EXISTS> error, but I don't have version 2022.3 at hand to test.

In any case, for IRIS versions that implement the %FromJSONFile() method, please note that the parameter to pass is the filename, not a %File object instance.

Form class reference of the %Library.DynamicAbstractObject class the first parameter is documented as:

filename A file URI where the source can be read.

So, the code should be:

set newObject = {}.%FromJSONFile("/tmp/longObjectFile.txt")

1 0
Steven Hobbs · Jun 24

The %FromJSONFile(file) method does not take a %File object as an argument but instead takes a string containing the name of file.  You need to replace

set newObject = {}.%FromJSONFile(file)

with one of

set newObject = {}.%FromJSONFile("/tmp/longObjectFile.txt")
set newObject = {}.%FromJSON(file)

Originally, %FromJSON(input) would accept the 'input' argument as a %File object, a %Stream object, a JSON text string or a filename text string.  However, accepting an argument string as either JSON or a filename was considered to be security issue because a malicious user could be requested to enter a json-string but, being malicious, the user would instead enter a filename-string, or vice-versa.  As a result %FromJSON no longer accepts filename strings while %FromJSONFile only accepts filename strings.

No longer an issue, but it would be possible that legal VMS filename syntax could also be legal JSON text string syntax and such an argument string might be ambiguous on a VMS system.  But IRIS no longer supports VMS and Caché for VMS did not support Dynamic Objects.

0 0