Parsing null values in JSON

Question

Question

Lucas Macori · Jun 11, 2021

#JSON #Caché

Hello everyone,

I'm creating a REST API with InterSystems Ensemble.

I already have a POST route that parses JSON content into an object defined in a class.

To parse the JSON content, I'm using the JSONStreamToObject method in the Ens.Util.JSON class

Here is how I use it:

 Set tSC = ..JSONStreamToObject(%request.Content, .tObject, "MyClass", 1)

While the route works and does what it should, in theory.

I noticed that when passing empty string or null values inside the JSON content, the values do not appear at all in the object's instance.

Therefore, there is not difference between a null value and an undefined value.

For exemple, if I pass the following JSON string when calling my REST route:

{
    "a": "1",
    "b": 2,
    "c": "",
    "d": null
}

The values C and D are completely ignored. As if they were undefined:

<MyClass>
    <a>1</a>
    <b>2</b>
</MyClass>

This is an issue because empty or null values are not used the same way as given values like A or B for instance.

I already tried setting XMLIGNORENULL to 1 in the class, but it does not work for JSON.

Even though, I noticed that it changed the XML parsing of the class by setting all values in the XML export to empty strings.

Did anyone ever experience a similar issue ?

Thank you,

Lucas

Product version: Caché 2018.1

$ZV: Cache for Windows (x86-64) 2018.1.4 (Build 505_1U) Thu May 28 2020 10:01:40 EDT

Discussion (1)2

Log in or sign up to continue

Steven Hobbs · Jul 22, 2021

The JSON classes in Ens.Util.JSON, %ZEN.Auxiliary.json* and %JSON.* all contain methods that convert JSON representation to/from ObjectScript classes. Once you have an ordinary ObjectScript Class then you are using ObjectScript data types for Property values. The JSON null is usually converted to "" (null string). Also, ordinary Property variables of an ObjectScript Class are never undefined but are automatically initialized to "" (the null string). [[ An exception is [MULTIDIMENSIONAL] Properties which can be undefined but by default such Property variables do not participate in %Save() and JSON/XML export/import operations. ]] SQL operations involving Class properties treat "" (the null string) as the SQL NULL value and SQL assumes a Class Property containing the ObjectScript string $C(0) is the empty string.

[[ Although the original question involved Caché and not IRIS, IRIS has signifcantly more complete support for the %DynamicAbstractObject classes so my examples will use IRIS. If possible, I recommend upgrading to IRIS. ]]

There is the class %Library.DynamicAbstractObject and its subclasses %DynamicArray and %DynamicObject that can contain elements which can either be JSON values or ObjectScript values. The ObjectScript statement:

USER>SET x={"a":null,"b":"","e":1.0,"f":"1.0","g":(00.1),"h":($c(0))}

makes x be a %DynamicObject oref where element a is a JSON null and where element "g" is the ObjectScript number .1 and element "h" is the ObjectScript string $c(0). Note that if an ObjectScript expression is a %DynamicObject constructor then ObjectScript parses the elements of that constructor using JSON syntax except for the extension to constructor syntax where an element inside round parentheses is parsed as an ObjectScript run-time expression generating an ObjectScript value.

You can convert a %DynamicObject to JSON string representation and any ObjectScript valued element will be converted to JSON representation.

[[ Note that JSON does not support certain ObjectScript values: $double("NAN"), $double("INFINITY") and orefs that are not a subclass of %DynamicAbstractObject. A %DynamicAbstractObject containing such an ObjectScript element cannot be converted to JSON. ]]

USER>WRITE x.%ToJSON()
{"a":null,"b":"","e":1.0,"f":"1.0","g":0.1,"h":"\u0000"}

Evaluating a %DynamicObject element in an ObjectScript expression converts that element to an ObjectScript value.

USER>ZWRITE x.a, x.b, x.c, x.e, x.f, x.g, x.h
""
""
""
1
"1.0"
.1
$c(0)

Notice that the undefined element x.c is converted to the ObjectScript null string. You never get an error evaluating x.%Get(key) for any value of the expression key as every undefined element in a %DynamicObject has the value of the null string. Also, x.a, which contains a JSON null, is converted to the ObjectScript null string. The JSON treatment of undefined elements and the ObjectScript treatment of undefined Properties means that when we convert an ordinary ObjectScript class to either XML or JSON then we can skip converting a Property with the null string value as converting JSON or XML back to an ordinary class object will result in all unrepresented properties getting the value of the null string.

If you need to know if a %DynamicObject element is JSON null, null string or undefined then evaluating the %GetTypeOf(key) will tell you that.

USER>ZWRITE x.%GetTypeOf("a"),x.%GetTypeOf("b"),x.%GetTypeOf("c"),x.%GetTypeOf("e")
"null"
"string"
"unassigned"
"number"

The %FromJSON(stream)/%ToJSON(stream) methods will let you read/write JSON representation from/to a %Stream.

[[ Things that only work in IRIS follows. ]]

The size of the %DynamicArray/%DynamicObject class objects is limited only by the amount of memory available to your process. A string valued %DynamicObject element can be significantly longer than the maximum length supported by ObjectScript string values. If you have such a long string element then you will have convert that element to an ObjectScript %Stream in order to manipulate it in ObjectScript.

USER>SET stream=x.%Get(key,,"stream") ;; Note 3 arguments with 2nd argument missing

will generate an in-memory, readonly %Stream that can be copied to a Global or File %Stream or can be examined by reading that string in small pieces.

In recent IRIS releases you can do

USER>SET binarystream=x.%Get(key,,"stream<base64")

which will convert a base64 encoded element into a readonly binary %Stream. You can do the reverse conversion by evaluating x.%Set(key,binarystream,"stream>base64"). See the the Class Reference documentation pages for the %Library.%DyanmicAbstractObject class and its %DynamicArray and %DynamicObject subclasses for more details.

0 0