New post
Question
· May 6, 2016

JSON Representation of a global structure

#Globals #JSON #Node.js #Caché

Hi:

I'm looking for some help with JSON notation for a global structure.  I need a way to represent a global structure in JSON where the global structure both has a value at a given node as well as sub-nodes.For example.

^Person=1

^Person(1)="Smith,John^M^1968-01-12"

^Person(1,"Address")="123 Main Street^Some City^CA^92627"

How would one go about representing this structure as JSON?

Thanks

jsonglobalprocessor.xml_.zip
Discussion (14)1
Log in or sign up to continue
Jenna Poindexter · May 6, 2016

Jeff- Thanks

What I am really looking for here is a generic way to represent any global structure as JSON.

In my example, the top node contains both data as well as sub-nodes.  

I'm not really looking for a way to represent a person in JSON, just a way to represent generic global structures taking into account the fact that a global node can have both data stored at that particular level as well as sub-nodes below it.

0 0
Stefan Wittmann · May 9, 2016

Kenneth, what is the use case for this? If you want to move Global data around, there are clearly optimized APIs for this task.

If you want to store JSON-like data you should be looking into the Document Data Model that is introduced with Caché 2016.2. The %ZEN.proxyObject has an interface for persisting dynamic data as well, but it clearly has been superseded by the new JSON capabilities available in Caché 2016.1 and up.

Without understanding your requirements, there are two potential approaches how you can serialize a Global in a JSON structure. One way is to represent the Global with a top-level object and all nodes in one big array. Each node is represented by an object, which includes the subscript as an array:

{"GlobalName":"Person","nodes":[

{"subscript":[],"value":1},

{"subscript":[1],"value":"Smith,John^M^1968-01-12"},

{"subscript":[1,"Address"],"value":"123 Main Street^Some City^CA^92627"}

]}

This serialization is easy to generate and consume as well. The complete subscript is available within each object, which makes the (de-)serialization logic simple. The downside of this approach is that you can only represent a Global as a whole. You could enhance this approach to start at a specific subnode, but you can not easily filter for specific sub-paths.

Another approach is to nest subnodes. This makes the (de-)serialization logic more complex as it has to be recursive:

{"GlobalName":"Person","nodes":[

{"subscript":[],"value":1,"children":[

{"subscript":[1],"value":"Smith,John^M^1968-01-12","children":[

{"subscript":["Address"],"value":"123 Main Street^Some City^CA^92627"}

]}

]}

]}

This allows you to easily skip sub-paths during processing, but the processing logic gets more complex. I realize there are more variations how you can represent a Global in JSON format, but I think there are two fundamental approaches, flat and nested.

Again it depends on what you want to achieve. I would be interested in learning more about what you have in mind.

1 0
Jenna Poindexter · Jun 3, 2016

Stefan-

My use case is that I have a Cache application (not object based) and I want to provide a set of REST services for accessing the raw global structures, all methods.  I want this REST interface to be generic in nature such that it can be used to read, set and kill any global node of any global.  In order to do this, I need an encoding method for sending and receiving the global data.  JSON seems the best way to package the data, thus I was looking for a good JSON structure that could represent the global structure.

0 0
Jenna Poindexter · Jun 3, 2016

And yes, there are many ways to accomplish this task from the Cache server side. 

The document data model doesn't really apply here as this is not a new application.

Once 2016.2 is released, we could just map the globals as objects using CacheSQLStorage and then use the .$toJSON() method to export to JSON at that point.

That's one of the great things about the Cache technology, there are so many different options and choices for doing the same thing.

This post is really just about a discussion on what the best way to represent a global structure in JSON, not the specifics of how to accomplish that in code.

0 0
Rob Tweed · May 29, 2016

The simple answer is that JSON can't naturally represent a node that is both an intermediate one and a leaf one , unless you introduce some arbitrary convention for defining a "special" node that represents a value also held by an intermediate node.  There's no "standard" convention for this, since, from a JSON point of view,  such a situation doesn't make sense.

The way I've tackled this with the Node.js ewd-document-store module (http://gradvs1.mgateway.com/download/ewd-document-store.pdf) which abstracts global structures directly to JSON, is to provide not only a JSON mapping (via the getDocument() and setDocument() methods, but also have a read/write value property for any specified "DocumentNode".  If you use getDocument(), you'll not see any values for intermediate nodes, but separately you can home in on such nodes individually and access their data via its value property.

A DocumentNode's "exists", "hasValue"  and "hasChildren" boolean properties can be used to determine the nature of each node.  In truly JSON-mapped global structures, hasValue and hasChildren will be mutually exclusive, whereas it's quite feasible for (typically legacy) global structures to have DocumentNodes where both are true.

Ideally the solution is to never have global structures with data values for intermediate nodes, but typically the problem is a legacy one.

1 0
Ted Peck · Jun 3, 2016

I'm looking into this now. My use case is exporting a global tree in human-friendly form to facilitate tracking and editing in source control. Specifically I want to manage the Trak Layout export globals.

Conceptually it seems straightforward. These are my observations:

- $List is isomorphic to JSON array.

- String is isomorphic to a JSON string, with JS escaping. For binary values the escaping is not very compact, but it should be unique.

- a global node is isomorphic to a JSON "object", with the addition of a "special" property for the node's top value.

There are only a couple glitches I've thought of so far and they seem surmountable. Please let me know if you think of problems besides these.

- Empty-string is a valid $List. I propose to always output empty-string as "" and not [].  This should round-trip all right in either case but "" is simpler.

- The name of the special top-value property must never collide with an actual subscript name. I propose to use "^_" for this name. If that ever appears as a literal subscript I would fail with an error and advise the user to override the default top-node name to something that never appears in their global.

- There might be problems with round-tripping binary data or obscure Unicode strings through the JS escaping and UTF8 transformation but I haven't thought of any specific problem yet.

I'm going to try it and I'll let you know how it turns out.  If you think of any reasons it can't work please let me know. Even if it  only works in restricted cases it might still be useful.

0 0
Ted Peck · Jun 10, 2016

Here is a working version of the input side as well.  It now has methods to export and import the Layout globals as well, and on import it compares the original with the imported version of the global.   The export and import distinguish between string and non-string types in a $List, which is necessary for a faithful round-trip.

I also ran into an issue where the JS output encoding erroneously escapes $C(11) as \v.  According to RFC 4627 it should not. 

However in order to successfully round-trip I had to add \v unescaping in the parser.

I'm not sure I've covered every possible round trip issue but it works for the Layout globals now.

The class that contains all the import and export methods is in the attached zip file. This is a rough cut but working.

Ted

tedjson1.zip

1 1
Jenna Poindexter · Apr 28, 2018

So a customer asked me a question about this and I decided to actually implement a mechanism to encode a global as JSON.  I haven't done the opposite which would be to take the encoded json and turn it back into a global, but the reverse is pretty simple

Here's the way I encode the Global as json  There is a top-level object with two properties, globalName and nodes.  globalName is a string and represents the actual global name, nodes is an array and contains an object for each node of the global which contains data.  To ensure that non-printable characters are handled properly,  I am using HTML Escaping to escape both the data and the subscript values

{

     "globalName":"^ISCSOAP",
     "nodes":[{
               "data":"io",
               "subscripts":[
                             ""Log""
                            ]
              },
              {
               "data":"c:\\temp\\SOAP.log",
               "subscripts":[
                             ""Log"",
                             ""LogFile""
                            ]
              }]
}

Here is an example of the class method that generates this output

Class ISC.JSONGlobalProcessor [ Not ProcedureBlock ]
{

ClassMethod Export(GlobalRoot As %String, Output JSON As %DynamicObject) As %Status [ ProcedureBlock = 0 ]
{
    if '$d(@GlobalRoot) quit $System.Status.Error(5001, "Nothing to export; "_GlobalRoot_" <undefined>")
    set root=$p(GlobalRoot,")",1,$l(GlobalRoot,")")-1),node=GlobalRoot s:root="" root=GlobalRoot
    set JSON=##class(%DynamicObject).%New()
    set JSON.globalName=$p(GlobalRoot,"(",1)
    set JSON.nodes=##class(%DynamicArray).%New()
    while $e(node,1,$l(root))=root {
        if $d(@node)#10 do ..addNode(node,.JSON)
        set node=$q(@node)
    }
    quit 1
}

ClassMethod addNode(node As %String, ByRef JSON As %DynamicObject) As %Status
{
    set nodeJSON=##class(%DynamicObject).%New()
    set data=@node,nodeJSON.data=##class(%CSP.Page).EscapeHTML(data)
    set subscripts=$p(node,"(",2,999),subscripts=$p(subscripts,")",1,$l(subscripts,")")-1)
    if ""'=subscripts {
        set nodeJSON.subscripts=##class(%DynamicArray).%New()
        set cp=1
        for {
            q:cp>$l(subscripts,",")
            set subscript=$p(subscripts,",",cp)
            f  {
                q:$l(subscript,"""")#2
                set cp=cp+1,subscript=subscript_","_$p(subscripts,",",cp)
            }
            set subArray=$i(subArray),subArray(subArray)=subscript
            set cp=cp+1
        }
        for i=1:1:subArray do nodeJSON.subscripts.%Push(##class(%CSP.Page).EscapeHTML(subArray(i)))
    }    
    do JSON.nodes.%Push(nodeJSON)
}

}

To call this code you can do the following

set sc=##class(ISC.JSONGlobalProcessor).Export($na(^SAMPLE.PersonD),.json)

Once you have the global encoded in a JSON object, you can output that JSON by:

do json.%ToJSON()
0 0