Question
· Aug 14, 2017

Load JSON from a file

Hello -- Is there a way to load a JSON from a file.

I have a "JSON" file which has a sting 1035164 long with the following format: 

[
    {
        "id": "12345",
        "title": "John Smith",
        "image_uri": "https://<some URL>",
        "image_timestamp": "1496781334",
        "image_url":"https://<some URL>",
        "is_restricted_under_18_only": false,
        "is_restricted_adult_only": false
    },
...
    {
        "id": "67890",
        "title": "Mary Jane",
        "image_uri": "https://<some URL>",
        "image_timestamp": "1496781334",
        "image_url": "https://<some URL>",
        "is_restricted_under_18_only": false,
        "is_restricted_adult_only": false
    }
]

There are ~ 4839 entries. 

Is it possible to parse a JSON string by loading a from a file?

I tried this...

  S filename="/tmp/pictures.json"
  S stream=##class(%Stream.FileCharacter).%New()
  S sc=stream.LinkToFile(filename)
  I ('sc) W "Error on linking file "_filename,! W
  try {
    set obj=##class(%DynamicAbstractObject).%FromJSON(stream)
  } catch ex {
    w "Error. Unable to parse file "_filename,!
    w "Error type   "_ex.Name,!
    w "Error code   "_ex.Code,!
    w "Error location "_ex.Location,!
    set obj=""
  }
  q obj
  Q 

... but I get the following error:

Error. Unable to parse file /epicdc/data/pictures.json
Error type   <CLASS DOES NOT EXIST>
Error code   150
Error location fromFile+7^XJSON

Thanks!

Discussion (15)1
Log in or sign up to continue

I am getting error:  <THROW>zfromJSON+28^%Library.AbstractObject.1 *%Exception.General Parsing error 3 Line 1 Offset 1

Below is the "raw" file I am using (stripped down to just two entries)...

/tmp/small.json

[{"id": "12345","title": "John Smith","image_uri": "https://<some URL>","image_timestamp": "1496781334","image_url":"https://<some URL>","is_restricted_under_18_only": false,"is_restricted_adult_only": false},{"id": "67890","title": "Mary Jane","image_uri": "https://<some URL>","image_timestamp": "1496781334","image_url": "https://<some URL>","is_restricted_under_18_only": false,"is_restricted_adult_only": false}]

I tried to run this code...

    N filename,ex
    S filename="/tmp/small.json"
    try {
    S array=[].$fromJSON(filename)
    } catch ex {
    W !,"    Type: "_ex.Name
    W !,"    Code: "_ex.Code
    W !,"Location: "_ex.Location
    }
    Q
   

This is the output...

    Type: Parsing error
    Code: 3
Location: Line 1 Offset 1

 

Is the file malformed? What is wrong with line 1 offset 1?

Pedro

Seems like ability to parse files directly was introduced after 2016.1 (I tested in 2017.1 and it works. Your initial code works after replacing %From with $from:

 S filename="/tmp/pictures.json"
  S stream=##class(%Stream.FileCharacter).%New()
  S sc=stream.LinkToFile(filename)
  I ('sc) W "Error on linking file "_filename,! W
  try {
    set obj= [].$fromJSON(stream)
  } catch ex {
    w "Error. Unable to parse file "_filename,!
    w "Error type   "_ex.Name,!
    w "Error code   "_ex.Code,!
    w "Error location "_ex.Location,!
    set obj=""
  }
  q obj
  Q 

I would recommend upgrading from 2016.1 to 2017.1.

Hello!

I made good progress and the code works with small files (less than 32K characters).

When I try to load the full file I get the following error...

Error. Unable to parse file /tmp/pictures.json
Error type   Escaped hex sequence too large
Error code   7
Error location Line 1 Offset 29356

For example, when I use this snippet of code...

    n
    r !,"Filename: ",filename Q:filename=""
    s stream=##class(%Stream.FileCharacter).%New()
    s sc=stream.LinkToFile(filename)
    i ('sc) W "Error on linking file "_filename,! Q
    try {
    S array=[].$fromJSON(stream)
    S iter=array.$getIterator()
    while iter.$getNext(.key,.value) { write !,"key "_key_":"_value."id",! }
    ;
    } catch ex {
    w "Unable to parse file "_filename,!
    w "Error type: "_ex.Name,!
    w "Error code: "_ex.Code,!
    w "Error location: "_ex.Location,!
    set obj=""
    }
    Q

It will work for small.json but error-out for file all.json (Error: Escaped hex sequence too large)...

-rw-------    1 pborges    user    1037250 Aug 14 14:08 /tmp/all.json
-rw-------    1 pborges    user      28750 Aug 14 14:07 /tmp/small.json

So, the question is...  how to parse a really long JSON string? (>1M+ characters)


BTW, the result I get from the REST call is stored in a global and I am saving it to a "flat" file at the OS level to concatenate in a long string.

What I tried 1st was to parse the global, but there is not a clear end for each entry.

For example...

ZW ^result(1)
^result(1)= "{""id"":""0107454"",""title"":""John Smith"",""image_uri"":""https://<some URL>"",""image_timestamp"":""1496781334"",""image_url"":""https://<so

ZW ^result(2)
^result(2)= "me URL>"",""is_restricted_under_18_only"":false,""is_restricted_adult_only"":false},{""id"":""01135433"" ...

Because the node #1 reached 32,000 characters, a new node gets created to have the remainder of the result (I have 42 nodes in total).

This should work. I assume the code you included resides in a routine named 'XJSON' and in a function named 'fromFile'? I can guess that the class referenced is %DynamicAbstractObject and that class should certainly exist in versions 2016.2 and later. In version 2016.1, this class was known as %Library.AbstractObject.

In 2016.1, the JSON syntax was much different. The correct expression for that version is:

 set obj = [].$fromJSON(stream)