Load JSON from a file

Question

Question

Pedro Borges · Aug 14, 2017

Hello -- Is there a way to load a JSON from a file.

I have a "JSON" file which has a sting 1035164 long with the following format:

[
    {
        "id": "12345",
        "title": "John Smith",
        "image_uri": "https://<some URL>",
        "image_timestamp": "1496781334",
        "image_url":"https://<some URL>",
        "is_restricted_under_18_only": false,
        "is_restricted_adult_only": false
    },
...
    {
        "id": "67890",
        "title": "Mary Jane",
        "image_uri": "https://<some URL>",
        "image_timestamp": "1496781334",
        "image_url": "https://<some URL>",
        "is_restricted_under_18_only": false,
        "is_restricted_adult_only": false
    }
]

There are ~ 4839 entries.

Is it possible to parse a JSON string by loading a from a file?

I tried this...

  S filename="/tmp/pictures.json"
  S stream=##class(%Stream.FileCharacter).%New()
  S sc=stream.LinkToFile(filename)
  I ('sc) W "Error on linking file "_filename,! W
  try {
    set obj=##class(%DynamicAbstractObject).%FromJSON(stream)
  } catch ex {
    w "Error. Unable to parse file "_filename,!
    w "Error type   "_ex.Name,!
    w "Error code   "_ex.Code,!
    w "Error location "_ex.Location,!
    set obj=""
  }
  q obj
  Q

... but I get the following error:

Error. Unable to parse file /epicdc/data/pictures.json
Error type <CLASS DOES NOT EXIST>
Error code 150
Error location fromFile+7^XJSON

Thanks!

Discussion (15)1

Log in or sign up to continue

Eduard Lebedyuk · Aug 14, 2017

Sure:

Set filename = "/tmp/pictures.json"
Set array = [].%FromJSON(filename)

0 0

score 0 · Answer 1 · 2017-08-14T11:36:32-04:00

I get the error: <METHOD DOES NOT EXIST> *%FromJSON,%Library.Array

I am running: Cache for UNIX (IBM AIX for System Power System-64) 2016.1.4 (Build 104) Mon Jun 19 2017 16:34:17 EDT

score 0 · Answer 2 · 2017-08-14T11:44:49-04:00

Dan Pasco · Aug 14, 2017

sorry - not that the JSON syntax is different - the class and method names were different!

0 0

score 0 · Answer 3 · 2017-08-14T11:57:34-04:00

Eduard Lebedyuk · Aug 14, 2017

It's $fromJSON on 16.1:

Set filename = "/tmp/pictures.json"
Set array = [].$fromJSON(filename)

0 0

score 0 · Answer 4 · 2017-08-14T12:23:15-04:00

I am getting error: <THROW>zfromJSON+28^%Library.AbstractObject.1 *%Exception.General Parsing error 3 Line 1 Offset 1

Below is the "raw" file I am using (stripped down to just two entries)...

/tmp/small.json

[{"id": "12345","title": "John Smith","image_uri": "https://<some URL>","image_timestamp": "1496781334","image_url":"https://<some URL>","is_restricted_under_18_only": false,"is_restricted_adult_only": false},{"id": "67890","title": "Mary Jane","image_uri": "https://<some URL>","image_timestamp": "1496781334","image_url": "https://<some URL>","is_restricted_under_18_only": false,"is_restricted_adult_only": false}]

I tried to run this code...

    N filename,ex
    S filename="/tmp/small.json"
    try {
    S array=[].$fromJSON(filename)
    } catch ex {
    W !,"    Type: "_ex.Name
    W !,"    Code: "_ex.Code
    W !,"Location: "_ex.Location
    }
    Q

This is the output...

Type: Parsing error
Code: 3
Location: Line 1 Offset 1

Is the file malformed? What is wrong with line 1 offset 1?

Pedro

score 1 · Answer 5 · 2017-08-14T13:45:27-04:00

Seems like ability to parse files directly was introduced after 2016.1 (I tested in 2017.1 and it works. Your initial code works after replacing %From with $from:

 S filename="/tmp/pictures.json"
  S stream=##class(%Stream.FileCharacter).%New()
  S sc=stream.LinkToFile(filename)
  I ('sc) W "Error on linking file "_filename,! W
  try {
    set obj= [].$fromJSON(stream)
  } catch ex {
    w "Error. Unable to parse file "_filename,!
    w "Error type   "_ex.Name,!
    w "Error code   "_ex.Code,!
    w "Error location "_ex.Location,!
    set obj=""
  }
  q obj
  Q

I would recommend upgrading from 2016.1 to 2017.1.

score 0 · Answer 6 · 2017-08-14T14:48:40-04:00

Assuming you have %request object what happens when you call:

 S array=[].$fromJSON(%request.Content)

on small and big requests?

score 0 · Answer 7 · 2017-08-14T16:44:01-04:00

I do not have the %request object... the call to retrieve the JSON string is not using Cache's %Net.HttpRequest.

At least that is what I understood you asked me to do. 8-)

How the snippet of code (using $fromJSON) would be modified to incorporate your suggestion?

Pedro

score 0 · Answer 8 · 2017-08-14T21:50:51-04:00

Jon -- Yes, there were two entries that had invalid characters...Thanks for the tip!

Eduard -- Thank you so very much for helping me on this!

score 0 · Answer 9 · 2017-08-15T03:58:55-04:00

Pedro - good to see that DC members were able to help. Now please set the "accepted answer" checkmark that only you (as OP) are able to set. If you're unsure where to look, see my screenshot below:

score 1 · Answer 10 · 2017-08-14T16:47:32-04:00

"Escaped hex sequence too large" means that a hexadecimal escape sequence couldn't be decoded. I suspect that you've encountered a Unicode character that can't be handled on your eight-bit instance.

score 0 · Answer 11 · 2017-08-14T14:35:54-04:00

Hello!

I made good progress and the code works with small files (less than 32K characters).

When I try to load the full file I get the following error...

Error. Unable to parse file /tmp/pictures.json
Error type   Escaped hex sequence too large
Error code   7
Error location Line 1 Offset 29356

For example, when I use this snippet of code...

    n
    r !,"Filename: ",filename Q:filename=""
    s stream=##class(%Stream.FileCharacter).%New()
    s sc=stream.LinkToFile(filename)
    i ('sc) W "Error on linking file "_filename,! Q
    try {
    S array=[].$fromJSON(stream)
    S iter=array.$getIterator()
    while iter.$getNext(.key,.value) { write !,"key "_key_":"_value."id",! }
    ;
    } catch ex {
    w "Unable to parse file "_filename,!
    w "Error type: "_ex.Name,!
    w "Error code: "_ex.Code,!
    w "Error location: "_ex.Location,!
    set obj=""
    }
    Q

It will work for small.json but error-out for file all.json (Error: Escaped hex sequence too large)...

-rw-------    1 pborges    user    1037250 Aug 14 14:08 /tmp/all.json
-rw-------    1 pborges    user      28750 Aug 14 14:07 /tmp/small.json

So, the question is... how to parse a really long JSON string? (>1M+ characters)

BTW, the result I get from the REST call is stored in a global and I am saving it to a "flat" file at the OS level to concatenate in a long string.

What I tried 1st was to parse the global, but there is not a clear end for each entry.

For example...

ZW ^result(1)
^result(1)= "{""id"":""0107454"",""title"":""John Smith"",""image_uri"":""https://<some URL>"",""image_timestamp"":""1496781334"",""image_url"":""https://<so

ZW ^result(2)
^result(2)= "me URL>"",""is_restricted_under_18_only"":false,""is_restricted_adult_only"":false},{""id"":""01135433"" ...

Because the node #1 reached 32,000 characters, a new node gets created to have the remainder of the result (I have 42 nodes in total).

score 0 · Answer 12 · 2017-08-14T11:35:03-04:00

I get the error: <METHOD DOES NOT EXIST> *%FromJSON,%Library.Array

I am running the following Cache' version: Cache for UNIX (IBM AIX for System Power System-64) 2016.1.4 (Build 104) Mon Jun 19 2017 16:34:17 EDT

score 0 · Answer 13 · 2017-08-14T11:44:02-04:00

This should work. I assume the code you included resides in a routine named 'XJSON' and in a function named 'fromFile'? I can guess that the class referenced is %DynamicAbstractObject and that class should certainly exist in versions 2016.2 and later. In version 2016.1, this class was known as %Library.AbstractObject.

In 2016.1, the JSON syntax was much different. The correct expression for that version is:

 set obj = [].$fromJSON(stream)

score 2 · Answer 14 · 2017-08-14T15:54:40-04:00

This tool helps to read and format JSON data JSON Formatter , It also loads file from desktop or load URL from the internet.