Robert Cemper · Jan 30 1m read

JSONfile-to-Global #2

The power of the presented tool becomes visible with a large JSON file.
The example I have prepared is composed of the daily results of AoC 2023.
Anonymization makes it usable for demo purposes.
As the original, it is an uninterrupted sequence of bytes without any NewLine or similar.
6 660 895 Bytes raw in sequence (as by NTFS). With this content (shrinked):

It is loaded into an %DynamicObject using the method %FromJSONfile.
If you try to investigate that object using ZWRITE  you end with <MAXSTRING>
And this is perfectly correct. The maximum string size is exceeded.

If you iterate down to the embedded JSON objects this conflict is gone as they are only ~30kB
The mystery is exercised by %Stream.DynamicCharacter which acts in the background.

The resulting Global is as huge as its source.
But now you can navigate and examine the individual elements.
A snapshot from SMP in the resulting Global top subscript

and total:

I had to run this from Terminal as SMP failed after a few 10000 lines and used this single line instead:

s g="^json" f i=1:1 s g=$q(@g,1,v) q:g=""  w i,":",?8,g,?78,"= ",v,!



0 118
Discussion (1)1
Log in or sign up to continue

interesting solution.
string limits are always a pain.