How to remove escape characters from JSON

Question

Question

Julie Marulappa · Apr 18, 2019

I am still new to cache objectscript and am trying to figure out how I would go about removing the escape characters from my JSON below. When I call the $toJSON method it's adding the "\" character in the file path.

{"FileStatus":"P","Path":"\/somepath\/test\/test123\/filename.txt","InterchangeOID":"100458"}

My Code:

set tJSONFile = {"FileStatus":pRequest.FileStatus,"Path":pRequest.Path,"InterchangeOID":pRequest.InterchangeOID}

set tJSONFile = tJSONFile.$toJSON()

Thanks.

Discussion (11)3

Log in or sign up to continue

Sean Connelly · Apr 23, 2019

There is nothing wrong with escaping a Solidus character. The JSON specification states that a Solidus "may" be escaped but does not have to be. You should find that most decoders will unescape the Solidus correctly, so you could just leave it as is.

0 0

score 0 · Answer 1 · 2019-04-18T17:50:00-04:00

What does this code print?

set ^dbg = $lb(pRequest.FileStatus, pRequest.Path, pRequest.InterchangeOID)

zw ^dbg

score 0 · Answer 2 · 2019-04-23T06:44:14-04:00

The only danger here is if the reverse solidus is itself escaped e.g. \\/, the replace would effectively remove the reverse solidus from the decoded JSON.

score 0 · Answer 3 · 2019-04-25T19:27:15-04:00

Agreed that there is nothing wrong with escaping a solidus. However, saying "most decoders" will unescape the solidus correctly" is not quite correct. "*ALL* decoders will unescape the solidus correctly." If a decoder does not correctly unescape the solidus then it is *NOT* a JSON decoder. The %FromJSON method in the %DynamicObject class in IRIS and Caché versions after 2016.1 will do the following:

USER>write x
{"FileStatus":"P","Path":"\/somepath\/test\/test123\/filename.txt","InterchangeOID":"100458"}
USER>set y={}.%FromJSON(x)

USER>write y.Path
/somepath/test/test123/filename.txt

So %FromJSON is the method that breaks JSON apart and removes all escaping from JSON string values. (I assume that $fromJSON will do the same in Caché version 2016.1. The JSON manipulation added to version 2016.1 was changed significantly in all future releases. Most JSON manipulation programs written in version 2016.1 will require rewriting when porting to a later release.)

score 0 · Answer 4 · 2019-05-02T14:05:45-04:00

I removed the $ZCONVERT and it is working fine. I think I was more confused on how JSON worked, and after reading thru the comments on this thread it makes more sense to me now. Thank you for your help.

score 0 · Answer 5 · 2019-04-23T17:57:37-04:00

I wouldn't recommend this solution, it will cause problems.

It gets a bit confusing talking about strings, so lets refer to a serialised object as a stringy, and a property of a JSON object of type string as a string.

The $ZCONVERT function should be used to decode JSON strings (properties of a JSON object). It should not be used to decode a stringy (a serialised JSON object).

It works in your use case because it sees the stringy as a string and since there are only unwanted reverse solidus's to remove it looks like its working the way you want it to.

Let's say at a later date someone adds a description property, and one of those descriptions has the value

A "special" file

In the JSON it would be escaped like so...

{"FileStatus":"P","Path":"\/somepath\/test\/test123\/filename.txt","InterchangeOID":"100458","Desc":"A \"special\" file"}

This gets passed into $ZCONVERT and you end up with...

{"FileStatus":"P","Path":"/somepath/test/test123/filename.txt","InterchangeOID":"100458","Desc":"A "special" file"}

The path looks good, but now the description is invalid.

Is there a technical reason for wanting to remove the reverse solidus from the path?

score 0 · Answer 6 · 2019-04-19T10:11:25-04:00

$toJSON() escaped forward slash unnecessarily in 2016.1. It's valid JSON, but unconventional. %ToJSON() doesn't do this in 2016.2 and later. Depending on your data, you may be able to get away with $replace(json,"\/","/"). To handle all cases would be a bit more work.

score 0 · Answer 7 · 2019-04-26T08:15:02-04:00

I can't presume to know all decoders will unescape the solidus correctly.

Escaping the solidus is not a strict rule in the specification and there may be implementations that do not unescape it.

I think it is safer to say "most" than "all" and not place any false confidence on the next developer to read this post. I would rather they execute due diligence and be 100% sure that the third party libraries they are using will handle the 2016.1 output.

score 0 · Answer 8 · 2019-04-27T18:03:53-04:00

I agree that comments here will have little effect on developers other than InterSystems developers or InterSystems customers. However, processing of JSON is controlled by the European standard "ECMA-404 The JSON Data Interchange Format". If a JSON decoder deliberately ignores this standard specification then it is not a *JSON* decoder. If any customer discovers that InterSystems's %FromJSON method is violating the ECMA-404 standard then I am sure InterSystems will fix this situation. I suspect that third-party developers would also fix such a violation of ECMA-404. The ECMA-404 document actually contains examples of 4 different strings that are identical to "/". ECMA-404 does not require that JSON encoders generate the "\/" string syntax but this standard does require that all decoders accept this string syntax.

score 0 · Answer 9 · 2019-05-02T12:03:24-04:00

Julie Marulappa · May 2, 2019

Ok, yes I didn't think of that and agree with you.

0 0

score 0 · Answer 10 · 2019-04-23T12:02:01-04:00

I was able to do this: set tJSONFile=$ZCONVERT(tJSONFile,"I","JSON"), and it removed the escape characters.

Result:

{"status":"P","path":"/somepath/test/test123/filename.txt","interchangeOid":"100811"}