Best practices for JSON transformation in IRIS interoperability
What are best practices for JSON transformation in IRIS interoperability? This is for a non-healthcare use case, so any tools we happen to have around FHIR might not be available. The motivating use case is trimming down a verbose and needlessly complex REST API response to feed to an LLM - trying to reduce token usage and maybe get better results from less noisy data.
Specifically, I'm imagining matching based on JSONPath expressions and simplifying structures based on the inferred type of elements rather than a full schema. This might look replacing a deeply nested object value with a string from one of its subfields, stripping out some fields, removing keys where all values underneath are null, etc.
If there aren't good answers I'll build my own approach.
Comments
I'd look at JSONata.
I played with it a while ago...
And if you want to utilize our Python support, take a look at the Python edition of it: jsonata-python.
Thanks! That's a useful reference point.
I would check out EnsLib.REST.DynamicObjVDoc, looks like it can parse something like JSONPath.
Ended up writing my own based on https://github.com/intersystems/isc-json/blob/main/cls/_pkg/isc/json/path.cls which implements https://goessner.net/articles/JsonPath/ - syntax looks like this, where inputObject and result are both %DynamicAbstractObjects:
Set result = ##class(%pkg.isc.json.transformer).For(inputObject
).Remove("$..url"// Remove URL
).Remove("$..requested_fields"// Remove requested_fields
).Remove("$..[?($IsObject(@) && (@.%Size() = 0))]"// Remove empty arrays/objects
).Remove("$..[?($IsObject(@) && (@.%Size() = 1) && (@.%GetTypeOf(1) = ""null""))]"// Remove one-element arrays containing only a null
).TransformElement("$..issue_type","$.name"// Replace issue type object with just the name
).TransformElement("$..priority","$.name"// Replace priority object with just the name
).TransformElement("$..status","$.name"// Replace status object with just the name
).TransformElement("$..[?($IsObject(@) && @.%IsA(""%DynamicObject"") && (@.""avatar_url"" '= """"))]","$.display_name"// Replace any user (indicated by avatar_url) with just the user's display name
).GetResult()I'll probably get this into https://github.com/intersystems/isc-json at some point (subject to some process hurdles); if you're interested nag me here / via DM.
It's not for JSON transformation directly, but a nice package by @Guillaume Rongier that lets you have messages contain data in JSON vs XML in the Interoperability pipeline.
Thanks! There's actually a reasonable level of built-in support for JSON display of messages at the platform level now (not *quite* as pretty as that package), you just need to have a JSONENABLED message class and override:
Method %GetContentType() As%String
{
Quit"application/json"
}Thahks @Timothy Leavitt ! Looks better with an example, of course. Hope you'll have some time for it in the future.
Sure! I have this class (extending %JSON.Adaptor rather than using https://github.com/intersystems/isc-json would be almost as good):
Class pkg.isc.mcp.message.BaseRequest Extends (Ens.Request, %pkg.isc.json.adaptor)
{
/// This method is called by the Management Portal to determine/// the content type that will be returned by the <method>%ShowContents</method>/// Override to application/json for pkg.isc.mcp.types.BaseModel.cls extends both/// %XML.Adaptor and %pkg.isc.json.adaptor but returns JSON
Method %GetContentType() As%String
{
Quit"application/json"
}
Storage Default
{
<Type>%Storage.Persistent</Type>
}
}With a subclass, pkg.isc.mcp.message.ToolCallRequest, that has a few more properties.
This shows up in the visual trace OOTB as:.png)
Neat! Thank you, looks easy and practical! Thank you for sharing!
@Timothy Leavitt, I had to create a custom business process to ingest the data from JSON into a Record Map I built for the request to FHIR. The JSON FHIR structures don't really work within the Data Transformation tool.
Our use case is the following: An EMS squad brings in a Trauma Case into our Emergency Room. The Trauma Case is Registered within Epic as Trauma. Until registration or Medical Records updates the Patient record within our EMR, the name will remain Trauma.
However, the EMS folks send a Result for the Medical Record which is Trauma... in our EMR, but they might send an updated name and demographics. Which causes the Result to fail based on Patient Validation errors in the EMR.
So, the design I came up, was to send a Patient Search FHIR query to the EMR to get the demographics that are currently assigned to that Medical Record, use that information to update the Result from the EMS vendor, and send the Result to the EMR to have it post in a timelier manner instead of waiting for a user in the EMR to work the error to post the result to the Patient's Chart.
We have not fully tested the workflow as of yet, but just getting the Request to be sent via FHIR to the EMR, and interpreting the result was the difficult part.