FHIR json property order part II

Since my initial question on ordering json properties a few things have happened.

Let me recap the issue at hand:
- In the FHIR specification, properties are listed in a certain order, for example, see https://www.hl7.org/fhir/patient.html#resource
- When you serialize resources to XML, the resulting document elements are ordered as defined in the specification.
- On the other hand, json objects returned from the IRIS for Health FHIR Repository will for example normally have the "id" as the last property, given that new properties are appended.

For me as a developer, this is annoying, even though I know json has no concept of order. When I read through received resources in e.g. Postman, I now need to look for "id" and "extension" properties at the end of the resource, instead of in the location that you would expect them based on the specification...

The most thorough but more expensive solution for ordering resource properties is to convert the FHIR resource to XML and back to json, like this:

/// Order FHIR resource properties according to the spec
/// With modifyOriginalObject = 1, also the original resource is re-ordered 
/// This is implemented by converting to XML and back so orders also deeper levels
/// This is 40-50 times slower than the shallow method, it takes 1.5 - 2 milliseconds 
/// With modifyOriginalObject = 1 it is "only" 20 times slower than the shallow method
ClassMethod FHIROrderResourcePropertiesDeep(schema As HS.FHIRServer.Schema, resource As %DynamicObject, modifyOriginalObject As %Boolean = 0) As %DynamicObject
{
    do ##class(HS.FHIRServer.Util.JSONToXML).JSONToXML(resource, .pOutStream, schema)
    set newresource = ##class(HS.FHIRServer.Util.XMLToJSON).XMLToJSON(.pOutStream, schema)

    if (modifyOriginalObject)
    {
        do ..CopyFromResource(resource, newresource) 
    }

    return newresource
}

This will properly order all properties in the object hierarchy based on the FHIR Schema.

I also have a more "shallow" method, where we just re-order each resource to start with "resourceType", "id", "meta", "text" and "extension", and we do not touch lower levels of the object hierarchy:

/// Order FHIR resource properties according to the spec
/// With modifyOriginalObject = 1, also the original resource is re-ordered 
/// This is a "shallow" method as it only looks at a few common attributes, and only orders the outer level of the object
/// This is however 40-50 times faster than the deep method, it only takes around 0.04 milliseconds
/// With modifyOriginalObject = 1 it takes around twice as much and so is "only" 20 times faster than the deep method
ClassMethod FHIROrderResourceProperties(resource As %DynamicObject, modifyOriginalObject As %Boolean = 0) As %DynamicObject
{
    set newresource = ..JsonOrderProperties(resource, [ "resourceType", "id", "meta", "text", "extension" ])

    if (modifyOriginalObject)
    {
        do ..CopyFromResource(resource, newresource) 
    }

    return newresource
}

/// Create a new json object with its properties in the specified order
ClassMethod JsonOrderProperties(object As %DynamicObject, order As %DynamicArray) As %DynamicObject
{
    #dim newObject as %DynamicObject = {}

    // First set the ordered properties in the new object 

    for index = 0:1:order.%Size() - 1
    {
        set name = order.%Get(index)
        set done(name) = 1
        set type = object.%GetTypeOf(name)

        if $EXTRACT(type, 1, 2) '= "un" // unassigned
        {
            do newObject.%Set(name, object.%Get(name)) 
        }
    }

    // Now copy remaining attributes not specified
    #dim iterator As %Iterator.Object = object.%GetIterator()

    while iterator.%GetNext(.name, .value, .type)
    {
        if '$DATA(done(name))
        {
            set type = object.%GetTypeOf(name)

            if (type = "boolean") || (type = "number") || (type = "null")
            {
                do newObject.%Set(name, value, type)
            }
            else
            {
                do newObject.%Set(name, value)
            }
        }
    }

    return newObject
}

I also wanted to be able to return a properly ordered FHIR resource from the Add() and Update() interaction methods in my IRIS for Health FHIR repository strategy.
This is implemented through the modifyOriginalObject parameter you will find in both code samples above:

    if (modifyOriginalObject)
    {
        do ..CopyFromResource(resource, newresource) 
    }

Goal for the CopyFromResource(resource, newresource) method is to remove all properties from the original resource, and too re-insert the properties from the new resource in the proper order. Surprise, surprise, I ended up with the resource properties being perfectly reversed order. :)

What I learned from this is that the %Set() adds new properties in the last know empty spot. After reversing the order before adding I ended up with the perfectly ordered resource:

/// Copy one json object and replace properties in the other
ClassMethod CopyFromResource(object As %DynamicObject, newobject As %DynamicObject)
{
    // First remove all original properties
    #dim iterator As %Iterator.Object = object.%GetIterator()
    while iterator.%GetNext(.name1, .value, .type)
    {
        do object.%Remove(name1)
    }

    // Properties are added back in at the last known empty slot in the object, so we end up with everything in reverse order
    // That is why we explicitly reverse that
    #dim reversedorder as %ListOfDataTypes = ##Class(%ListOfDataTypes).%New()
    #dim newiterator As %Iterator.Object = newobject.%GetIterator()
    while newiterator.%GetNext(.name, .value, .type)
    {
        do reversedorder.Insert(name)
    }

    // Now add back all properties from the new object in reverse order!!
    for index = reversedorder.Count():-1:1
    {
        set name = reversedorder.GetAt(index)
        set value = newobject.%Get(name)
        set type = newobject.%GetTypeOf(name)

        if (type = "boolean") || (type = "number") || (type = "null")
        {
            do object.%Set(name, value, type)
        }
        else
        {
            do object.%Set(name, value)
        }
    }
}

Yesterday I tried using the %Clear() method available in 2023.3 on %DynamicAbstractObject, to simplify the code. Unfortunately this throws an UNIMPLEMENTED error.

To be continued!