Added zpm/docker support (though struggling w/ M1 docker a bit I think)

While technically this could be written using a custom class extending Ens.BusinessService, what you describe has you playing the 'Operation' role more than the 'Service' role. We do this with many integrations and have a design pattern that works well for us.

In short, you need:

• Custom Adapterless Trigger Service (extends Ens.BusinessService). Only purpose is to send a simple Ens.Request to a Busines Process (BPL of custom class that extends Ens.BusinessProcess) on a timed interval... either using a schedule or the call interval.
• Custom Business Operation likely extending EnsLib.HTTP.GenericOperation or something similar.
• Custom Business Process to handle the business logic flow...
  • at time of Ens.Request from the trigger service, it formats a request object and sends to the Business Operation that executes your GET call against the webservice to receive the JSON payload.
  • JSON Payload returned by Business Operation to the Business Process as a custom message object ideally (no longer JSON) and from there, any manner of normal Ensemble workflows can take place (data transforms, ID logic, call outs to other business operations and so forth.)

You appear to be on a very old version of Ensemble so not sure how much recent documentation will be relevant to your use case and you likely will face a lot more difficultly using Ensemble 2014.1 than you would with something 2019.x or newer, but here's a few reference links to get the thought processes going:
Using the HTTP Outbound Adapter | Using HTTP Adapters in Productions | InterSystems IRIS for Health 2021.2
Creating REST Operations in Productions | Using REST Services and Operations in Productions | InterSystems IRIS for Health 2021.2
 

Your code:

Do HTTPRequestPat.EntityBody.Write(JsonArrayOBJ) // like this ? didnt work
          set ..Adapter.HTTPServer = "185.36.164.222"
          set ..Adapter.URL = UrlPats_"/api/patient/signaletic"
        set st = ..Adapter.SendFormDataURL(..Adapter.URL,.callResponsePat,"POST",HTTPRequestPat,,JsonArrayOBJ)

Try this instead:

Set json=JSONArrayOBJ.%ToJSON()
Do HTTPRequestPat.EntityBody.Write(json)
set ..Adapter.HTTPServer = "185.36.164.222"
set ..Adapter.URL = UrlPats_"/api/patient/signaletic"
set st = ..Adapter.SendFormDataURL(..Adapter.URL,.callResponsePat,"POST",HTTPRequestPat)

Attempting to get this to run on RHEL8 w/ podman-compose instead of docker.
 

Updated start.sh and generally everything starts up ok but nginx can’t seem to reach the iris instance. Think it’s a difference in how the ports get exposed. Only spent a few minutes on it but if anyone is aware, just probing to see if someone else has already solved. 

For maintenance releases, we use the document you linked to for a few key purposes:

• Are there any items reflected that we're impacted by potentially, even if unknowingly? (i.e. something causing a data integrity issue or unclean data in Interoperability msg)
• Are there any items that brought to light features/capabilities we may not have even know but ultimately decided to go after because of the release notes?
• If yes to either of the above...
  • Are there any fixes that will require extensive retesting of our integrations in PROD/TEST(STAGE)?
    • If so, does that outweigh the benefits of bullets 1 and 2 and thus wait for a major EM release?

For example, I recently upgraded our IRIS for Health 2021.1 instances to 2021.1.2 to take advantage of the (enhanced) debugging in VS Code IDE after determining the other items wouldn't 'break' anything else we were doing and/or happy with... while also seeing a few other issues we've noted but were not show stoppers for us were also addressed!

So yes, very much like the format and in fact, would love to see MORE like this for the EM releases. (i.e., 2021.1.0) as some of us nerdy folks like to see the low-level changes/updates. :-)

Thanks for all you do!

Unclear what you mean exactly but if you're wondering if IRIS will install on something like a Synology Disk Station - yes - it works just fine using either Docker w/ containers or full VM Server experience where you first install an OS like RedHat or Ubuntu into a VM and then install the appropriate kit.

Obviously would only be done for development purposes and not a production like scenario. Most NAS servers can't push the required IOPS to properly support a production IRIS server.

Believe what you’re after us a query like this:

select *

from enslib_hl7.message as hl7

where hl7.id not in (select messagebodyid from ens.messageheader where messagebodyclassname = ‘EnsLib.HL7.Message’)

apologies for poor formatting, typing on a phone while a kiddo naps. But that should help identify the message bodies that are orphaned. 

I have tackled a challenge like this - with additional complex wrinkles where the related records were all in separate files in one big zip-file pulled down via SFTP by my production Service and I needed to generate a combined file for the Complex Record Map to process with the appropriate prefixes in place.

I hope this will be useful to you - after getting it working, the vendor I had to write it for went belly up so I didn't finish cleaning up my traces or comments but this was fully working.  The general flow is an SFTP pickup of a zip file daily that contains 4 files... a PAT (patient) file, a data element 1, data element 2 and data element 3 file - all comma separated. the Patient Identifier in the first file could link to the other 3 files in one of the columns.

At the end, I up with a single file I pushed into CRM that looks like (where PAT ID is 123456):

PAT|123456,SMITH,JOHN,M,moredata,etc
DATA1|49492,123456,data1data,data1moredata,data1etc
DATA2|577545,123456,data1data,data1moredata,data1etc
DATA3|454543,123456,data1data,data1moredata,data1etc
I hope it's useful to you to at least get an idea of how to get started on your particular use case. Happy to try and clarify anything if needed.
 

/// Custom business service to handle ingesting multiple related delimited flat files (contained in a single ZIP!) and 
/// combining them into a single message per Patient that is then fed into a Complex RecordMap.
Class MyPkg.Services.CRM.BatchZip Extends EnsLib.RecordMap.Service.ComplexBatchStandard [ Final ]
{

Parameter ADAPTER = "EnsLib.FTP.InboundAdapter";

Parameter SETTINGS = "ZipUtility:Basic";

Parameter Slash = {$Case($System.Version.GetOS(),"Windows":"\",:"/")};

/// Operating System Utility, with parameters, that will be executed to extract zip file from vendor.
/// Use {filename} as placeholder for the dynamic ZIP file name within the parameters.
/// 
/// Note that any utility used must write the filenames of the contents to stdout for interrogation
/// by this service.
/// 
/// Default: unzip (GNU) linux utility for unix/linux based operating systems
Property ZipUtility As %String [ InitialExpression = "unzip -o {filename}" ];

Method OnProcessInput(
    pInput As %FileBinaryStream,
    pOutput As %RegisteredObject,
    ByRef pHint As %String) As %Status
{
    Set tSC = $$$OK
    Set instanceID = $System.Util.CreateDecimalGUID()
    $$$TRACE("Unique InstanceID: "_instanceID)
    Set ^MyPkg.Temp(instanceID) = "Creating CRM-compatible masterfile for this batch file input: "_pInput.Filename
    $$$TRACE("Starting to process "_##class(%File).GetFilename(pInput.Filename))
    $$$TRACE("Executing GetZipContents")
    Set tSC = ..GetZipContents(pInput.Filename, .files)
    If $$$ISERR(tSC) Quit tSC

    // Process each sub-file into a temporary global so we can add our CRM fixed leading data and join the records together
    $$$TRACE("Processing each file into a temporary global: ^MyPkg.Temp("_instanceID_")")
    Set ptr=0
    While $ListNext(files,ptr,file)
    {
        $$$TRACE("Processing file "_file)
        If ..startsWith($P(file,..#Slash,*,*),"patients_")
        {
            Set tSC = ..ProcessFile(file, ",", instanceID)
        }
        ElseIf ..startsWith($P(file,..#Slash,*,*),"dataElement1_")
        {
            Set tSC = ..ProcessFile(file, ",", instanceID)
        }
        ElseIf ..startsWith($P(file,..#Slash,*,*),"dataElement2_")
        {
            Set tSC = ..ProcessFile(file, ",", instanceID)
        }
        ElseIf ..startsWith($P(file,..#Slash,*,*),"dataElement3_")
        {
            Set tSC = ..ProcessFile(file, ",", instanceID)
        }
        Else
        {
            Do ##class(%File).Delete(file)
        }
    }
    
    $$$TRACE("Creating MasterInputFile that we'll feed into a Complex Record Map.")
    Set tSC = ..CreateMasterInputFile(pInput.Filename, instanceID, .masterInputFile)
    
    $$$TRACE("MasterInputFile: "_masterInputFile)
    $$$TRACE("Now processing MasterInputFile into Complex RecordMap.")
    Try {
        Set masterInputFileStream = ##class(%FileBinaryStream).%New()
        Set masterInputFileStream.Filename = masterInputFile
        Set tLookAhead = ""
        Set tIOStream = ##class(EnsLib.RecordMap.Service.FileServiceStream).%New(masterInputFileStream)
        Set tIOStream.Name = ..GetFileName(masterInputFileStream)
        
        While 'tIOStream.AtEnd {
            Set tPosition = tIOStream.Position
            Set tSC = ..GetBatch(tIOStream, .tBatch,,.tLookAhead)
            If $$$ISERR(tSC) || (tPosition=tIOStream.Position) Quit
            
            Set ..%SessionId = ""
            Set tStatus = ..ForceSessionId()
            If $$$ISERR(tStatus) Quit
            
            Set tSC = ..SendRequest(tBatch,'..SynchronousSend)
            If $$$ISERR(tSC) Quit
        }
        If $$$ISERR(tSC) Quit
        
        If 'tIOStream.AtEnd {
            $$$LOGWARNING($$$FormatText($$$Text("Failed to advance record stream. Stopped reading file '%1' at position %2, not at end.","Ensemble"),tIOStream.Name,tIOStream.Position))
        }
    }
    Catch ex {
        Set tSC = $$$EnsSystemError
    }
    If $get(tLookAhead) '= "" {
        $$$LOGINFO("Discarding trailing characters: '"_tLookAhead_"'")
    }
    
    $$$TRACE("Cleaning up the temporary global we created.")
    Set tSC = ..CleanUp(instanceID)
    $$$TRACE("Completed "_##class(%File).GetFilename(pInput.Filename))
    Quit tSC
}

Method ProcessFile(
    pFilename As %String,
    pDelimiter As %String = ",",
    pInstanceID As %String) As %Status [ Private ]
{
    Set tSC = $$$OK
    Set skipHeader = 1
    
    Set file=##class(%File).%New(pFilename)
    Set tSC = file.Open("RU")
    While 'file.AtEnd
    {
        Set line = file.ReadLine()
        If skipHeader
        {
            Set skipHeader = 0
            Continue    
        }
        
        If line '[ pDelimiter Continue
        
        If ..startsWith($P(pFilename,..#Slash,*,*),"patients_")
        {
            // How do we identify the 'key' value to link up the other pieces? Get a piece of the row and store it as a part of the global key!
            Set key = $Piece(line,pDelimiter,1,1)_","_$Piece(line,pDelimiter,2,2)
            // Let's give ourselves a prefix! PAT|
            Set ^MyPkg.Temp(pInstanceID,"PAT|",key) = "PAT|"_line
        }
        ElseIf ..startsWith($P(pFilename,..#Slash,*,*),"dataElement1_")
        {
            // Each dataElement has a key for itself but also a linking key to the PAT
            Set ^MyPkg.Temp(pInstanceID,"DATA1|",$Piece(line,pDelimiter,1,1),$Piece(line,pDelimiter,2,2)) = "DATA1|"_line
        }
        ElseIf ..startsWith($P(pFilename,..#Slash,*,*),"dataElement2_")
        {
            // Each dataElement has a key for itself but also a linking key to the PAT
            Set ^MyPkg.Temp(pInstanceID,"DATA2|",$Piece(line,pDelimiter,2,2),$Piece(line,pDelimiter,1,1)) = "DATA2|"_line
        }
        ElseIf ..startsWith($P(pFilename,..#Slash,*,*),"dataElement3_")
        {
            // Each dataElement has a key for itself but also a linking key to the PAT
            Set ^MyPkg.Temp(pInstanceID,"DATA3|",$Piece(line,pDelimiter,3,3),$Piece(line,pDelimiter,5,5)) = "DATA3|"_line
        }
    }
    
    Do file.Close()
    Do ##class(%File).Delete(pFilename)
    Quit tSC
}

/// Let's start putting everything together into one big file that CRM will process!
Method CreateMasterInputFile(
    pSourceFilename As %String,
    pInstanceID As %String,
    Output MasterInputFilename) As %Status [ Private ]
{
    Set tSC = $$$OK
    Set MasterInputFilename = $Replace(pSourceFilename,".zip",".txt")
    
    Set fileObj = ##class(%File).%New(MasterInputFilename)
    Set tSC = fileObj.Open("WSN")
    If ($SYSTEM.Status.IsError(tSC)) {
        Do $System.Status.DisplayError(tSC)
        Quit $$$NULLOREF
    }
    
    Set key=$Order(^MyPkg.Temp(pInstanceID,"PAT|",""))
    While key'=""
    {
        Set patID = $Piece(key,",",2,2)
        
        // Write out PAT| 
        Do fileObj.WriteLine(^MyPkg.Temp(pInstanceID,"PAT|",key))
        
        // Get dataElement1 for that PAT next... patID Key 1, dataElement1 Key 2
        Set data1Key = $Order(^MyPkg.Temp(pInstanceID,"DATA1|",""))
        While data1Key'=""
        {
            If data1Key = patID
            {
                Set data1Key2 = $Order(^MyPkg.Temp(pInstanceID,"DATA1|",data1Key,""))
                While data1Key2'=""
                {
                    Do fileObj.WriteLine(^MyPkg.Temp(pInstanceID,"DATA1|",data1Key,data1Key2))    
                    Set data1Key2 = $Order(^MyPkg.Temp(pInstanceID,"DATA1|",data1Key,data1Key2))
                }
            }
            Set data1Key = $Order(^MyPkg.Temp(pInstanceID,"DATA1|",data1Key))
        }
        
        // Get dataElement2 for that PAT next... patID Key 1, dataElement2 Key 2
        Set data2Key = $Order(^MyPkg.Temp(pInstanceID,"DATA2|",""))
        While data2Key'=""
        {
            If data2Key = patID
            {
                Set data2Key2 = $Order(^MyPkg.Temp(pInstanceID,"DATA2|",data2Key,""))
                While data2Key2'=""
                {
                    Do fileObj.WriteLine(^MyPkg.Temp(pInstanceID,"DATA2|",data2Key,data2Key2))    
                    Set data2Key2 = $Order(^MyPkg.Temp(pInstanceID,"DATA2|",data2Key,data2Key2))
                }
            }
            Set data2Key = $Order(^MyPkg.Temp(pInstanceID,"DATA2|",data2Key))
        }

        // Get dataElement3 for that PAT next... patID Key 1, dataElement3 Key 2
        Set data3Key = $Order(^MyPkg.Temp(pInstanceID,"DATA3|",""))
        While data3Key'=""
        {
            If data3Key = patID
            {
                Set data3Key2 = $Order(^MyPkg.Temp(pInstanceID,"DATA3|",data2Key,""))
                While data3Key2'=""
                {
                    Do fileObj.WriteLine(^MyPkg.Temp(pInstanceID,"DATA3|",data3Key,data3Key2))    
                    Set data3Key2 = $Order(^MyPkg.Temp(pInstanceID,"DATA3|",data3Key,data3Key2))
                }
            }
            Set data3Key = $Order(^MyPkg.Temp(pInstanceID,"DATA3|",data3Key))
        }
        
        Set key = $Order(^MyPkg.Temp(pInstanceID,"PAT|",key))
    }
    
    Do fileObj.Close()
    
    Quit tSC
}

/// Using full path, will extract Zip file using $ZF(-100) - OS-level execution - and read in the filenames
/// of what was extracted for further processing, returning as a list to OnProcessInput
Method GetZipContents(
    pFilename As %String,
    Output pContentFilenames As %List) As %Status [ Private ]
{
    Set tSC = $$$OK, tempFilenames = ""
    Set stdoutFilename = ##class(%File).TempFilename("myTempCRMBatch")
    
    Set unzipCmd = $Replace(..ZipUtility,"{filename}",pFilename)
    $$$TRACE("Executing OS command: "_unzipCmd)
    Set workingDirectory = $Piece(pFilename,..#Slash,1,*-1)
    Set unzipCmd = "cd "_workingDirectory_";"_unzipCmd
    Set sc = $ZF(-100,"/SHELL /NOQUOTE /STDOUT+="""_stdoutFilename_""" /STDERR+="""_stdoutFilename_"""",unzipCmd)
    
    Set stdout=##class(%File).%New(stdoutFilename)
    Set stdout.LineTerminator = $char(10)
    Set tSC = stdout.Open("RU")
    While 'stdout.AtEnd
    {
        Set stdoutLine = stdout.ReadLine()
        If stdoutLine [ ".csv"
        {
            Set temp = $LFS(stdoutLine," ")
            Set ptr = 0
            While $ListNext(temp,ptr,piece)
            {
                If $ZStrip(piece,"*W") [ ".csv"
                {
                    //$$$TRACE("Found file in zip: "_$ZStrip(piece,"*W"))
                    If tempFilenames '= "" Set tempFilenames = tempFilenames_","
                    Set tempFilenames = tempFilenames_workingDirectory_..#Slash_$ZStrip(piece,"*W")
                }
            }
            
        }
    }
    
    Do stdout.Close()
    Set tSC = ##class(%File).Delete(stdoutFilename)
    Set pContentFilenames = $LFS(tempFilenames,",")
    Quit tSC
}

Method CleanUp(pInstanceID As %String) As %Status [ Private ]
{
    Kill ^MyPkg.Temp(pInstanceID)
    Quit $$$OK
}

Method startsWith(
    value As %String,
    string As %String) As %Boolean [ CodeMode = expression, Internal, Private ]
{
($E($g(value),1,$L($g(string)))=$g(string))
}

}

Hey Scott - Your questions require a bit of clarification to best answer but I can help a bit as I just went through this for both internally served and secured IRIS Management Portal and externally served and secured IRIS-hosted web services. 
 

There’s two layers to securing to consider and that’s where I would need clarification on which part your questions are after:

• Mutual TLS 1.2 encryption to/from the web gateway module installed on Apache that acts as a reverse proxy of sorts between the web server and the IRIS server’s SuperServer port. (Actual users don’t use this port directly in a web browser) 
• HTTPS/SSL Encryption on the Apache Web Server that encrypts the traffic between the client browser and web server itself.

For the a production quality/secure setup, you want to always achieve both of these in my opinion.

For the first bullet, if you control both sides of the equation (the IRIS server and the web server), you could easily do a self-signed cert using your redhat server’s CA as you can specify the CA Chain of Authority that validates the signed cert on both sides. 
 

For the second bullet, you really want to use a certificate authority that your user’s web browsers will natively trust. Eg if youre just serving up internally and all your users are joined to an internal domain, that domain’s CA could generate a web server cert you could install to be used by port 443 on apache httpd and your user’s browsers will likely be a-ok with it as domain CAs generally update their domain members Keystore on login (keyword being usually.) That CA could also be used to generate appropriate Server/Client profile certs to be used for the mutual tls of the first bullet  

But easiest approach for the second bullet is using a external trusted CA (think Thawte, VeriSign, and many others) as browsers will generally trust these “out of the box.” External CAs can also be used for the mutual TLS piece but generally overkill if the web gateway and iris server are all on the same internal network (again, in my opinion) - proper securing of private keys is important with use of internal CA for mutual TLS especially but really should be doing that anyway.

Reference for the mutual tls: https://docs.intersystems.com/irisforhealthlatest/csp/docbook/DocBook.U…

From the OS side in AIX, I can see it in parameters.isc (example from a QA env I'm playing with)

security_settings.iris_user: irisusr
security_settings.iris_group: irisusr
security_settings.manager_user: irisusr
security_settings.manager_group: irisusr

I do not recall how to see it in IRIS itself (or if it's even possible) but I remember wanting to figure out how to change the values after installation (due to someone goofing up an entry on a dev environment) and without a lot of effort, it is pretty difficult.

I posted this on a similar question a couple weeks back - this worked for me on my MBP M1:

If you prefer or need to use a kit (like myself), give multipass a look. Super easy to spin up a ubuntu VM in seconds.

Note that you need to pull down the arm64 version of ubuntu AND the arm64 iris kits, not traditional x86_64 architecture.

https://github.com/canonical/multipass
https://9to5linux.com/canonical-makes-it-easier-to-run-ubuntu-vms-on-ap…