File Access Control for Inbound Adapter running in multiple IRIS instances

My team is working on redesigning an Interoperability solution that currently runs on a HealthShare server that is part of a mirror. Most of the messages are delimited records processed using Complex Record Mapping. We were told to utilize cloud services available in AWS, use containers, autoscaling…

Once we figured out how to use Docker, we imported the Health Connect code in multiple containers with a shared Elastic File System with the directory structure we copied from the existing server. We soon saw errors in Event Logs due to the same File Service configuration running in several containers looking at the same files. We considered to add file locking to the Inbound File Adapter code. We tested the modification and it appeared if one container instance locked a file, another instance could not open the same file with “WL” flag. We also read that Linux file locking is not always reliable.

We already had a remote database (part of a mirrored data tier) in place, which stored one table that gets updated by Health Connect Business Processes. I decided to use the Lock table on the Primary data server to control access to files from the Inbound File Adapter.

Most incoming messages originate from “DLA”, which sends the contents of files as the body of HTTP requests to a CSP page. We learned that this used a license and we converted the CSP page to REST. The incoming messages are consumed by a service in Health Connect and sent to a File Passthrough operation to write the files to disk, so they can be read by File Inbound Adapters and processed using Record Maps.

To prevent File Access contentions, I modified the Outbound File Adapter to place each file received into a queue which is maintained in a global mapped to the remote database. I also modified the Inbound File Adapter to not look for files directly in the directory by the Filespec setting but get the next file to process from the queue maintained on the remote database. The code to get the next file from the queue uses a lock on the remote data server to allow only one File Adapter access to one file. The filename of the next file is used in Inbound Adapter code in place of the Filespec setting to get it to look for the file that was pulled from the top of the queue.