Load balancing multiple Ensemble operations to handle bulk load of data
What is the recommended approach to handle the following scenario: Large number of HL7 messages are serviced via Ensemble. Those messages are then transformed and routed to an Operation which calls a web service which does a number of different activities. If will do address normalization, location geo coding, MPI lookup and resolution, and insert into a SQL Server database to stage for an analytics product. Unfortunately all of this work takes time and the first thing that happens in the web service is a lock taken out on the Patien in SQL Server. Each call can take seconds where the message input is taking milliseconds. This starts to impact the operation queue as the soap calls can be blocked by the locks. The Ensemble operation is waiting on the web service to return.
One work around was to create more operations to split the load so that when multiple HL7 messages are coming in for one patient, other operations can handle messages that are not being blocked. Is there a recommended approach to routing messages to separate operations based either randomly or keyed off of a hash of the patient identifier involved?