Work Queue Manager (WQM) is a feature of InterSystems IRIS that enables you to improve performance by distributing work to multiple concurrent processes programmatically. The idea is that you split the work into chunks, and WQM distributes the chunks across worker processes and can provide the info that the work is done.
However, recently, I got an interesting question: there's a large logical transaction composed of ~1,000,000 individual objects and SQL inserts and updates. Some updates are CPU-intensive, so the original idea was to use WQM to split an update into chunks to speed things up.
But, here's a catch: if one of the individual 1,000,000 changes fails (there's a variety of application-level checks so that it can fail, and that's not even that abnormal a behavior), the entire transaction must be rolled back. That creates a problem: each chunk must report success before committing their individual transactions, and someone must get all these reports and decide if we are committing or not.
Unfortunately, it looks like WQM does not have a bidirectional communication between workers and manager, so I suggested an approach using events:
- Start jobs.
- Wait for all jobs to report success using $System.Event.WaitMsg().
- Send Commit or Rollback using the $System.Event.Signal().
