Using the Work Queue Manager to process data on multiple cores

First problem is with Work Queue Manager, that it does not work in CSP processes. In my project CacheBlocksExplorer, I tried to use it for async scan of database blocks. Here you can find my source, where I use it with WebSocket. Another problem is It does not work as well, if you just have only one core. 

So, you can force set how many jobs use:

Set ^||%ISC.WorkQueueMgr("ForceJobs")=2

Both these issues are addressed in the version we have scheduled for 2016.2.

Previousy the worker jobs are owned by the process asking for the work, in the new version we have a global pool of jobs available for any work. This allows the work queue manager to work in CSP processes. Also when you ask for a work group you can now specify how many processes you require.

Note that the 'ForceJobs' setting is unsupported and undocumented so we may remove it at some point.

One thing I am unclear on is whether the worker jobs have access to the in-memory objects of the process initiating the workers?  I have a potential use case for this that I am investigating.

Are there any practical examples of using this?

The worker jobs do not have access to the memory of the master process. These are just regular Cache processes. If you wish to share information between jobs then use globals and locks to synchronize access to pieces you need to change.

Have you got an example of how to use the Pause and Resume methods?

All the examples I've seen so far use the Queue and WaitForComplete methods, i.e. the process that has access to the workmgr object is waiting for everything to be completed and is therefore not accepting any other signals.

Please find below this test code of mine, using QueueCallBack and Wait methods that allows stopping the work, even if it's not very elegant. But it looks as if the Wait method is waiting for %exit =1 only. How would I suspend / resume?

Class DEV.WK.Tst1 Extends %RegisteredObject
{

ClassMethod Run() As %Status
{
k ^wk.Tst1
s sc=..ManageQ()
q sc
}

ClassMethod ManageQ() As %Status
{
s sc=$$$OK
s queue=$SYSTEM.WorkMgr.Initialize("/multicompile=1",.sc) if ('sc) { s sc=$$$ERROR($$$GeneralError,"Error initialising work queues: "_$system.Status.GetErrorText(sc)) q sc }
for i=1:1:100 {
s startchunk=i*1000,endchunk=((i+1)*1000)-1
s sc=queue.QueueCallback("##class("_$classname()_").Work","##class("_$classname()_").QCallBack",startchunk,endchunk) if 'sc { s sc=$$$ERROR($$$GeneralError,"Error queuing up work: "_$system.Status.GetErrorText(sc)) return sc }
}
s sc=queue.Wait(,.AtEnd) s ^wk.Tst1("AtEnd")=AtEnd
if sc&&'AtEnd { s sc=$$$ERROR($$$GeneralError,"Processing halted") }
d queue.Clear()
q sc
}

ClassMethod Work(StartId As %Integer, StopId As %Integer) As %Status
{
s ^wk.Tst1("work",$j,StartId)=""
h 60
q $$$OK
}

ClassMethod QCallBack(StartId As %Integer, StopId As %Integer) As %Status
{
if $d(^wk.Tst1("work",%job)) { s ^wk.Tst1("work",%job,StartId)=1 }
if $g(^wk.Tst1("halt"))=1 { s %exit=1 }
q $$$OK
}
 

ClassMethod Halt() As %Status
{
s ^wk.Tst1("halt")=1
q $$$OK
}

}