Question
· 19 hr ago

Fire-and-forget async/background non-blocking tasks

#Tips & Tricks #InterSystems IRIS for Health

I need to implement a retry policy for an incoming message queue containing thousands of relatively small messages.

Successfully processed messages should be immediately removed from the queue.

If an error occurs while processing a message, the message should be sent back at the end of the queue, and the pause before re-processing this message should increase geometrically (1-2-4-8-16 seconds, and so on). In languages that support the async/await pattern, I'd simply create a delayed timer that triggers a fire-and-forget task. This would prevent blocking the main thread. How can this be implemented in IRIS?

Product version: IRIS 2025.1
Discussion (6)3
Log in or sign up to continue
DC AI Bot · 19 hr ago

I can’t find the information you’re looking for. If you rephrase your question, I might be able to help.

0 0
Dmitrii Baranov · 10 hr ago

Hi Timo,

Yesterday I experimented a lot with both Python and workers (and failed).

Here is an example. In this code I want the workers to be started as background tasks, so the program should first print "This should be printed first", then each worker (each started with random delay interval) shoud print its own message. This obviously doesn't happen because of the queue.Sync() call but I don't want to wait for all the workers to complete.

Class DelayedTest Extends %RegisteredObject
{

ClassMethod Callback(interval As %String) As %Status
{
    Hang interval
    Write "Interval = ", interval, !
    Return $$$OK
}

Method RunWorkers()
{
    #Dim queue as %SYSTEM.WorkMgr
    Set queue = ##class(%SYSTEM.WorkMgr).%New()
    For i = 1:1:5
    {
        Set status = queue.Queue("..Callback", $RANDOM(5) + 1) // Minimal delay is 1 second
        $$$ThrowOnError(status)
    }

    Set status = queue.Sync()
    $$$ThrowOnError(status)
}

ClassMethod Main()
{
    #Dim d = ##class(DelayedTest).%New()
    Do d.RunWorkers() 
    Write "This should be printed first" 
}

}

Also, I'm not sure that Hang is appropriate here to emulate some delay. If it works like Sleep it should block the main thread.

0 0
Alexey Maslov · 10 hr ago

Hi Dmitrii,

Why not Sync from main()? 

Class User.DelayedTest Extends %RegisteredObject
{

ClassMethod Callback(interval As %String) As %Status
{
    Hang interval
    Write "Interval = ", interval, !
    Return $$$OK
}

Method RunWorkers(queue)
{
    #Dim queue as %SYSTEM.WorkMgr
    Set queue = ##class(%SYSTEM.WorkMgr).%New()
    For i = 1:1:5
    {
        Set status = queue.Queue("..Callback", $RANDOM(5) + 1) // Minimal delay is 1 second
        $$$ThrowOnError(status)
    }
}

ClassMethod Main()
{
    #Dim d = ##class(DelayedTest).%New()
    Do d.RunWorkers(.queue)
    Write "This should be printed first",!
    Set status = queue.Sync()
    $$$ThrowOnError(status)
    Write "Exiting...",!
}

}
0 0
Tani Frankel · 8 hr ago

Hi Dmitrii,

From the discussion in the comments it seems like you're trying to use "Workers", but did you try to use the Event mechanism for your use-case?

See also this related article (and this one as well).

By the way here's a Docs reference of using this from Python.

I don't have a wider context of this, but in the IRIS Interoperability functionality, behind the scenes of the Interoperability components, and the Messages and Queues managed there, this Event mechanism is used. So perhaps if you are already using IRIS's Interoperability capabilities, you can implement this in the higher level of the Business Components in your Interoperability Production, rather than with the lower level code using the Event class.

The Workers mechanism you tried to use is intended more for distributing parallel work, and the Event API is more for messaging and queuing scenarios, which sounds more like your use-case.

This article might also be of interest to you as it discusses moving from "Workers" to "Events".

1 0