Can I build a custom task scheduler inside IRIS with CRON-like rules, retry policies, and distributed execution?

You can build a distributed and robust custom job scheduler in InterSystems IRIS with CRON-like rules, retry policies, dependencies, and failover recovery, but it will require leveraging and extending several available tools and methods.

Starting Points

1. Using InterSystems IRIS Task Manager:
   While the built-in Task Manager is not entirely sufficient for your needs, it allows you to create and manage tasks programmatically using the %SYS.Task class. You can also create custom tasks by subclassing %SYS.Task.Definition and implementing the OnTask() method, which will execute the desired task logic. This provides an extendable base for custom scheduling capabilities [1][2].

2. Custom Use of CRON-like Expressions:
   The iris-cron-task library available on InterSystems Open Exchange provides an easy way to create CRON-like task schedules. Install the package using ZPM (zpm "install iris-cron-task") and define tasks with CRON syntax. For example:

   ##class(dc.cron.task).Start("MyTask", "0 * * * *", "do ^MyRoutine", 1, .taskid)
   

   This can help automate periodic tasks without needing to set up additional classes [3][4].

3. Distributed Execution with Enterprise Cache Protocol (ECP):
   To distribute execution across nodes, consider using ECP for coordinating tasks between application servers and data servers. ECP settings allow for high availability and recovery from network interruptions, which can help maintain distributed execution [5].

4. Retry Policies and Failover Recovery:
   For robust retry and failover, you need a combination of:

   • Task retry using Task Manager's built-in functionalities (SuspendTaskOnError and rescheduling options).
   • Configuring the system's high availability (mirroring, clustering, or virtualization) to ensure task processing continuity during node failures [6].

Enhancing with Dependencies

Handling task dependencies will require creating a mechanism that executes a task only after the completion status of its dependent task is confirmed. For this:
- Use %SYS.Task programmatically to check the history and status of tasks by querying task execution records.
- Implement this logic in your custom task class or use a wrapper to manage workflows through dependency-based triggers.

Final Recommendation

Modifying or extending the framework via %SYS.Task alongside integrating iris-cron-task or ECP functionality seems practical for your described requirements. You may also refer to the suggested methods for logging, reporting, and ensuring fault tolerance.

Sources: