Are the DTLs referenced in routing rules or are they being called from custom Business Processes (or BPLs)? I verified the first option using a production at my current client and the DTLs are all there in the export list.

Ah, now it makes sense. The Document type you'd specified for the inbound message did not match the structure of the message itself, so the RemoveSegmentAt() method didn't know how to "find" the EVN segment ... it didn't exist in that schema.

The recommended approach would be to use a schema/document type (custom if necessary) that matches the structure of your inbound message, making the segment to be removed optional in the definition. With that in place, the RemoveSegmentAt() method would have worked as expected.

I would expect to see that error on the original message (before the EVN segment was deleted) but not on the cloned/modified version. Are you sure you're looking at the right message?

Mmmyyyeaaaaaah, You're going to want to do that in a DTL:

Create a rule that appends the text to the target.AlertText property:

You can add conditionals to provide different alert explanation values based on the SourceConfigName or by parsing the source.AlertText text to find something interesting.

Finally, stick the DTL in the Send rule for whatever operation you're sending alerts to.

Apparently there's a patch for the task so that it doesn't pollute the audit database when the <PROTECT> error occurs on a R/O database, but I don't think it's available for 2017.2.1.

I've been told dev key MAK5003 in Caché/Ensemble 2018.1.2+ is the solution, but the next upgrade will be to IRIS, which should also have that fix.

 Second followup: R/O mode did something bad to the "Update SQL query statistics" task that comes standard with Caché. Caused it to dump millions of <PROTECT> records into the Audit database, which ballooned in size and ran the application directory out of disk space. Terminating  and restarting the job had the same effect. Not sure what's going on, have a ticket open with the WRC now.

Just a quick follow-up ... no issues at all with making the database R/O. Works just as well as I'd hoped!

That's a great suggestion, Eduard, and far simpler than my code-based solution. My only concern is that I will be extracting the messages from the retired Ensemble message database so that it can be deleted, and I'm not sure whether the task I've written to do this will require any temporary, behind-the-scenes global storage on the database itself. Easy enough to test, though, as I have two other environments with the same configuration (Dev and QA).

You can check to see if the internal web server is enabled with:

%SYS>d ##class(Config.Startup).Get(.Prop)
 
%SYS>w Prop("WebServer"),!

1

You could write a simple classmethod that starts and stops the offending component when an inactivity alert is received. It would do little more than executing:

Do  ##class(Ens.Director).EnableConfigItem("service name",0,1)

Do  ##class(Ens.Director).EnableConfigItem("service name",1,1)

That would almost certainly reset the inactivity timer. As long as the class it's in extends Ens.Rule.FunctionSet, you'll have it available in the function selector in the Rule Editor drop-down list.