Notification of Mirror Failure

There appears to be this Using Log Monitor.

I use external tools to examine logs like cconsole.log though.

I have been attempting to try to use ZMIRROR (https://docs.intersystems.com/iris20232/csp/docbook/DocBook.UI.Page.cls?KEY=GHA_mirror_set_config#GHA_mirror_set_tunable_params_zmirror_routine) to execute certain scripting when $$NotifyBecomePrimary is kicked off, however it is still a work in progress.

You should be able to call an external script or something to notify you in some fashion that you developed. In our case we have several different notifications available to use via Shell or Perl that could notify us.

For a-sync mirror membwers, you could use the query:

and then to iterate on result and do the necessary cheeks. 

Hi: 

As responded below in the list yes you can use ^MONMGR in the terminal to email out to recipients whenever you have an error above a certain alert level. 

As alexander says and we do use an external service that also reads the cconsole/messages log and texts for high alerts

I use the NotifyBecomePrimary() hook in the ZMIRROR routine to call some code to send an email.

Here's the ZMIRROR routine from the %SYS namespace:

ZMIRROR    // User-defined code for mirror environment events
    QUIT
    
    //
    // The member has fully become primary so it is now safe to job off processes
    //
    // 20230314 - TM - Initial version
    //
NotifyBecomePrimary()
    
    //
    // Send an alert email to notify support staff that a mirror failover has occurred
    //
    // 20230726 - TM - Initial version
    //
    set Namespace = $znspace
    try {
        znspace "HAPPI"
        do ##class(System.AlertEmail).MirrorFailover()
        znspace Namespace
    }
    catch {
        znspace Namespace
    }
    
    QUIT

The email-sending code from the application namespace references configuration data in a global and constructs a html-formatted message to inform interested parties that a mirror failover has occurred.  You should be able to make out what's going on and get it working to your own needs.  Here's the class:
 

/// Description
/// -----------
///
/// Grapevine Solutions - HAPPI
///
/// This class implements the alert email methods for various system events.
///
/// Revision History
/// ----------------
///
/// 20230726 - TM - Initial version
///
Class System.AlertEmail Extends %RegisteredObject
{

/// This method sends an alert email message when a mirror failover occurs.  It is called from
/// the ZMIRROR routine in the %SYS namespace.
///
/// Revision History
/// ----------------
///
/// 20230726 - TM - Initial version
///
ClassMethod MirrorFailover()
{
    // get the current time
    set EmailTimeStampH = $now()
    
    // get the hostname, e.g. HCPGPO11-V, HCDDMC11-V
    set HostName = $zconvert($SYSTEM.INetInfo.LocalHostName(), "U")
    
    // get the environment, e.g. LIVE, TEST, DEV
    set Environment = ^HAPPIConfig("HostName", HostName)
    
    // get a connection to the email server
    set MailServer = ##class(%Net.SMTP).%New()
    set MailServer.smtpserver = ^HAPPIConfig("Environment", Environment, "Email", "SMTPServer")
    set MailServer.port = ^HAPPIConfig("Environment", Environment, "Email", "SMTPPort")
    
    // check for email server authentication details
    set MailServerUserName = $get(^HAPPIConfig("Environment", Environment, "Email", "Authentication", "UserName"))
    set MailServerPassword = $get(^HAPPIConfig("Environment", Environment, "Email", "Authentication", "Password"))
    
    // if there are authentication details then add them to the connection
    if ((MailServerUserName '= "") & (MailServerPassword '= "")) {
        
        set MailAuthentication = ##class(%Net.Authenticator).%New()
        set MailAuthentication.UserName = MailServerUserName
        set MailAuthentication.Password = MailServerPassword
        set MailServer.authenticator = MailAuthentication
        
        }
    
    // create a new HTML-formatted mail message
    set MailMessage = ##class(%Net.MailMessage).%New()
    set MailMessage.IsHTML = 1
    
    // set the recipient list
    set To = $order(^HAPPIConfig("Environment", Environment, "Email", "To", ""))
    while (To '= "") {
        if (^HAPPIConfig("Environment", Environment, "Email", "To", To) = 1) do MailMessage.To.Insert(To)
        set To = $order(^HAPPIConfig("Environment", Environment, "Email", "To", To))
    }
    
    // set the sender
    set MailMessage.From = ^HAPPIConfig("Environment", Environment, "Email", "From")
    
    // set the subject
    set Subject = "Mirror Failover Alert on " _ HostName _ " (" _ Environment _ ")"
    set MailMessage.Subject = Subject
    
    //--------------------------------
    // build the message html document
    //--------------------------------
    
    // start of html document
    do MailMessage.TextData.WriteLine("<!DOCTYPE html>")
    do MailMessage.TextData.WriteLine("<html>")
    
    // start of header
    do MailMessage.TextData.WriteLine("<head>")

    do MailMessage.TextData.WriteLine("<meta charset=""UTF-8"">")
    do MailMessage.TextData.WriteLine("<title>" _ Subject _ "</title>")
    
    // style sheet
    do MailMessage.TextData.WriteLine("<style>")
    do MailMessage.TextData.WriteLine("body {font-family: Arial, Helvetica, sans-serif;}")
    do MailMessage.TextData.WriteLine("h2 {color: blue; font-size: '125%';}")
    do MailMessage.TextData.WriteLine("h3 {color: blue; font-size: '100%';}")
    do MailMessage.TextData.WriteLine("table {border: 1px solid #ddd; border-spacing: 10px; border-collapse: separate;}")
    do MailMessage.TextData.WriteLine(".good {color: green;}")
    do MailMessage.TextData.WriteLine(".bad {color: red;}")
    do MailMessage.TextData.WriteLine("</style>")
    
    // end of header
    do MailMessage.TextData.WriteLine("</head>")
    
    // start of body
    do MailMessage.TextData.WriteLine("<body>")
    
    // write subject as message header
    do MailMessage.TextData.WriteLine("<h2>" _ Subject _ "</h2>")
    
    // write alert details
    do MailMessage.TextData.WriteLine("<h3>Alert Details</h3>")
    do MailMessage.TextData.WriteLine("<table>")
    do MailMessage.TextData.WriteLine("<tr>")
    do MailMessage.TextData.WriteLine("<td>Application</td>")
    do MailMessage.TextData.WriteLine("<td>HAPPI</td>")
    do MailMessage.TextData.WriteLine("</tr>")
    do MailMessage.TextData.WriteLine("<tr class=""bad"">")
    do MailMessage.TextData.WriteLine("<td>Alert Reason</td>")
    do MailMessage.TextData.WriteLine("<td>Mirror Failover")
    do MailMessage.TextData.WriteLine("</tr>")
    do MailMessage.TextData.WriteLine("<tr>")
    do MailMessage.TextData.WriteLine("<td>Host Name</td>")
    do MailMessage.TextData.WriteLine("<td>" _ HostName _ "</td>")
    do MailMessage.TextData.WriteLine("</tr>")
    do MailMessage.TextData.WriteLine("<tr>")
    do MailMessage.TextData.WriteLine("<td>Environment</td>")
    do MailMessage.TextData.WriteLine("<td>" _ Environment _ "</td>")
    do MailMessage.TextData.WriteLine("</tr>")
    do MailMessage.TextData.WriteLine("<tr>")
    do MailMessage.TextData.WriteLine("<td>Alert Time</td>")
    do MailMessage.TextData.WriteLine("<td>" _ $zdatetime($System.Util.UTCtoLocalWithZTIMEZONE(EmailTimeStampH), 3, 1, 3) _ "</td>")
    do MailMessage.TextData.WriteLine("</tr>")
    do MailMessage.TextData.WriteLine("</table>")
    
    // write support instructions
    do MailMessage.TextData.WriteLine("<h3>Support Instructions</h3>")
    do MailMessage.TextData.WriteLine("<p>")
    do MailMessage.TextData.WriteLine("A mirror failover has occurred in the " _ Environment _ " environment and " _ HostName _  " is now the primary failover member.")
    do MailMessage.TextData.WriteLine("</p>")
    do MailMessage.TextData.WriteLine("<p>")
    do MailMessage.TextData.WriteLine("Please ensure that the failover has not disrupted any application processes.")
    do MailMessage.TextData.WriteLine("</p>")
    do MailMessage.TextData.WriteLine("<p>")
    do MailMessage.TextData.WriteLine("Please do not reply to this message as it was automatically generated.")
    do MailMessage.TextData.WriteLine("</p>")
    
    // end of body
    do MailMessage.TextData.WriteLine("</body>")
    
    // end of html document
    do MailMessage.TextData.WriteLine("</html>")
    
    // send the message
    do MailServer.Send(MailMessage)
    
    QUIT
}

}