How to implement Enterprise Message Bank

The rise of distributed systems and cloud-native architectures within large institutions has redefined hosting for InterSystems IRIS interoperability applications (productions). In modern environments, the standard layout involves at least two production instances operating on distinct nodes under a load-balancing configuration to handle requests.

The benefits are clear: you can now scale dynamically based on request volume, increasing the number of active productions during peak demand and going down when traffic subsides. However, a trade-off arises when analyzing one or more interoperability messages is necessary. In which production instance should one look? If a cluster contains more than three active instances, it turns into a significant management challenge.  If there is a need to analyze messages collectively, it becomes even more critical.

InterSystems IRIS addresses this issue via the Enterprise Message Bank (https://docs.intersystems.com/iris20261/csp/docbook/Doc.View.cls?KEY=EGDV_message_bank#EGDV_message_bank_overview). This utility aggregates logs and messages from various production instances into a message server (message bank), providing a central management hub with a unified management interface. It removes the need to log into each instance to search for messages or analyze history. Instead, you simply go to the hub. This article demonstrates, with a practical application, how to configure a message server and clients (active production instances) to send all their messaging and logs to this server, enabling centralized production management.

The Concepts of the Message Bank

The Message Bank components include the following (source: docs.intersystems.com):

• The Message Bank Server: A lightweight production consisting exclusively of a Message Bank service designed to receive submissions from any number of client nodes. 
• The Message Bank Operation: A client-side component added to a production environment and configured with the address of a Message Bank server.

Below you can see a conceptual example: 

How to Set Up the Message Bank: An Example Application

The Open Exchange repository provides a demonstrative application illustrating how to configure Message Bank for your distributed applications. The sample includes three containers:

irisserver: The message bank server that receives messages from all clients and provides a centralized view.
irisclient1: An interoperability application, configured as a client, that forwards messages to the irisserver via a Message Bank Operation. 
irisclient2: A second interoperability node acting as an additional client.
Check out the Docker Compose file with the container definitions below: 
 

services:
  irisserver:
    container_name: irisserver
    hostname: irisserver
    build:
      context: .
      dockerfile: Dockerfile
    restart: always
    environment:
      - ISC_CPF_MERGE_FILE=/home/irisowner/dev/merge.cpf
    ports:
      - 1972
      - 54773:52773
      - 53773
      - 9192:9192
    volumes:
      - ./:/home/irisowner/dev
    networks:
      - iris_network
  irisclient1:
    container_name: irisclient1
    hostname: irisclient1
    build:
      context: .
      dockerfile: Dockerfile
    restart: always
    environment:
      - ISC_CPF_MERGE_FILE=/home/irisowner/dev/merge.cpf
    ports:
      - 1972
      - 54774:52773
      - 53773
    volumes:
      - ./:/home/irisowner/dev
    networks:
      - iris_network
  irisclient2:
    container_name: irisclient2
    hostname: irisclient2
    build:
      context: .
      dockerfile: Dockerfile
    restart: always
    environment:
      - ISC_CPF_MERGE_FILE=/home/irisowner/dev/merge.cpf
    ports:
      - 1972
      - 54775:52773
      - 53773
    volumes:
      - ./:/home/irisowner/dev
    networks:
      - iris_network

networks:
  iris_network:
    driver: bridge

Our sample will run with three containers listed below:

Note: Port 9192 on irisserver1 is dedicated to TCP communication between the server and clients using the Message Bank service from the Server and the Operations from the clients. 
 

To obtain the msg-banking-sample (the prototype application used in this article), follow the steps below:

1. Clone/git pull the repo into any local directory:

$ git clone https://github.com/yurimarx/msg-banking-sample.git

2. Open the terminal in this directory and execute the following code:

$ docker-compose build

3. Run the IRIS container with your project:

$ docker-compose up -d

To configure the Message Bank for your applications, do the following:

1. Open the irisserver production (http://localhost:54773/csp/user/EnsPortal.ProductionConfig.zen?$NAMESPA…;) and start it. All services must be labeled green. 

• Port property is used to configure the port employed by clients to send their messages. 
• Call Interval is applied to get new messages every 5 seconds.

The source code for the production is the following:

Class dc.sample.MessageBankProduction Extends Ens.Enterprise.MsgBank.Production
{

XData ProductionDefinition
{
<Production Name="dc.sample.MessageBankProduction" TestingEnabled="false" LogGeneralTraceEvents="false">
 <Description>Production for receiving and collating message bank submissions from one or more client
  interoperability-enabled namespaces and for maintaining a local repository of production status
  information about each client namespace, for display on the Enterprise Monitor page. Open the Monitor
  page on the same machine that is hosting this Production.</Description>
 <ActorPoolSize>0</ActorPoolSize>
 <Setting Target="Production" Name="ShutdownTimeout">120</Setting>
 <Setting Target="Production" Name="UpdateTimeout">10</Setting>
 <Item Name="MonitorService" Category="" ClassName="Ens.Enterprise.MonitorService" PoolSize="1" Enabled="true" Foreground="false" Comment="Populates global         ^IRIS.Temp.Ens.EntMonitorStatus by polling namespaces from Systems List every CallInterval seconds" LogTraceEvents="false" Schedule="">
   <Setting Target="Host" Name="InactivityTimeout">0</Setting>
   <Setting Target="Host" Name="AlertGracePeriod">0</Setting>
   <Setting Target="Host" Name="AlertOnError">0</Setting>
   <Setting Target="Host" Name="ArchiveIO">0</Setting>
   <Setting Target="Adapter" Name="CallInterval">10</Setting>
 </Item>
 <Item Name="MsgBankService" Category="" ClassName="Ens.Enterprise.MsgBank.TCPService" PoolSize="100" Enabled="true" Foreground="false" Comment="" LogTraceEvents="true" Schedule="">
   <Setting Target="Host" Name="InactivityTimeout">20</Setting>
   <Setting Target="Host" Name="AlertGracePeriod">0</Setting>
   <Setting Target="Host" Name="AlertOnError">0</Setting>
   <Setting Target="Host" Name="ArchiveIO">0</Setting>
   <Setting Target="Adapter" Name="Endian">Big</Setting>
   <Setting Target="Adapter" Name="UseFileStream">0</Setting>
   <Setting Target="Adapter" Name="JobPerConnection">1</Setting>
   <Setting Target="Adapter" Name="AllowedIPAddresses"></Setting>
   <Setting Target="Adapter" Name="QSize">100</Setting>
   <Setting Target="Adapter" Name="CallInterval">5</Setting>
   <Setting Target="Adapter" Name="Port">9192</Setting>
   <Setting Target="Adapter" Name="StayConnected">-1</Setting>
   <Setting Target="Adapter" Name="ReadTimeout">10</Setting>
   <Setting Target="Adapter" Name="SSLConfig"></Setting>
   <Setting Target="Host" Name="BankHelperClass">dc.sample.MessageBankHelper</Setting>
 </Item>
</Production>
}
}

Some important points about this production include the following: 
 

• The Message Bank Server production must be a subclass of Ens.Enterprise.MsgBank.Production.
• The MonitorService (class type Ens.Enterprise.MonitorService) will receive tracking metrics information from each client.
• The MsgBankService (class type Ens.Enterprise.MonitorService) will consume the client production messages and save them in a central repository on the server.

2. Open the Configure Message Bank link on irisclient1 (http://localhost:54774/csp/user/EnsPortal.FindMsgBank.zen?$NAMESPACE=USER&), set the following values, and click the Save button:

• Web Server IP Address: irisserver
• Web Server Port Number: 54773
• Message Bank Production Namespace: USER

Each client must connect to the server via the Message Bank Link. However, to send messages, proceed with the next step.

3. Open the irisclient1 production (http://localhost:54774/csp/user/EnsPortal.ProductionConfig.zen?$NAMESPACE=USER&) and start it. All services and operations must be marked green. 

• The IP Address property specifies the IP address of the Message Bank server. 
• The Port property configures the Message Bank server port. 
• The Enable Archiving property must be checked to ensure the messages are sent to the server. 
• To enable archiving, ensure the property is checked, and the Archive Items filter is set (e.g., [],-Ens.ScheduleService[],-Ens.ScheduleHandler[]) to send all messages, except scheduled ones. 
• You must call the operation named Ens.Enterprise.MsgBankOperation.

The source code for the production is as follows:

Class dc.sample.HelloProduction Extends Ens.Production
{

XData ProductionDefinition
{
<Production Name="dc.sample.HelloProduction" TestingEnabled="true" LogGeneralTraceEvents="false">
 <Description>Sample of a Production: Sends hello every 5 seconds</Description>
 <ActorPoolSize>2</ActorPoolSize>
 <Item Name="HelloService" Category="" ClassName="dc.sample.HelloService" PoolSize="1" Enabled="true" Foreground="false" Comment="" LogTraceEvents="false" Schedule="">
   <Setting Target="Adapter" Name="CallInterval">5</Setting>
 </Item>
 <Item Name="FileOperation" Category="" ClassName="dc.sample.FileOperation" PoolSize="1" Enabled="true" Foreground="false" Comment="" LogTraceEvents="false" Schedule="">
   <Setting Target="Adapter" Name="FilePath">/tmp/</Setting>
 </Item>
 <Item Name="Ens.Enterprise.MsgBankOperation" Category="" ClassName="Ens.Enterprise.MsgBankOperation" PoolSize="1" Enabled="true" Foreground="false" Comment="" LogTraceEvents="false" Schedule="">
   <Setting Target="Host" Name="EnableArchiving">1</Setting>
   <Setting Target="Adapter" Name="IPAddress">irisserver</Setting>
 </Item>
</Production>
}

}

As you can see, it's just another production, but with an added detail: a message bank operation called Ens.Enterprise.MsgBankOperation. This Operation picks up all messages from production and sends them to the message server with a configured IP address and Port.

For the next 2 steps with irisclient2, we should use the same settings as for irisclient1.

4. Open the Configure Message Bank link on irisclient2 (http://localhost:54775/csp/user/EnsPortal.FindMsgBank.zen?$NAMESPACE=USER&), set the following values, and click the Save button:

• Web Server IP Address: irisserver
• Web Server Port Number: 54773
• Message Bank Production Namespace: USER

5. Open the irisclient2 production (http://localhost:54775/csp/user/EnsPortal.ProductionConfig.zen?PRODUCTION=dc.sample.HelloProduction) and start it. All services and operations must be highlighted in green.

6. Now, all client messages are arriving at the message server. Consequently, we can use the Message Bank viewer of the irisserver container to query messages originating from all connected clients. To set it up, go to the irisserver Message Bank Viewer (http://localhost:54773/csp/user/Ens.Enterprise.Portal.MsgBankViewer.zen) and click Search to see the irisclient1 and irisclient2 messages. 

The Message Bank Viewer has the same options as the local (client) message viewer, except for the Client property, used to identify the message bank client.
When you click on a message, you can see all the details about it: 

Pay attention to the Description property. Our sample uses a Helper Class to set it to the same value as the property ClientBodyClassName.

You can also query the event log messages. To do it, go to http://localhost:54773/csp/user/Ens.Enterprise.Portal.MsgBankEventLog.zen:

Using a Helper Class (Optional) to Change Message Bank Behavior 
 

You can optionally modify the Message Bank's behavior using a Helper class that inherits from Ens.Enterprise.MsgBank.BankHelperClass, and implements the OnBankMsg method. Below, there is an example source code that alters the message's Description property, setting the name of the source class in its content:

Class dc.sample.MessageBankHelper Extends Ens.Enterprise.MsgBank.BankHelperClass
{

ClassMethod OnBankMsg(pHeader As Ens.Enterprise.MsgBank.MessageHeader, pFullHeaderID As %String, pBody As %RegisteredObject = "", pFullBodyID As %String, pService As Ens.Enterprise.MsgBank.TCPService) As %Status
{
  
   Try {
       If $IsObject(pHeader) && (pHeader.MessageBodyClassName'="") {
           Set pHeader.Description = pHeader.ClientBodyClassName
           Do pHeader.%Save()
       }
   }
   Catch ex {
       Return ex.AsStatus()
   }
  
   Return 1
}

ClassMethod GetBankClassName(pOriginalClassName As %String) As %String
{
   Quit pOriginalClassName
}

}

For the class to be used, you must point out its features on the irisserver, inside the MsgBankService in the Bank Helper Class property: 

At this point, all messages will have the  Description property content containing the ClientBodyClassName.

A real use case for Bank Helper Class is to convert from Global String Stream to the Client Message Body:

Class dc.sample.MessageBankHelper Extends Ens.Enterprise.MsgBank.BankHelperClass
{

ClassMethod OnBankMsg(pHeader As Ens.Enterprise.MsgBank.MessageHeader, pFullHeaderID As %String, pBody As %RegisteredObject = "", pFullBodyID As %String, pService As Ens.Enterprise.MsgBank.TCPService) As %Status
{
  
   Try {
       If $IsObject(pHeader) && (pHeader.ClientBodyClassName '= "") {
          
           // Checks if the class exists and is compiled in the current namespace.
           If ##class(%Dictionary.CompiledClass).%ExistsId(pHeader.ClientBodyClassName) {
              
               // Checks if the body is not of the expected class
               If $IsObject(pBody) && (pBody.%ClassName(1) '= pHeader.ClientBodyClassName) {
                  
                   // If the body is a Stream, we attempt to deserialize the XML.
                   If pBody.%IsA("%Stream.Object") {
                       Set tReader = ##class(%XML.Reader).%New()
                       Set tSC = tReader.OpenStream(pBody)
                      
                       If $$$ISOK(tSC) {
                           // The root tag of the generated XML is commonly the class name without the package name.
                           Set tRootTag = $Piece(pHeader.ClientBodyClassName, ".", *)
                           Do tReader.Correlate(tRootTag, pHeader.ClientBodyClassName)
                          
                           If tReader.Next(.tMessage, .tSC) {
                               Set tSC = tMessage.%Save()
                              
                               If $$$ISOK(tSC) {
                                   Set pHeader.MessageBodyClassName = pHeader.ClientBodyClassName
                                   Set pHeader.MessageBodyId = tMessage.%Id()
                                   Do pHeader.%Save()
                               }
                           }
                       }
                   } Else {
                       // If it's not a stream, we try to instantiate it and copy the properties.
                       Set tMessage = $ClassMethod(pHeader.ClientBodyClassName, "%New")
                       Do tMessage.%CopyFrom(pBody)
                       Set tSC = tMessage.%Save()
                      
                       If $$$ISOK(tSC) {
                           Set pHeader.MessageBodyClassName = pHeader.ClientBodyClassName
                           Set pHeader.MessageBodyId = tMessage.%Id()
                           Do pHeader.%Save()
                       }
                   }
               }
           }
       }
      
   }
   Catch ex {
       Return ex.AsStatus()
   }
  
   Return 1
}

ClassMethod GetBankClassName(pOriginalClassName As %String) As %String
{
   Quit pOriginalClassName
}

}

 
Links to Learn More About Message Bank 

If you wish to learn more about the Message Bank functionality of InterSystems IRIS, please explore the following links:

• Official documentation about Message Bank: https://docs.intersystems.com/iris20261/csp/docbook/DocBook.UI.Page.cls?KEY=EMULT_message_bank 
• How to add a Helper Class to change Message Bank behavior: https://docs.intersystems.com/iris20261/csp/docbook/Doc.View.cls?KEY=EMULT_configmb#EMULT_configmb_helper 
• Message Bank explanation on the InterSystems Community page: https://community.intersystems.com/tags/enterprise-message-bank
• Using the Enterprise Message Viewer: https://docs.intersystems.com/healthconnectlatest/csp/docbook/DocBook.UI.Page.cls?KEY=EMONITOR_accessing_emv 
• Message Bank sample application on the Open Exchange: https://openexchange.intersystems.com/package/msg-banking-sample