Open Exchange App Building an Alternative IRIS Message Viewer

Primary tabs

If you had the opportunity to change something in the IRIS Interoperability Message Viewer, what would you do?

After publishing the article Dashboard IRIS History Monitor, I received some great feedback and some requests. One request was for an enhanced Message Viewer.
 

If you haven’t yet done so, check out the project—it’s definitely worth your time, and it won the Bronze award as one of The Best InterSystems Open Exchange Developers and Applications in 2019.


I started drafting some ideas about the features I’d want to include in the “new” Message Viewer, but how could I show these resources in the fastest and easiest way?


Well, first things first. You generally start by setting up an interoperability production, then exporting and deploying it on the target system, as indicated in the documentation. This is a process I really don’t like. Not that there’s anything wrong with it, really. I’ve just idealized doing everything using code.


I expect that every time someone runs this sort of project, they start like this:

$ docker-compose build

$ docker-compose up -d

And voilá!!!


With those simple steps in mind, I started to look in the InterSystems community and found a few tips. One of the posts brought up the question I was asking myself: How to create productions via routine?

In that post,  @Eduard Lebedyuk  answered, showing how to create a production using code.

"To create production class automatically you need to:

  1. Create %Dictionary.ClassDefinition object for your test production
  2. Create Ens.Config.Production object
  3. Create %Dictionary.XDataDefinition
  4. Serialize (2) into (3)
  5. Insert XData (3) into (1)
  6. Save and compile (1)"

I also found a comment from @Jenny Ames :

 "One best practice we often recommend is to build backward. Build business operations first, then business processes, then business services…"

So, let’s do it!


Requests, Business Operations, and Business Services

The class diashenrique.messageviewer.util.InstallerProduction.cls is, as the name suggests, the class responsible for installing our production. The installer manifest invokes the ClassMethod Install from that class:

/// Helper to install a production to display capabilities of the enhanced viewer

ClassMethod Install() As %Status

{

    Set sc = $$$OK

    Try {

        Set sc = $$$ADDSC(sc,..InstallProduction()) quit:$$$ISERR(sc)

        Set sc = $$$ADDSC(sc,..GenerateMessages()) quit:$$$ISERR(sc)

        Set sc = $$$ADDSC(sc,..GenerateUsingEnsDirector()) quit:$$$ISERR(sc)

    }

    Catch (err) {

        Set sc = $$$ADDSC(sc,err.AsStatus())

    }

    Return sc

}

The classmethod InstallProduction brings together the main structure for creating a production by creating:

  • a request
  • a business operation
  • a business service
  • an interoperability production

Since the idea is to create an interoperability production using code, let’s go into full coding mode to create all classes for the request, the business operation, and the business services. In doing so, we’ll make extensive use of some InterSystems library packages:

  • %Dictionary.ClassDefinition
  • %Dictionary.PropertyDefinition
  • %Dictionary.XDataDefinition
  • %Dictionary.MethodDefinition
  • %Dictionary.ParameterDefinition

The classmethod InstallProduction creates two classes that extend from Ens.Request, using the following lines:

 

Set sc = $$$ADDSC(sc,..CreateRequest("diashenrique.messageviewer.Message.SimpleRequest","Message")) quit:$$$ISERR(sc)

Set sc = $$$ADDSC(sc,..CreateRequest("diashenrique.messageviewer.Message.AnotherRequest","Something")) quit:$$$ISERR(sc)

ClassMethod CreateRequest(classname As %String, prop As %String) As %Status [ Private ]

{

    New $Namespace

    Set $Namespace = ..#NAMESPACE

    Set sc = $$$OK

    Try {

        Set class = ##class(%Dictionary.ClassDefinition).%New(classname)

        Set class.GeneratedBy = $ClassName()

        Set class.Super = "Ens.Request"

        Set class.ProcedureBlock = 1

        Set class.Inheritance = "left"

        Set sc = $$$ADDSC(sc,class.%Save())

        #; create adapter

        Set property = ##class(%Dictionary.PropertyDefinition).%New(classname)

        Set property.Name = prop

        Set property.Type = "%String"

        Set sc = $$$ADDSC(sc,property.%Save())

        Set sc = $$$ADDSC(sc,$System.OBJ.Compile(classname,"fck-dv"))

    }

    Catch (err) {

        Set sc = $$$ADDSC(sc,err.AsStatus())

    }

    Return sc

}

 

Now let’s create the class for a business operation that extends from Ens.BusinessOperation:

Set sc = $$$ADDSC(sc,..CreateOperation()) quit:$$$ISERR(sc)

Besides creating the class, we create the MessageMap and the method Consume:

ClassMethod CreateOperation() As %Status [ Private ]
{
    New $Namespace
    Set $Namespace = ..#NAMESPACE
    Set sc = $$$OK
    Try {
        Set classname = "diashenrique.messageviewer.Operation.Consumer"
        Set class = ##class(%Dictionary.ClassDefinition).%New(classname)
        Set class.GeneratedBy = $ClassName()
        Set class.Super = "Ens.BusinessOperation"
        Set class.ProcedureBlock = 1
        Set class.Inheritance = "left"
 
        Set xdata = ##class(%Dictionary.XDataDefinition).%New()
        Set xdata.Name = "MessageMap"
        Set xdata.XMLNamespace = "http://www.intersystems.com/urlmap"
        Do xdata.Data.WriteLine("<MapItems>")
        Do xdata.Data.WriteLine("<MapItem MessageType=""diashenrique.messageviewer.Message.SimpleRequest"">")
        Do xdata.Data.WriteLine("<Method>Consume</Method>")
        Do xdata.Data.WriteLine("</MapItem>")
        Do xdata.Data.WriteLine("<MapItem MessageType=""diashenrique.messageviewer.Message.AnotherRequest"">")
        Do xdata.Data.WriteLine("<Method>Consume</Method>")
        Do xdata.Data.WriteLine("</MapItem>")
        Do xdata.Data.WriteLine("</MapItems>")      
        Do class.XDatas.Insert(xdata)
        Set sc = $$$ADDSC(sc,class.%Save())
 
        Set method = ##class(%Dictionary.MethodDefinition).%New(classname)
        Set method.Name = "Consume"
        Set method.ClassMethod = 0
        Set method.ReturnType = "%Status"
        Set method.FormalSpec = "input:diashenrique.messageviewer.Message.SimpleRequest,&output:Ens.Response"
        Set stream = ##class(%Stream.TmpCharacter).%New()
        Do stream.WriteLine("   set sc = $$$OK")
        Do stream.WriteLine("   $$$TRACE(input.Message)")
        Do stream.WriteLine("   return sc")
        Set method.Implementation = stream
        Set sc = $$$ADDSC(sc,method.%Save())
 
        Set sc = $$$ADDSC(sc,$System.OBJ.Compile(classname,"fck-dv"))
    }
    Catch (err) {
        Set sc = $$$ADDSC(sc,err.AsStatus())
    }
    Return sc
}

In the last step before actually creating the interoperability production, let’s create the class responsible for the business service:

Set sc = $$$ADDSC(sc,..CreateRESTService()) quit:$$$ISERR(sc)

This class has UrlMap and Routes to receive Http requests.

ClassMethod CreateRESTService() As %Status [ Private ]
{
    New $Namespace
    Set $Namespace = ..#NAMESPACE
    Set sc = $$$OK
    Try {
        Set classname = "diashenrique.messageviewer.Service.REST"
        Set class = ##class(%Dictionary.ClassDefinition).%New(classname)
        Set class.GeneratedBy = $ClassName()
        Set class.Super = "EnsLib.REST.Service, Ens.BusinessService"
        Set class.ProcedureBlock = 1
        Set class.Inheritance = "left"
 
        Set xdata = ##class(%Dictionary.XDataDefinition).%New()
        Set xdata.Name = "UrlMap"
        Set xdata.XMLNamespace = "http://www.intersystems.com/urlmap"
        Do xdata.Data.WriteLine("<Routes>")
        Do xdata.Data.WriteLine("<Route Url=""/send/message"" Method=""POST"" Call=""SendMessage""/>")
        Do xdata.Data.WriteLine("<Route Url=""/send/something"" Method=""POST"" Call=""SendSomething""/>")
        Do xdata.Data.WriteLine("</Routes>")
        Do class.XDatas.Insert(xdata)
        Set sc = $$$ADDSC(sc,class.%Save())
 
        #; create adapter
        Set adapter = ##class(%Dictionary.ParameterDefinition).%New(classname)
        Set class.GeneratedBy = $ClassName()
        Set adapter.Name = "ADAPTER"
        Set adapter.SequenceNumber = 1
        Set adapter.Default = "EnsLib.HTTP.InboundAdapter"
        Set sc = $$$ADDSC(sc,adapter.%Save())
 
        #; add prefix
        Set prefix = ##class(%Dictionary.ParameterDefinition).%New(classname)
        Set prefix.Name = "EnsServicePrefix"
        Set prefix.SequenceNumber = 2
        Set prefix.Default = "|demoiris"
        Set sc = $$$ADDSC(sc,prefix.%Save())
 
        Set method = ##class(%Dictionary.MethodDefinition).%New(classname)
        Set method.Name = "SendMessage"
        Set method.ClassMethod = 0
        Set method.ReturnType = "%Status"
        Set method.FormalSpec = "input:%Library.AbstractStream,&output:%Stream.Object"
        Set stream = ##class(%Stream.TmpCharacter).%New()
        Do stream.WriteLine("   set sc = $$$OK")
        Do stream.WriteLine("   set request = ##class(diashenrique.messageviewer.Message.SimpleRequest).%New()")
        Do stream.WriteLine("   set data = {}.%FromJSON(input)")
        Do stream.WriteLine("   set request.Message = data.Message")
        Do stream.WriteLine("   set sc = $$$ADDSC(sc,..SendRequestSync(""diashenrique.messageviewer.Operation.Consumer"",request,.response))")
        Do stream.WriteLine("   return sc")
        Set method.Implementation = stream
        Set sc = $$$ADDSC(sc,method.%Save())
 
        Set method = ##class(%Dictionary.MethodDefinition).%New(classname)
        Set method.Name = "SendSomething"
        Set method.ClassMethod = 0
        Set method.ReturnType = "%Status"
        Set method.FormalSpec = "input:%Library.AbstractStream,&output:%Stream.Object"
        Set stream = ##class(%Stream.TmpCharacter).%New()
        Do stream.WriteLine("   set sc = $$$OK")
        Do stream.WriteLine("   set request = ##class(diashenrique.messageviewer.Message.AnotherRequest).%New()")
        Do stream.WriteLine("   set data = {}.%FromJSON(input)")
        Do stream.WriteLine("   set request.Something = data.Something")
        Do stream.WriteLine("   set sc = $$$ADDSC(sc,..SendRequestSync(""diashenrique.messageviewer.Operation.Consumer"",request,.response))")
        Do stream.WriteLine("   return sc")
        Set method.Implementation = stream
        Set sc = $$$ADDSC(sc,method.%Save())
 
        Set sc = $$$ADDSC(sc,$System.OBJ.Compile(classname,"fck-dv"))
    }
    Catch (err) {
        Set sc = $$$ADDSC(sc,err.AsStatus())
    }
    Return sc
}

 

Using Visual Studio Code


Creating the classes using the %Dictionary package can be difficult, and difficult to read as well, but it’s handy. To demonstrate a slightly more straightforward approach with better code readability, I’ll also create new request, business service, and business operations classes using Visual Studio Code:

  • diashenrique.messageviewer.Message.SimpleMessage.cls
  • diashenrique.messageviewer.Operation.ConsumeMessageClass.cls
  • diashenrique.messageviewer.Service.SendMessage.cls
Class diashenrique.messageviewer.Message.SimpleMessage Extends Ens.Request [ Inheritance = left, ProcedureBlock ]
{
Property ClassMessage As %String;
}

Class diashenrique.messageviewer.Operation.ConsumeMessageClass Extends Ens.BusinessOperation [ Inheritance = left, ProcedureBlock ]
{
Method Consume(input As diashenrique.messageviewer.Message.SimpleMessage, ByRef output As Ens.Response) As %Status
{
    Set sc = $$$OK
    $$$TRACE(pRequest.ClassMessage)
    Return sc
}
XData MessageMap [ XMLNamespace = "http://www.intersystems.com/urlmap" ]
{
  <MapItems>
    <MapItem MessageType="diashenrique.messageviewer.Message.SimpleMessage">
      <Method>Consume</Method>
    </MapItem>
  </MapItems>
} 
}

Class diashenrique.messageviewer.Service.SendMessage Extends Ens.BusinessService [ ProcedureBlock ]
{ 
Method OnProcessInput(input As %Library.AbstractStream, ByRef output As %Stream.Object) As %Status
{
    Set tSC = $$$OK
    // Create the request message
    Set request = ##class(diashenrique.messageviewer.Message.SimpleMessage).%New()
    // Place a value in the request message property
    Set request.ClassMessage = input
    // Make a synchronous call to the business process and use the response message as our response 
    Set tSC = ..SendRequestSync("diashenrique.messageviewer.Operation.ConsumeMessageClass",request,.output)
    Quit tSC
}
}

From a code readability perspective, it’s a huge difference! 

Creating the Interoperability Production

Let’s finish up the creation of our interoperability production. To do so, we’ll create a production class, then associate it with the business Operation and Service classes.

Set sc = $$$ADDSC(sc,..CreateProduction()) quit:$$$ISERR(sc)

ClassMethod CreateProduction(purge As %Boolean = 0) As %Status [ Private ]
{
    New $Namespace
    Set $Namespace = ..#NAMESPACE
    Set sc = $$$OK
    Try {
         #; create new production
        Set class = ##class(%Dictionary.ClassDefinition).%New(..#PRODUCTION)
        Set class.ProcedureBlock = 1
        Set class.Super = "Ens.Production"
        Set class.GeneratedBy = $ClassName()
        Set xdata = ##class(%Dictionary.XDataDefinition).%New()
        Set xdata.Name = "ProductionDefinition"
        Do xdata.Data.Write("<Production Name="""_..#PRODUCTION_""" LogGeneralTraceEvents=""true""></Production>")  
        Do class.XDatas.Insert(xdata)
        Set sc = $$$ADDSC(sc,class.%Save())
        Set sc = $$$ADDSC(sc,$System.OBJ.Compile(..#PRODUCTION,"fck-dv"))
        Set production = ##class(Ens.Config.Production).%OpenId(..#PRODUCTION)
        Set item = ##class(Ens.Config.Item).%New()
        Set item.ClassName = "diashenrique.messageviewer.Service.REST"
        Do production.Items.Insert(item)
        Set sc = $$$ADDSC(sc,production.%Save())
        Set item = ##class(Ens.Config.Item).%New()
        Set item.ClassName = "diashenrique.messageviewer.Operation.Consumer"
        Do production.Items.Insert(item)
        Set sc = $$$ADDSC(sc,production.%Save())    
        Set item = ##class(Ens.Config.Item).%New()
        Set item.ClassName = "diashenrique.messageviewer.Service.SendMessage"
        Do production.Items.Insert(item)
        Set sc = $$$ADDSC(sc,production.%Save())    
        Set item = ##class(Ens.Config.Item).%New()
        Set item.ClassName = "diashenrique.messageviewer.Operation.ConsumeMessageClass"
        Do production.Items.Insert(item)
        Set sc = $$$ADDSC(sc,production.%Save())    
    }
    Catch (err) {
        Set sc = $$$ADDSC(sc,err.AsStatus())
    }
    Return sc
}

We use the class Ens.Config.Item to associate the production class with the business Operation and Service classes. You can do this whether you created your class using the %Dictionary package or with VS Code, Studio, or Atelier.

In any case, we did it! We created an interoperability production using code.

But remember the original purpose of all this code: to create a production and messages to show the capabilities of the enhanced Message Viewer. Using the classmethods that follow, we’ll execute both of our business services and generate the messages. 

Generating Messages using %Net.HttpRequest:

ClassMethod GenerateMessages() As %Status [ Private ]
{
    New $Namespace
    Set $Namespace = ..#NAMESPACE
    Set sc = $$$OK
    Try {
        Set action(0) = "/demoiris/send/message"
        Set action(1) = "/demoiris/send/something"
        For i=1:1:..#LIMIT {
            Set content = { }
            Set content.Message = "Hi, I'm just a random message named "_$Random(30000)
            Set content.Something = "Hi, I'm just a random something named "_$Random(30000)
            Set httprequest = ##class(%Net.HttpRequest).%New()
            Set httprequest.SSLCheckServerIdentity = 0
            Set httprequest.SSLConfiguration = ""
            Set httprequest.Https = 0
            Set httprequest.Server = "localhost"
            Set httprequest.Port = 9980
            Set serverUrl = action($Random(2))
            Do httprequest.EntityBody.Write(content.%ToJSON())
            Set sc = httprequest.Post(serverUrl) 
            Quit:$$$ISERR(sc)
        }
    }
    Catch (err) {
        Set sc = $$$ADDSC(sc,err.AsStatus())
    }
    Return sc
}

Generating Messages using EnsDirector:

ClassMethod GenerateUsingEnsDirector() As %Status [ Private ]
{
    New $Namespace
    Set $Namespace = ..#NAMESPACE
    Set sc = $$$OK
    Try {
        For i=1:1:..#LIMIT {
            Set tSC = ##class(Ens.Director).CreateBusinessService("diashenrique.messageviewer.Service.SendMessage",.tService)
            Set message = "Message Generated By CreateBusinessService "_$Random(1000)
            Set tSC = tService.ProcessInput(message,.output)
            Quit:$$$ISERR(sc)
        }
    }
    Catch (err) {
        Set sc = $$$ADDSC(sc,err.AsStatus())
    }
    Return sc
}
 
}

That’s it for the code. You’ll find the complete project at https://github.com/diashenrique/iris-message-viewer

Running the Project

Now let’s see the project in action. First, git clone or git pull the repo into any local directory:

git clone https://github.com/diashenrique/iris-message-viewer.git

Next, open the terminal in this directory and run:

docker-compose build

Finally, run the IRIS container with your project:

docker-compose up -d

Now we’ll access the Management Portal using http://localhost:52773/csp/sys/UtilHome.csp. You should see our interoperability namespace MSGVIEWER, as in the image below:

And here’s our baby Production, with two business services and two business operations:

We have so many messages:

With everything up and running in our custom Message Viewer, let’s take a look at some of its features.

The Enhanced Message Viewer

Keep in mind that only namespaces that are enabled for interoperability productions will be displayed.

http://localhost:52773/csp/msgviewer/messageviewer.csp

Interoperability Message Viewer
 

The enhanced Message Viewer brings features and flexibility that allow you to create different filters, group the columns into n-levels, export to Excel, and much more.
Interoperability Message Viewer
You can use different filters to achieve the results you need. You can also use multiple sorts by pressing Shift and clicking on the column header. You even export the data grid to Excel!

In addition, you can create complex filters with the filter builder option.


You can group data against any column available, grouping the information using the n-levels you want. By default, the group is constructed using the Date Created field.
Group Data

 

And there’s a feature that allows you to select columns. The following page has all the columns from Ens.MessageHeader, showing only the default columns in the initial view. But you can choose the other columns using the "Column Chooser" button.

Column Chooser

You can collapse or expand all groups with a single click.

The information in the SessionId field has a link to the Visual Trace feature.

Visual Trace

You can resend messages if you need to. Simply select the messages you need and click to resend. This feature uses the following classMethod:

##class(Ens.MessageHeader).ResendDuplicatedMessage(id)

Finally, as mentioned, you can export your data grid to Excel:

The result in Excel will show the same format, content, and group defined in the cache server pages (CSP).

PS: I want to give special thanks to @Rhenan Lourenco  who helped me a lot on this journey.

 

Replies

Henrique!

Thank you so much for sharing this fantastic addon! I hope developers will benefit from it very much and even contribute!

Small hint: I haven't found the link in the article where the Enhanced Viewer can be opened - here is the link if you run with docker: http://localhost:52773/csp/msgviewer/messageviewer.csp

I apologize for the lack of the link. I ended up extending myself in the article and ended up just informing the Management Portal link.

I'm fixing it right now.

Thank you for your kind words

 

Tried in VSCode with docker-compose: works like a charm, great job!

Great project, @Henrique Gonçalves Dias!

Sent you a small pr.

I have three questions:

  1. Why CSP instead of REST? It doesn't seem too highload of an app, being an admin tool essentially, so I just wonder why you decided to go with CSP.
  2. Let's move filtering to the server. Currently only the last 200 messages can be retrieved.
  3. Are there any plans to add visual trace? Our default visual trace is great, but when a single session exceeds 300-500 thousands of messages it's not as responsive. So I've been searching for a enhanced tool for session viewing.

Hi @Eduard Lebedyuk 

Thanks for your PR on GitHub.

Answering your questions:
1. I'm always trying to make things simple and using those opportunities to make Caché/IRIS to shiny a little bit in my work environment. And choosing CSP over REST+Angular or any other technology stack is one of those attempts to show what CSP (Caché/IRIS) is capable of creating. I can say that it was a personal choice.

2. I'm improving these points, and I'll publish the fix as soon as possible.

3. Right now, I don't have any plans to create a new visual trace. Unfortunately, I didn't face a situation as you mention "...  single session exceeds 300-500 thousands of messages ..." But, I'll search for something that can improve your scenario.

Thank you again for spending your time with my article and my application.

Hi Henrique,

Great work.

Just few points as my 2 cents

1. I am assuming you are using the js.devexpress js files to build this. As much as I know, it comes with licencing.  You should include that in your post / documentation so nobody gets in trouble without realizing they have been using a licensed library without paying. 

2. Are you pulling the data via sql calls through csp ?? As Eduard said, why csp not rest.  but with csp are you able to provide the login credentials and roles as needed to view this information as this is admin privilege. 

3. Yes, breaking the visual trace is a big problem. We should be able to define break points. I give you example of my own integration.

We get one edi. We sent to 3 operations. That do their thing.

Then we call delegate rule which calls further 4 more processes, who internally call 3-4 more ops and   processes. And this is just a start, it may become more complex as we have more requirements. If i am able to break it somehow, it will be awesome. 

Another option is to change my whole logic. somehow make a new session at some point on every send of delegate (Which I currently am not aware how to do)

@Eduard Lebedyuk may know it.

But as I said, a great work.
 

Hi Neerav, 

Thanks for pointing the license situation. But, as we discussed before in LinkedIn messages and the post of History Monitor. 

The license for the DevExtreme is free to develop Non-Commercial applications. I updated the README.md file to make the situation clear following the instructions provided by the DevExtreme website.

https://js.devexpress.com/Licensing/#NonCommercial

Yes, agreed. It is good for people to know so they can make their own decision.

Did you think about my suggestion of breaking down the visual trace?

Thanks

Sure,

Visual Trace appears to need improvements in huge productions. But, right now, I'm just collecting information to found out a way to create something different.

This maybe a silly question but what if you are not using a docker image? What is docker-compose? 

Load the code and open main csp page. That's enough for the project to work.

Hi Scott, 

Thanks for your interest in my project.

A similar question happened with the History Monitor Dashboard. 

https://community.intersystems.com/post/dashboard-iris-history-monitor#comment-73721

You can choose to import the code in an existent Namespace, and just like Eduard replied before, for the non-docker environment, import the code in your Namespace and access the main csp. 

Or you can choose to install the Message Viewer in a new Namespace and make use of the Installer.cls.

If that was your choice change the parameters in the Installer.cls

<Default Name="APPPATH" Dir="/opt/app/" /> 

Change the /opt/app/ to whatever directory you want.

Next, you can use the class Installer.cls to create the Database, Namespace, and Web Application.

So, run the following lines of the Dockerfile in the Caché/Ensemblé/IRIS Terminal: 

Do $system.OBJ.Load("/opt/app/Installer.cls","ck")

Set sc = ##class(App.Installer).setup(,3)

HTH

Answering the question, "What is docker-compose?"

"Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application's services. Then, with a single command, you create and start all the services from your configuration."

for more info: https://docs.docker.com/compose/