Hi Basem,

I think that you need to make sure that the overall setup here is as robust as possible. To help with that, I think you probably need some application-level tracking that ensures you know when the web service client has successfully retrieved messages. I say this because if the caller makes an API call, but crashes before it finishes processing the response we send, we can think we've sent the response, but the caller may not have processed it correctly.

At a basic level, I think it would make sense to have your Business Operation write to a new persistent class that looks something like the following:

/// Class to expose internal information about messages to an external caller.
Class My.WebService.Tracker Extends %Library.Persistent
{

/// Reference to the original Ensemble Message Header that the caller should be told about.
/// Note that this could also be the actual message, depending on how you want to track which messages have been fetched.
Property MessageHeader As Ens.MessageHeader [ Required ];

}

For each message you want to "queue" up for the caller, you'd create a new instance of this class, reference your message, and then save the value.

You can then write a fairly straightforward web service class that accepts a "Last Processed ID" value that allows the caller to request only new entries in the Tracker table, or even to request old ones to reprocess them. Your logic could simply query My_WebService.Tracker WHERE ID > ? when a last processed ID is supplied. You probably want to specify a maximum number of messages per response, and have some flag in the response to indicate whether more messages exist

The question of working out which messages have been processed gets slightly more complicated. You can track the last returned ID in your web service, though that gets tricky if you have multiple systems fetching from this queue. If you do have multiple systems, you'd need some way of tracking the status of each system. The complexity here really stems from your reporting needs.

As a final question, how important is it that you know when messages are actually received by the other system? If it's really important to see that within Ensemble, you may want to use the "Deferred Sending" mechanism in Ensemble, which is generally described here and specifically described for your business service here. You could store the token for each message in the Tracker table. (Note that using this approach could be tricky for effectively tracking the state for multiple other systems.) Be careful - this means that the original callers will wait until the message has been retrieved by the calling system, which could take a long time. This can lead to complex resource usage patterns in your production, so it may not be a good idea.

My personal preference for SQL is to make use of %SQL.Statement to manage my actual SQL queries, as it allows for *very* flexible query definitions. (You can also use the older %Library.ResultSet classes, but I strongly prefer using the newer code in %SQL.Statement.)

I also like to wrap the underlying query in an API that abstracts out some of my search options, but still allows me to control exactly what is queried.

Here is a sample of a wrapped  query of this kind:

ClassMethod PersonSearch(
  Output pIDList As %Library.ListOfDataTypes,
  Output pHasMore As %Boolean,
  pHomeState As %String = "",
  pOfficeState As %String = "",
  pHasSpouse As %Boolean = "",
  pMaxRows As %Integer = 50) As %Status
{
  Set tStatus = $$$OK
  Try {
    Set pIDList = ##class(%Library.ListOfDataTypes).%New()
    Set pHasMore = 0
    // The selected columns and other clauses (e.g. ORDER BY) can also be affected by parameters
    Set tSQL = "SELECT ID FROM Sample.Person"
    Set tArguments = 0
    Set tWHERE = ""
    If (pHomeState '= "") {
      Set tWHERE = tWHERE _ $listbuild("Home_State = ?")
      Set tArguments = tArguments + 1
      Set tArguments(tArguments) = pHomeState
    }
    If (pOfficeState '= "") {
      Set tWHERE = tWHERE _ $listbuild("Office_State = ?")
      Set tArguments = tArguments + 1
      Set tArguments(tArguments) = pOfficeState
    }
    If (pHasSpouse '= "") {
      Set tWHERE = tWHERE _ $listbuild("Spouse IS " _ $select(''pHasSpouse: "NOT ", 1: "") _ "NULL")
    }
    If (tWHERE '= "") {
       Set tSQL = tSQL _ " WHERE " _ $listtostring(tWHERE," AND ")
    }
    Set tStatement = ##class(%SQL.Statement).%New()
    Set tStatus = tStatement.%Prepare(tSQL)
    If $$$ISERR(tStatus) {
      Quit
    }
    Set tRowCount = 0
    // Use tArguments... to allow the array to be pushed out into individual parameters
    Set tRS = tStatement.%Execute(tArguments...)
    While tRS.%Next() {
      If (pMaxRows) {
        If (tRowCount = pMaxRows)) {
          Set pHasMore = 1
          Quit
        }
        Else {
          Set tRowCount = tRowCount + 1
        }
      }
      // You can reference select columns by name, but can also use variants of Get to retrieve them -- I find the name to be easy to understand
      Do pIDList.Insert(tRS.ID)
    }
    // Check for SQL errors - 0 is all OK, 100 is no more rows
    If (tRS.%SQLCODE && (tRS.%SQLCODE '= 100)) {
      Set tStatus = $$$ERROR($$$SQLCode,tRS.%SQLCODE,tRS.%Message)
      Quit
    }
  }
   Catch ex {
    Set tStatus = ex.AsStatus()
  }
  Quit tStatus

A rather subtle point that I haven't seen discussed here is actually about how TSTART/TCOMMIT/TROLLBACK should be handled when triggering a TROLLBACK from code that may be part of a nested transaction. Given that a lot of the code I write may be called from various contexts and those calling contexts may already have open transactions, my preferred transaction pattern is as follows:

Method SaveSomething() As %Status {
  Set tStatus = $$$OK
  Set tInitTLevel = $TLevel
  Try {
    // Validate input before opening transaction so you can exit before incurring any major overhead
    TSTART
    // Do thing 1
    // Do thing 2
    // Check status or throw exception
    TCOMMIT
  }
  // Handle exception locally due to local cleanup needs
  Catch ex {
    Set tStatus = ex.AsStatus()
  }
  While ($TLevel > tInitTLevel) {
    // Only roll back one transaction level as you make it possible for the caller to decide whether the whole transaction should be rolled back
    TRollback 1
  }
  Quit tStatus
}

(Edited for formatting.)

I prefer doing this more correctly using the right API for matching status values:

If $system.Status.Equals(sc,$$$ERRORCODE($$$GeneralError),$$$ERRORCODE($$$MoreSpecificStatusCode),...) {
  // Handle specific error(s)
}

The use of ' can result in unexpected behaviour when you are checking for a 4 digit code, but are handling a 3 digit code...

Note that it's also safer to wrap status codes in $$$ERRORCODE($$$TheErrorCode), but that may not be necessary depending on the specific context.