Hi Motti,

in some projects we faced similar situations. We finally took as Sean mentioned as #2  the Enterprise Message Bank solution and spent some effort in having build specialised SearchTables  - which are populated through Ens.Enterprise.MsgBank.BankHelperClass and implement its OnBankMsg().

The regular message searches are now done in the MessageBank.

And sure, some custom purge methods would help to leave everything in one namespace as you mentioned.

Kind regards,

Markus

In typical scenarios you'll find SNMP to be used for Error Alerting .
SNMP Traps can be issued from the Ensemble installation and send to a centralized monitoring (As far as documentation of Splunk shows it does support SNMP). 

MIB's are also available for monitoring via SNMP. In your case you could think of extending the MIB to support your information requirements (http://docs.intersystems.com/latest/csp/docbook/DocBook.UI.Page.cls?KEY=GCM_snmp#GCM_snmp_mib_extend). 

Ensemble further supports WS-Management API http://docs.intersystems.com/latest/csp/docbook/DocBook.UI.Page.cls?KEY=GCM_wsmon.

The following way might be handy for a one time import.

The ClassMethod %UpdateValue() in Ens.Util.LookupTable is to Update or Insert Entries:
Copying from Excel and pasting it to Terminal (Shift-Insert) will result in Tab delimited Key Value pairs.

The following commands insert one entry into the LookupTable "mytable":

ENSEMBLE>read line
ENSEMBLE>write ##class(Ens.Util.LookupTable).%UpdateValue("mytable",$P(line,$C(9)),$P(line,$C(9),2)

Copying a series of Values directly from Excel might work as follows.

Looping in Terminal using the above idea would be a one line:
ENSEMBLE>do {read line  if line'="" {write ##class(Ens.Util.LookupTable).%UpdateValue("mytable",$P(line,$C(9)),$P(line,$C(9),2),1)}} while line'=""

There are multiple ways as Bredan also pointed out.

If you're Excel specialist ;-) you might want to introduce a third column.

Use the CONCATENATE string function or whatever it is called in your Excel version.

The following will lead to a XML entry which you could copy & paste in an standard XML Export of a lookup table,

So just mockup a Lookiup table, export it. Modify the file with the new entries and Import the resulting file.

Just 2 other cents,

functionGER=VERKETTEN("<entry table=""TestLookupTable"" key=""";B2;""">";C2;"</entry>")
functionENG=CONCATENATE("<entry table=""TestLookupTable"" key=""";B2;""">";C2;"</entry>")
T1V1<entry table="TestLookupTable" key="T1">V1</entry>
T2V2<entry table="TestLookupTable" key="T2">V2</entry>
T3V3<entry table="TestLookupTable" key="T3">V3</entry>
T4V4<entry table="TestLookupTable" key="T4">V4</entry>

Troubleshoot #1

Just in case you find a java UnsatisfiedLinkError...
It seemed that the test installation wrongly choose the 32-bit version.
It was solved on my maschine by reinstalling the Java JRE 64 bit.

This was on Java(TM) SE Runtime Environment (build 1.8.0_92-b14).

java.lang.UnsatisfiedLinkError: C:\....\sapjco3.dll: Can't load AMD 64-bit .dll on a IA 32-bit platform

For reporting as asked in the heading we have standard mechanisms which where already mentioned.
Of course you might want to be more flexible (less standard). Then you might query the virtual documents in scripting directly and act very flexible on errors.

If there has been tried to build a map (correlation to a schema) with the VDoc method BuildMap() the virtualDocument.BuildMapStatus on that virtualDocument is available. This is how the Message Router decides on map errors.

It looks as  for $-notation one needs some DocType to be available, at least this is what %objlasterror tells.
Using [] you might get what you expect.

ENSEMBLE>write !,vDoc.GetValueAt("$1")
ENSEMBLE>write %objlasterror
0 ñ<Ens>ErrGeneral:Can't evaluate property path because DocType  is not set
                                                                             ENSEMBLE,e^zpropParsePath+2^EnsLib.EDI.XML.Prop.1^1-e^zpropGetValueAt+3^EnsLib.EDI.XML.Prop.1^1.e^zGetValueAt+16^EnsLib.EDI.XML.Document.1^1e^^^0

Variants with []-notation:

ENSEMBLE>write !,vDoc.GetValueAt("/Body/[2]")
30
ENSEMBLE>write !,vDoc.GetValueAt("/[1]/[2]")
30

The internal structure of EnsLib.SQL.Snapshot allows an easy direct access to the current row as a Listbuild using EnsLib.SQL.Snapshot.GetRowList(). This could be used to calculate a hash, but you would need to manage this in the business host's code. Because the amount of data to transport is not in all cases small I would suggest or evaluate the following approach:
 Obviously it is not that easy to judge whether a row from an external db has changed or not. The easiest approach if possible at all in such scenarioes would be to ask for an additional field in the external database table which gets filled on the external database side with an last update timestamp or a version id (like Caché has for objects). Then the SQL statement could honor only the new/changed records. This would minimize the amount of data to transport compared to the orignal proposal.

This is an excellent overview with easy to understand examples.

Walking through it I foundthat the "WriteUpdateToTargetDB" method is concatenating the SQL-String which then gets executed against the external database. 

Even if this is just an example it would allow a possible attack of a 2nd order sql injection - if the attacker has knowledge of the system. The easiest way to be more secure is to use placeholders in the sql statement and parameters when calling the external database.
Another advantage of using placeholders is that there is no need for quotation marks.
 

If you disable the Business Operation you will see e.g. the XML file in the message browser.
The inbound file adapter doesn't delete the stream as long as it is in transit.


As far as I know there is no explicit storage in the Ensemble database for long term availability.
As soon as the Business Operation has **successfully** written the stream to it's device the incoming stream is deleted. 
To archive the incoming file in the filesystem, use the setting ArchivePath. To be able to work asynchronously with the business operation look at the setting WorkPath.

To have permanent view to the stream content it would be needed to be available. You might think of building your own message class and inherit from the Passthrough Service. The Passthrough Operation uses the property named Stream in its Request Message.  But be careful with the potential needed amount of storage. The current implementation is intended to achieve a high throughput.


Regards,
Markus