One possibility:

You can handle the two files with two separate interfaces:

The interface for the tracking file would just load it into a record map and then stop. This has the effect of saving the information as a row in a database table.

The BPL for the PO file interface would just query the tracking file record map table to find the relevant entry, open it, and add the necessary info to the PO. To avoid timing issues where a PO file is processed before the corresponding tracking file you could add a check in the BPL: if a row doesn't exist in the tracking table for this PO, delay for 5 seconds using the BPL "Delay" action and try again before failing with an error.

Have a look at the StayConnected setting under Connection Settings. By default, it is set to -1 which means the adapter expects to always have an active connection and will throw an error if it doesn't.

Setting StayConnected to 0 would mean the remote system can connect and disconnect as needed without triggering an error.

Stay Connected

Applies to all TCP adapters.

If StayConnected is a positive value, the adapter stays connected to the remote system for this number of seconds between input events. A zero value means to disconnect immediately after every input event. The default of –1 means to stay permanently connected, even during idle times. Adapters are assumed idle at startup and therefore only auto-connect if they are configured with a StayConnected value of –1.

The value of StayConnected controls how the TCP adapter treats disconnections. If StayConnected
has a value of –1, the TCP adapter treats a disconnection as an error. If it has a value of 0 or a positive integer, the TCP adapter does not consider a disconnection an error.

$TRANSLATE might be a possibility. It accepts a list of characters and replaces them either with other characters or just removes them. You could compare the length of the original column with the length of the column after using $TRANSLATE to remove illegal characters. For rows without illegal characters the length will match.

This would identify rows that have tilde (~), pipe (|), or backtick (`) in MyField:

SELECT * FROM MyTable WHERE CHAR_LENGTH($TRANSLATE(MyField,'~`|'))  <  CHAR_LENGTH(MyField)

It's worth noting that a statement like this can't make use of indices, so it will have to scan every row in the table.

I would be tempted to, as a next troubleshooting step, take the BPL and JDBCGateway out of the equation and test the stored procedure calls using the same JDBC driver from a Java-based SQL query tool such as Squirrel.

It's not SQL Server specific, but this step-by-step walkthrough for using Squirrel to connect to Caché might save some time.

[Edit: should have said "queries of the views" rather than "stored procedure calls"]

It looks like that error message is coming directly from SQL Server and has something to do with the linked-server config within SQL Server:

https://docs.microsoft.com/en-us/archive/blogs/mdegre/access-to-the-remo...

Business rule classes store the rules as XML in an XData block with the name "RuleDefinition":

Class Demo.Rule.GenericRouter Extends Ens.Rule.Definition
{

Parameter RuleAssistClass = "EnsLib.MsgRouter.RuleAssist";

XData RuleDefinition [ XMLNamespace = "http://www.intersystems.com/rule" ]
{
<ruleDefinition alias="" context="EnsLib.MsgRouter.RoutingEngine" production="TESTINGPKG.FoundationProduction">
<ruleSet name="" effectiveBegin="" effectiveEnd="">
<rule name="">
<when condition="1">
<send transform="Demo.DTL.Generic" target="Test.DummyOperation"></send>
<return></return>
</when>
</rule>
</ruleSet>
</ruleDefinition>
}

}

One approach would be to find all of your business rule classes, retrieve the "RuleDefinition" XData block for each one, and then parse the XML to see which DTLs are called.

To find the business rule classes, have a look at %Dictionary.ClassDefinition. You could do a query like this to find your business rule classes:

select ID from %Dictionary.ClassDefinition where super='Ens.Rule.Definition'

Then, for each business rule class you can find the %Dictionary.XDataDefinition for the RuleDefinition XData block with a query like this:

select ID from %Dictionary.XDataDefinition where parent='Demo.Rule.GenericRouter' and name='RuleDefinition'

The raw XML from the XData block can be accessed via the stream object stored in the "Data" property of the %Dictionary.XDataDefinition row.

Have you considered using web services to exchange messages between the productions? This has the advantage of allowing the two productions to later be placed on separate instances.

If you want to do it through the mapped DB, you could write a custom business operation to store the records in a custom table which is mapped to from both namespaces and then write a custom business service that polls that table for new entries.

I have 2 productions A and B on the same IRIS instance sharing one operational database.

To be sure the two productions aren't conflicting with each other, I'm assuming that only custom tables/globals are mapped to the shared database and not any of the Ens* globals/tables/routines/packages.

The error is complaining that the message doesn't have a doctype category.

In the message trace for this test message, under "Body", what is the value of the "DocType" field?

Are these standard HL7 batch headers and footers (BHS/BTS, FHS/FTS, etc) or something custom to your organization?

If they are one of the HL7 standards, have a look at these docs on HL7 batches:
https://docs.intersystems.com/healthconnect20191/csp/docbook/Doc.View.cl...

One tricky part will be triggering when an old batch ends/new batch begins -- this will depend on your local requirements.

If they are custom, there are a few approaches you can consider:

1. You could use the Record Mapper to define a record map class with header/footer and a single field record for the HL7 content. You would then use one of the  EnsLib.RecordMap.Operation.* classes instead of EnsLib.HL7.Operation.FileOperation: https://docs.intersystems.com/healthconnect20191/csp/docbook/DocBook.UI.... In particular, have a look at RolloverSchedule and/or RolloverLimit to control when a new batch file is created.
2. If you're comfortable creating a custom class that extends EnsLib.HL7.Operation.FileOperation, you could override the outputDocument method. In your custom version of outputDocument you could check if the file already exists using ##class(%File).Exists(pathToFile) and if it doesn't you would write out the footer to the previous file and the header to the new file before calling the standard version of outputDocument using ##super. https://docs.intersystems.com/healthconnect20191/csp/docbook/Doc.View.cl...

A few questions:

• You didn't mention what format these files are in -- XML? flat-file/CSV? Something else?
• Which InterSystems product are you working with and what kind of application is this (interoperability production, web service, etc.)?

If these are flat-files or CSV, and if you're working with a Health Connect or IRIS interoperability production, you can look at using the Record Mapper, which will read a flat file using a format you define and allow you to work with records from the file as objects:
https://docs.intersystems.com/healthconnect20191/csp/docbook/DocBook.UI....

If the files are XML you can do something similar by importing an XSD and using XML virtual documents, or use %XML.Adaptor methods:
https://docs.intersystems.com/healthconnect20191/csp/docbook/DocBook.UI....

How about:

&sql(select count(*) into :tDefined from ENS_Config.Item where Name=:tHostName and Production=:tProductionName)

Ok, EnsLib.REST.GenericOperation also expects to receive an EnsLib.HTTP.GenericMessage.

Are you using EnsLib.File.PassthroughService to pick up the files? If so, it is sending an Ens.StreamContainer message to the target component (your business operation). You'll need to create a data transformation that creates a new EnsLib.HTTP.GenericMessage and populates it with the stream content from the Ens.StreamContainer. Then you'll need a router in the middle to run the data transformation and send the resulting EnsLib.HTTP.GenericMessage to the business operation.

Neerav, I'll just note that if you use Jenna's suggestion your code doesn't need to be a part of a business service -- it can be any normal ObjectScript code and can be called from anywhere (CSP, scheduled task, etc.). And your code can make as many calls as it needs to and can receive response messages.

It will appear in message traces as if a business service sent requests to the process or operation, but in reality it's just your code sending the requests.

Jenna's approach is a good one, and it's the standard way to achieve what you describe. Having said that, if you can provide more details on your use case and/or why this approach doesn't fit your need we can help you explore alternatives.

Which business service are you using to read the file? Are you using EnsLib.RecordMap.Service.FileService and specifying the record map type in the configuration?

It sounds like you might be using EnsLib.File.PassthroughService

You could use Apache's mod_rewrite to take all requests that fall under a certain sub-path and transform them on the fly to point to your CSP page. It could add the information about which specific page/resource was requested as a URL parameter that could be accessed by your CSP page.

For example, if a client makes a request for:
http://myserver/reporting/Dashboard1/resource2.js

mod_rewrite could change it to:
http://myserver/csp/dashboards/proxy.csp?targetResource=Dashboard1/resou...

After mod_rewrite changes the URL in the request, Apache continues processing it as usual using the new URL. Since the new URL refers to a CSP page Apache will pass it to CSP Gateway as we want.

Here's a stored procedure that accepts a setting name and returns the setting value for all components that have it. It's not SQL, but can be executed from SQL :)

You can call it this way -- this example returns the port setting for all components that have it:

call Sample.Util_SettingsByName('Port')

Here's the source code as XML export format. Copy this into a file and then import it using Studio, terminal, or the System Management Portal.

<?xml version="1.0" encoding="UTF-8"?>
<Export generator="Cache" version="25" zv="Cache for Windows (x86-64) 2016.1.1 (Build 108U)" ts="2016-10-12 16:15:39">
<Class name="Sample.Util">
<Super>%RegisteredObject</Super>
<TimeChanged>64203,58380.929948</TimeChanged>
<TimeCreated>64202,44682.614801</TimeCreated>

<UDLText name="T">
<Content><![CDATA[
/*
         *****************************************************
         *                 ** N O T I C E **                 *
         *               - TEST/DEMO SOFTWARE -              *
         * This and related items are not supported by       *
         * InterSystems as part of any released product.     *
         * It is supplied by InterSystems as a demo/test     *
         * tool for a specific product and version.          *
         * The user or customer is fully responsible for     *
         * the maintenance of this software after delivery,  *
         * and InterSystems shall bear no responsibility nor *
         * liabilities for errors or misuse of this item.    *
         *                                                   *
         *****************************************************
*/
]]></Content>
</UDLText>

<Query name="SettingsByName">
<Type>%Query</Type>
<FormalSpec>SettingName:%String</FormalSpec>
<SqlProc>1</SqlProc>
<Parameter name="ROWSPEC" value="BusinessHost:%String,SettingName:%String,SettingValue:%String"/>
</Query>

<Method name="SettingsByNameExecute">
<ClassMethod>1</ClassMethod>
<FormalSpec><![CDATA[&qHandle:%Binary,SettingNames:%String=""]]></FormalSpec>
<ReturnType>%Status</ReturnType>
<Implementation><![CDATA[
    s qHandle=##class(%ArrayOfObjects).%New()

    &sql(select %DLIST(id) into :tHostIDs from ENS_Config.Item order by Name desc)
    s tHostIDList=##class(%Library.ListOfDataTypes).%New()
    s tSC=tHostIDList.InsertList(tHostIDs)
    s tSC=qHandle.SetAt(tHostIDList,"HostIDs")

    s tSC=qHandle.SetAt(##class(%ArrayOfDataTypes).%New(),"Counters")
    s tSC=qHandle.GetAt("Counters").SetAt(0,"CurrHost")
    s tSC=qHandle.GetAt("Counters").SetAt(0,"CurrSetting")
    
    if ($L(SettingNames)>1) {
        s SettingNames=$ZCONVERT(SettingNames,"U")
        s tFilterList=##class(%Library.ListOfDataTypes).%New()
        s tSC=tFilterList.InsertList($LISTFROMSTRING(SettingNames))
         s tSC=qHandle.SetAt(tFilterList,"FilterList")
    }

    Quit $$$OK
]]></Implementation>
</Method>

<Method name="SettingsByNameClose">
<ClassMethod>1</ClassMethod>
<FormalSpec><![CDATA[&qHandle:%Binary]]></FormalSpec>
<PlaceAfter>SettingsByNameExecute</PlaceAfter>
<ReturnType>%Status</ReturnType>
<Implementation><![CDATA[    Quit $$$OK
]]></Implementation>
</Method>

<Method name="SettingsByNameFetch">
<ClassMethod>1</ClassMethod>
<FormalSpec><![CDATA[&qHandle:%Binary,&Row:%List,&AtEnd:%Integer=0]]></FormalSpec>
<PlaceAfter>SettingsByNameExecute</PlaceAfter>
<ReturnType>%Status</ReturnType>
<Implementation><![CDATA[
    s tCurrHost=qHandle.GetAt("Counters").GetAt("CurrHost")
    s tCurrSetting=qHandle.GetAt("Counters").GetAt("CurrSetting")
    s tHostIDs=qHandle.GetAt("HostIDs")
    s tFilterList=qHandle.GetAt("FilterList")
    s oHost=qHandle.GetAt("Host")

    do {
        if ('$IsObject(oHost)||(oHost.VirtualSettings.Count()<tCurrSetting)) {
            if (tCurrHost=tHostIDs.Count()) {
                s AtEnd=1
                q
            }

            s tCurrHost=tCurrHost+1
            s tCurrSetting=1
            
            s tHostID=tHostIDs.GetAt(tCurrHost)
            s oHost=##class(Ens.Config.Item).%OpenId(tHostID,0)
            
            s tSC=oHost.PopulateVirtualSettings()

            s tSC=qHandle.SetAt(oHost,"Host")                
            s tSC=qHandle.GetAt("Counters").SetAt(tCurrHost,"CurrHost")
        }

        s tSettings=oHost.VirtualSettings
        s tSetting=tSettings.GetAt(tCurrSetting)
        s tStngName=$LISTGET(tSetting,2)
        s tStngValue=$LISTGET(tSetting,3)
        
        s tCurrSetting=tCurrSetting+1
    } while ($IsObject(tFilterList)&&('tFilterList.Find($ZCONVERT(tStngName,"U"))))
        
    if ('AtEnd) {
        s Row=$LB(oHost.Name,tStngName,tStngValue)
    }
    
    s tSC=qHandle.GetAt("Counters").SetAt(tCurrSetting,"CurrSetting")
        
    Quit $$$OK
]]></Implementation>
</Method>
</Class>
</Export>

Ens.StreamContainer's %New() method expects a string as the first parameter rather than a stream object.

Something like this should work:

set tRequest=##class(Ens.StreamContainer).%New()
set tSC=tRequest.StreamSet(pInput)

Or if you're trying to send one Ens.StreamContainer for each line from the input file you could do this:

while 'pInput.AtEnd {
    set tReadLength=32000
    set tLine=pInput.ReadLine(.tReadLength,.tSC)
    set tRequest=##class(Ens.StreamContainer).%New(tLine)
    //... do other stuff
}

One other thing you should be aware of. The following will not work if TargetConfigNames has more than one target selected:

set tSC = ..SendRequestAsync(..TargetConfigNames,tRequest)

You should add a loop using $LENGTH and $PIECE and do a SendRequestAsync for each item in TargetConfigNames' comma-separated string.

The resulting XML files can be imported again using %System.OBJ.Load().

If you prefer GOF format you can use %Global.Export() instead, however it doesn't accept wildcards so you would need to first put together a list of which globals you want to export.

For automation you can execute these methods from your own custom class or routine. If you want to schedule it to run automatically, you can create your custom class as a %SYS.Task.Definition and schedule it to run using Task Manager.

You're correct -- you can't restore specific globals from an Online Backup (.cbk) file.

Have a look at %System.OBJ.Export():
https://irisdocs.intersystems.com/irisforhealth20192/csp/documatic/%25CSP.Documatic.cls

TESTING>s itm="*.GBL"
TESTING>s file="C:\Projects\export_mm.xml"
TESTING>d $System.OBJ.Export(.itm,.file,,.errors)

Having said that, since this is just for backups you might also consider just using the standard Online Backup utility which will backup all globals in the selected database(s) into a file:

https://irisdocs.intersystems.com/irisforhealth20192/csp/docbook/DocBook.UI.Page.cls?KEY=GCDI_backup#GCDI_backup_methods_online

Here's a (very simple) working example of uploading to an AWS S3 bucket. I used essentially the same approach that Cliff originally used to generate the hex. I didn't experience the same problem, but I may have just been lucky and not had any zeroes in the value I'm converting.

<?xml version="1.0" encoding="UTF-8"?>
<Export generator="Cache" version="25" zv="Cache for Windows (x86-64) 2017.2.1 (Build 801_3U)" ts="2018-10-04 08:14:32">
<Class name="Demo.Cloud.Storage.Util">
<Super>%RegisteredObject</Super>
<TimeChanged>64925,29407.276303</TimeChanged>
<TimeCreated>64924,56047.115234</TimeCreated>

<Method name="TestS3Upload">
<ClassMethod>1</ClassMethod>
<Implementation><![CDATA[
    set tAccessKeyId="[Shh... it's a secret]"
    set tSecretAccessKey="[Shh... it's a secret]"
    set tAWSRegion="us-east-1"
    set tAWSService="s3"
    set tHost="s3.us-east-1.amazonaws.com"
    set tPort="443"
    set tBucket="MyBucket12345"
    set tContentType="text/plain; charset=UTF-8"

    set tSSLConfig="AWS"
    set tUseHTTPS=1
    
    set tContentStream=##class(%Stream.TmpCharacter).%New()
    do tContentStream.Write("this is some test content oh yeah!")
    do tContentStream.Rewind()

    set tHorolog=$h
    set tDateKey=$System.Encryption.HMACSHA(256,$ZD(tHorolog,8),"AWS4"_tSecretAccessKey)
    set tDateRegionKey=$System.Encryption.HMACSHA(256,tAWSRegion,tDateKey)
    set tDateRegionServiceKey=$System.Encryption.HMACSHA(256,tAWSService,tDateRegionKey)
    set tSigningKey=$System.Encryption.HMACSHA(256,"aws4_request",tDateRegionServiceKey)
        
    set tHashedPayload=..BytesToHex($System.Encryption.SHAHashStream(256,tContentStream,.tSC))
    
    set tTmpTime=$ZDT(tHorolog,2,7)
    set tHeaderDate=$ZD(tHorolog,11)_", "_$E(tTmpTime,1,11)_" "_$E(tTmpTime,13,*-1)_" GMT"
    set tAmzDate=$TR($zdt(tHorolog,8,7),":","")

    set tKeyName="test_"_tHorolog_".txt"

    set tCanonicalURI="/"_tBucket_"/"_tKeyName
    set tCanonicalURI=$zconvert(tCanonicalURI,"O","URL")

    set tHttpVerb="PUT"
    set tCanonicalHeaders="content-length:"_tContentStream.Size_$c(10)_"content-type:"_tContentType_$c(10)_"host:"_tHost_$c(10)_"x-amz-content-sha256:"_tHashedPayload_$c(10)_"x-amz-date:"_tAmzDate_$c(10)
    set tSignedHeaders="content-length;content-type;host;x-amz-content-sha256;x-amz-date"
    set tCanonicalRequest=tHttpVerb_$c(10)_tCanonicalURI_$c(10,10)_tCanonicalHeaders_$c(10)_tSignedHeaders_$c(10)_tHashedPayload
    set tStringToSign="AWS4-HMAC-SHA256"_$c(10)_$TR($zdt(tHorolog,8,7),":","")_$c(10)_$zd(tHorolog,8)_"/"_tAWSRegion_"/"_tAWSService_"/aws4_request"_$c(10)_..BytesToHex($System.Encryption.SHAHash(256,tCanonicalRequest))
    set tSignature=..BytesToHex($System.Encryption.HMACSHA(256,tStringToSign,tSigningKey))    
    set tAuthHeader="AWS4-HMAC-SHA256 Credential="_tAccessKeyId_"/"_$zd(tHorolog,8)_"/"_tAWSRegion_"/"_tAWSService_"/aws4_request,SignedHeaders="_tSignedHeaders_",Signature="_tSignature

    set tHReq=##class(%Net.HttpRequest).%New()
    set tHReq.Server=tHost
    set tHReq.Port=tPort
    set tHReq.SSLConfiguration=tSSLConfig
    set tHReq.Https=tUseHTTPS

    do tHReq.SetHeader("Authorization",tAuthHeader)
    do tHReq.SetHeader("Content-Type","text/plain")
    do tHReq.SetHeader("Content-Length",tContentStream.Size)
    do tHReq.SetHeader("Host",tHost)
    do tHReq.SetHeader("x-amz-content-sha256",tHashedPayload)
    do tHReq.SetHeader("x-amz-date",tAmzDate)

    do tHReq.EntityBody.CopyFrom(tContentStream)

    //do tHReq.OutputHeaders()

    s tSC=tHReq.Put(tCanonicalURI)
    do $System.OBJ.DisplayError(tSC)
    
    write "Status Code:",tHReq.HttpResponse.StatusCode,!
    
    //Do tHReq.HttpResponse.OutputToDevice()
    
    if $isobject(tHReq.HttpResponse.Data) {
        s ^zresp=tHReq.HttpResponse.Data.Read(1000000)
    } else {
        s ^zresp=tHReq.HttpResponse.Data
    }
]]></Implementation>
</Method>

<Method name="BytesList">
<ClassMethod>1</ClassMethod>
<FormalSpec>pBytes:%String</FormalSpec>
<ReturnType>%String</ReturnType>
<Implementation><![CDATA[
    s l=$LISTFROMSTRING(pBytes," ")
    f i=1:1:$LL(l)  s st=$G(st)_$CHAR($ZHEX($LISTGET(l,i)))
    return st
]]></Implementation>
</Method>

<Method name="BytesToHex">
<ClassMethod>1</ClassMethod>
<FormalSpec>pBytes:%String</FormalSpec>
<ReturnType>%String</ReturnType>
<Implementation><![CDATA[
    set tHex=""
    
    for i=1:1:$L(pBytes) {
        set tHexByte=$ZHEX($ASCII($E(pBytes,i)))
        if $L(tHexByte)=1 s tHexByte="0"_tHexByte
        set tHex=tHex_tHexByte
    }
    
    return $ZCONVERT(tHex,"L")
]]></Implementation>
</Method>
</Class>
</Export>