go to post Jeff Morgan · Jan 16, 2020 Matthew, This is difficult to debug from the screenshots provided. I would suggest reaching out to InterSystems directly. Either your Sales Engineer or the WRC would be able to help you work through this road block building out your Batch Record Map.
go to post Jeff Morgan · Jan 16, 2020 Jeffrey, The scenario that I can hypothesize that would produce these results are around the client timing out before the query finishes executing on the server. Query 1 fails because you have to do a join on the two large tables and your only where clause is against a property that is not indexed. Query 2 succeeds because object IDs are "indices by nature". Query 3(from your comment) succeeds because you are filtering out hundreds, thousands, maybe millions of Ens.MessageHeader objects in your join by using a where clause against and indexed MessageBodyClassName property. I would recommend reaching out to InterSystems WRC to review the query cost and overall performance to confirm or dive deeper into the failures.
go to post Jeff Morgan · Oct 15, 2019 Kyle, This looks like something worth escalating to the InterSystems Support department. Feel free to reach out to support@intersystems.com or call 617-621-0700.
go to post Jeff Morgan · Feb 28, 2019 Scott,Specific schema structures can be added to the suite of InterSystems Products to accommodate new HL7.org structures. This can cause compatibility issues when trying to export new HL7 schema packages to older products. So the answer is, "It depends on the schema and the version being imported to."For your specific question, 2.8.HL7 is expected to import into 2015.2.2 without issues. If you experience any problems with this, please reach out to InterSystems for additional investigation.Regards,Jeff
go to post Jeff Morgan · Sep 26, 2018 David,I just confirmed this behavior on Health Connect 2017.2.2.I set up an HL7 Business Services to treat messages as individual HL7. See the Business Service setting Batch Handling of "Individual". I then passed in an HL7 with FHS and BHS headers, but no matching footers. This sample HL7 data generated separate HL7 objects within the application.I then set up an HL7 Business Operation to write this object to a file. I confirmed that "Auto Batch Parent Segs" was unchecked. The resulting HL7 data was saved without any HL7 batch header information.I hope this helps.
go to post Jeff Morgan · May 23, 2018 I cannot get the file to upload. Here is the raw XML: Include EnsembleClass Sample.CustomPurgeCriteria Extends (%SYS.Task.Definition, Ens.Util.BitmapPurge) [ System = 4 ]{/// How many days of messages should not be purgedProperty NumberOfDaysToKeep As %Integer(MINVAL = 0) [ InitialExpression = 7 ];/// Preserve messages not yet completely processedProperty KeepIntegrity As %Boolean [ InitialExpression = 1 ];/// Delete message bodies whenever their message header is deleted. This is off by default because some Productions may/// use message objects that are part of a larger environment and not transitory.Property BodiesToo As %Boolean [ InitialExpression = 1 ];/// The type of thing to purgeProperty TypesToPurge As %String(DISPLAYLIST = ",All Types,Events,Messages,Business Processes,Rule Logs,I/O Logs,Host Monitor Data,Managed Alerts", VALUELIST = ",all,events,messages,processes,rulelogs,iologs,hostcounters,managedalerts") [ InitialExpression = "events", Required ];/// The name of the component to purge/// This only applies to TypesToPurge = 'Messages'Property SourceConfigName As %String;/// The OnTask() method is called to execute the taskMethod OnTask() As %Status{Set tTime = $ZH Set tDeletedCount = -1 Set tBitmapCount = 0 If ..TypesToPurge = "all" {Set tSC = ##class(Ens.Purge).PurgeAll(.tDeletedCount,..NumberOfDaysToKeep,..KeepIntegrity,..BodiesToo)Set (type,tDeletedCount)="" For { Set type=$O(tDeletedCount(type),1,count) Quit:""=type Continue:'countSet tDeletedCount=tDeletedCount_", "_count_" "_type} Set tDeletedCount=$E(tDeletedCount,3,$L(tDeletedCount)) // get rid of the leading comma and spaceSet tBitmapCount = -1 } ElseIf ..TypesToPurge = "events" {Set tSC = ##class(Ens.Purge).PurgeEventLogsByDate(..NumberOfDaysToKeep,.tDeletedCount,..KeepIntegrity) } ElseIf ..TypesToPurge = "messages" { // Call local Purge() ClassMethod instead of standard codeSet tSC = ..Purge(..NumberOfDaysToKeep,.tDeletedCount,..KeepIntegrity,..BodiesToo,,..SourceConfigName)Set tDeletedCount=tDeletedCount_$S($G(tDeletedCount("bodies")):"("_tDeletedCount("bodies")_" bodies)",1:"")// ============================ } ElseIf ..TypesToPurge = "processes" {Set tSC = ##class(Ens.Purge).PurgeBusinessProcessesByDate(..NumberOfDaysToKeep,.tDeletedCount,..KeepIntegrity) } ElseIf ..TypesToPurge = "rulelogs" {Set tSC = ##class(Ens.Purge).PurgeRuleLogsByDate(..NumberOfDaysToKeep,.tDeletedCount,..KeepIntegrity) } ElseIf ..TypesToPurge = "iologs" {Set tSC = ##class(Ens.Util.IOLog).Purge(.tDeletedCount,..NumberOfDaysToKeep,..KeepIntegrity) } ElseIf ..TypesToPurge = "hostcounters" {Set tSC = ##class(Ens.MonitorService).Purge(.tDeletedCount,..NumberOfDaysToKeep,..KeepIntegrity) } ElseIf ..TypesToPurge = "managedalerts" { Set tSC = ##class(Ens.Alerting.ManagedAlert).Purge(.tDeletedCount,..NumberOfDaysToKeep,..KeepIntegrity) } ElseIf $ZStrip(..TypesToPurge,"*WC") = "" {Set tSC = $$$ERROR($$$EnsErrGeneral,"No TypesToPurge specified") } Else {Set tSC = $$$ERROR($$$EnsErrGeneral,"Unrecognized TypesToPurge value '"_..TypesToPurge_"'") } Set tTime = $ZH - tTime If $$$ISOK(tSC) {If (tBitmapCount = 0) {Set tBitmapCount = +$G(tDeletedCount("bitmaps"))}$$$LOGINFO("Purged "_tDeletedCount_" "_..TypesToPurgeLogicalToDisplay(..TypesToPurge)_$S(tBitmapCount>0: " and "_tBitmapCount_" bitmap chunks", 1: "")_" keeping the last "_..NumberOfDaysToKeep_" days with KeepIntegrity="_..KeepIntegrity_$S($Case(..TypesToPurge,"all":1,"messages":1,:0):" and BodiesToo="_..BodiesToo,1:"")_" in "_tTime_"s")}Else { $$$LOGERROR("Error purging"_..TypesToPurgeLogicalToDisplay(..TypesToPurge)_" keeping the last "_..NumberOfDaysToKeep_" days with KeepIntegrity="_..KeepIntegrity_$S($Case(..TypesToPurge,"all":1,"messages":1,:0):" and BodiesToo="_..BodiesToo,1:"")_" : "_ $$$StatusDisplayString(tSC)) }Quit tSC}/// Purge the message store, and event logClassMethod PurgeAll(pTypesToPurge As %String = "all", pNumberOfDaysToKeep As %Integer = 0, pBodiesToo As %Boolean = 1) As %Status{Set tTask = ..%New()Set tTask.TypesToPurge = pTypesToPurgeSet tTask.NumberOfDaysToKeep = pNumberOfDaysToKeepSet tTask.BodiesToo = pBodiesTooQuit tTask.OnTask()}Method Purge(Output pDeletedCount As %Integer, pDaysToKeep As %Integer = 7, pKeepIntegrity As %Boolean = 1, pBodiesToo As %Boolean = 1, pBitmapChunkLimit As %Integer = 500, pSourceConfigName As %String) As %Status{Kill ^CacheTemp.EnsPurgeMessage($Job) // This global will hold errors and warnings for the message purgeSet ^CacheTemp.EnsPurgeMessage($Job) = $ZDT($H,3)New %tDoNotDeleteDate,%tID,%tBodyId,%tBodyClassname Set %tID="", %tDoNotDeleteDate = $$$timeUTCHtoUTC($s($ztimezone'<0:($H-pDaysToKeep+1)_","_($ztimezone*60),1:($H-pDaysToKeep)_","_($ztimezone*60+86400)))If '$data($$$EnsJobLocal) New $$$EnsJobLocal Set $$$EnsJobLocal = ""Set tSC=$$$OK, SQLCODE=0, pDeletedCount=0, pDeletedCount("bodies")=0If pBodiesToo {If pKeepIntegrity {&sql(DECLARE C1 CURSOR FORSelect TOP 100000000 ID,MessageBodyId,MessageBodyClassName Into :%tID,:%tBodyId,:%tBodyClassname From Ens.MessageHeader hWhere (TimeCreated < :%tDoNotDeleteDate)--Added filter by componentAnd(SourceConfigName = :pSourceConfigName) And 0 = ( Select Count(*) From Ens.MessageHeaderWhere (SessionId = h.SessionId)And (Status<>$$$eMessageStatusCompleted)And (Status<>$$$eMessageStatusAborted)And (Status<>$$$eMessageStatusError)And (Status<>$$$eMessageStatusDiscarded) )Order By TimeCreated)&sql(OPEN C1)For { &sql(FETCH C1) Quit:SQLCODEIf %tBodyId'="" {#; Delete body if body class exists and is persistent and ENSPURGE is NOT explicitly set to 0 (i.e. ENSPURGE=1 by default)Set:""=%tBodyClassname&&(%tBodyId=+%tBodyId) %tBodyClassname="Ens.MessageBody"If ""'=%tBodyClassname {Set tExists=$G(aClass(%tBodyClassname))If 0'=tExists {If ""=tExists&&'($$$comClassDefined(%tBodyClassname)&&($classmethod(%tBodyClassname,"%IsA","%Persistent")||$classmethod(%tBodyClassname,"%IsA","%Stream.Object")))||($parameter(%tBodyClassname,"ENSPURGE")=0) {Set aClass(%tBodyClassname)=0} Else {try {Set tSC1=$classmethod(%tBodyClassname,"%DeleteId",%tBodyId)Set:""=tExists aClass(%tBodyClassname)=1, tExists=1 ; , aClass(%tBodyClassname,"extent")=##class(Ens.VDoc.SearchTableGenerator).GetExtentSuperclass(%tBodyClassname)} catch {Set tSC1 = $$$SystemError//Set:""=tExists aClass(%tBodyClassname)=0 Set:'$G(aClass(%tBodyClassname,"doneErr")) tSC1=$$$SystemError // This was preventing subsequent messages to be deleted (HCR374)}If $$$ISOK(tSC1) || $$$StatusEquals(tSC1,$$$DeleteObjectNotFound,$$$FileCanNotDelete,$$$NotAnInstanceError) {Set tSC2 = ##class(Ens.SearchTableBase).RemoveSearchTableEntries(%tBodyClassname,%tBodyId,1)If $$$ISERR(tSC2)&&'$G(aClass(%tBodyClassname,"doneErrST")) && '$$$StatusEquals(tSC2,$$$DeleteObjectNotFound,$$$FileCanNotDelete,$$$NotAnInstanceError) { ; || ($$$StatusEquals(tSC2,$$$NotAnInstanceError) && '$classmethod(aClass(%tBodyClassname,"extent"),"%ExistsId",%tBodyId))//Set aClass(%tBodyClassname,"doneErrST")=1 // This was preventing subsequent message's search table entries to be deleted (HCR374)Set tMsg = "Failed to purge SearchTable entries for deleted body with BodyClassname='"_%tBodyClassname_"', BodyId='"_%tBodyId_"' from header "_%tID_" :"_$$$StatusDisplayString(tSC2)Set tSC = ..PurgeSetTemp(3,tMsg,tSC,tSC2)}} Else { //ElseIf '$G(aClass(%tBodyClassname,"doneErr")) { // This was causing inadequate error reporting (HCR374)//Set aClass(%tBodyClassname,"doneErr")=1 // This was preventing subsequent message bodies to be deleted (HCR374)Set tMsg = "Failed to purge body for header "_%tID_", BodyClassname='"_%tBodyClassname_"':"_$$$StatusDisplayString(tSC1)Set tSC = ..PurgeSetTemp(2,tMsg,tSC,tSC1)}Set pDeletedCount("bodies")=pDeletedCount("bodies")+$$$ISOK(tSC1)}}}}&sql(DELETE From Ens.MessageHeader Where ID = :%tID)Set pDeletedCount=pDeletedCount+%ROWCOUNTIf SQLCODE {Set tMsg = "Failed to purge message header "_%tID_": SQLCODE="_SQLCODE Set:$G(%msg)'="" tMsg = tMsg_", %msg="_%msgSet tSC = ..PurgeSetTemp(1,tMsg,tSC,$$$ERROR($$$EnsErrGeneral,tMsg))}} Set tCode=SQLCODE &sql(CLOSE C1) Set:'SQLCODE SQLCODE=tCode} Else {&sql(DECLARE C2 CURSOR FORSelect ID,MessageBodyId,MessageBodyClassName Into :%tID,:%tBodyId,:%tBodyClassname From Ens.MessageHeaderWhere (TimeCreated < :%tDoNotDeleteDate)--Added filter by componentAnd(SourceConfigName = :pSourceConfigName))&sql(OPEN C2)For { &sql(FETCH C2) Quit:SQLCODEIf %tBodyId'="" {#; Delete body if body class exists and is persistent and ENSPURGE is NOT explicitly set to 0 (i.e. ENSPURGE=1 by default)Set:""=%tBodyClassname&&(%tBodyId=+%tBodyId) %tBodyClassname="Ens.MessageBody"If ""'=%tBodyClassname {Set tExists=$G(aClass(%tBodyClassname))If 0'=tExists {If ""=tExists&&'($$$comClassDefined(%tBodyClassname)&&($classmethod(%tBodyClassname,"%IsA","%Persistent")||$classmethod(%tBodyClassname,"%IsA","%Stream.Object")))||($parameter(%tBodyClassname,"ENSPURGE")=0) {Set aClass(%tBodyClassname)=0} Else {try {Set tSC1=$classmethod(%tBodyClassname,"%DeleteId",%tBodyId)Set:""=tExists aClass(%tBodyClassname)=1, tExists=1 ;, aClass(%tBodyClassname,"extent")=##class(Ens.VDoc.SearchTableGenerator).GetExtentSuperclass(%tBodyClassname)} catch {Set tSC1 = $$$SystemError//Set:""=tExists aClass(%tBodyClassname)=0 Set:'$G(aClass(%tBodyClassname,"doneErr")) tSC1=$$$SystemError // This was preventing subsequent messages to be deleted (HCR374)}If $$$ISOK(tSC1) || $$$StatusEquals(tSC1,$$$DeleteObjectNotFound,$$$FileCanNotDelete,$$$NotAnInstanceError) {Set tSC2 = ##class(Ens.SearchTableBase).RemoveSearchTableEntries(%tBodyClassname,%tBodyId,1)If $$$ISERR(tSC2)&&'$G(aClass(%tBodyClassname,"doneErrST")) && '$$$StatusEquals(tSC2,$$$DeleteObjectNotFound,$$$FileCanNotDelete,$$$NotAnInstanceError) { ; || ($$$StatusEquals(tSC2,$$$NotAnInstanceError) && '$classmethod(aClass(%tBodyClassname,"extent"),"%ExistsId",%tBodyId))//Set aClass(%tBodyClassname,"doneErrST")=1 // This was preventing subsequent message's search table entries to be deleted (HCR374)Set tMsg = "Failed to purge SearchTable entries for deleted body with BodyClassname='"_%tBodyClassname_"', BodyId='"_%tBodyId_"' from header "_%tID_" :"_$$$StatusDisplayString(tSC2)Set tSC = ..PurgeSetTemp(3,tMsg,tSC,tSC2)}} Else { //ElseIf '$G(aClass(%tBodyClassname,"doneErr")) { // This was causing inadequate error reporting (HCR374)//Set aClass(%tBodyClassname,"doneErr")=1 // This was preventing subsequent message bodies to be deleted (HCR374)Set tMsg = "Failed to purge body for header "_%tID_", BodyClassname='"_%tBodyClassname_"':"_$$$StatusDisplayString(tSC1)Set tSC = ..PurgeSetTemp(2,tMsg,tSC,tSC1)}Set pDeletedCount("bodies")=pDeletedCount("bodies")+$$$ISOK(tSC1)}}}}&sql(DELETE From Ens.MessageHeader Where ID = :%tID)Set pDeletedCount=pDeletedCount+%ROWCOUNTIf SQLCODE {Set tMsg = "Failed to purge message header "_%tID_": SQLCODE="_SQLCODE Set:$G(%msg)'="" tMsg = tMsg_", %msg="_%msgSet tSC = ..PurgeSetTemp(1,tMsg,tSC,$$$ERROR($$$EnsErrGeneral,tMsg))}} Set tCode=SQLCODE &sql(CLOSE C2) Set:'SQLCODE SQLCODE=tCode}} Else {If pKeepIntegrity {&sql(DECLARE C3 CURSOR FORSelect TOP 100000000 ID Into :%tID From Ens.MessageHeader hWhere TimeCreated < :%tDoNotDeleteDate--Added filter by componentAnd(SourceConfigName = :pSourceConfigName)And 0 = ( Select Count(*) From Ens.MessageHeaderWhere (SessionId = h.SessionId)And (Status<>$$$eMessageStatusCompleted)And (Status<>$$$eMessageStatusAborted)And (Status<>$$$eMessageStatusError)And (Status<>$$$eMessageStatusDiscarded) )Order By TimeCreated)&sql(OPEN C3) For { &sql(FETCH C3) Quit:SQLCODE&sql(Delete From Ens.MessageHeader Where ID=:%tID)Set pDeletedCount=pDeletedCount+%ROWCOUNTIf SQLCODE {Set tMsg = "Failed to purge message header "_%tID_": SQLCODE="_SQLCODE Set:$G(%msg)'="" tMsg = tMsg_", %msg="_%msgSet tSC = ..PurgeSetTemp(1,tMsg,tSC,$$$ERROR($$$EnsErrGeneral,tMsg))}} Set tCode=SQLCODE &sql(CLOSE C3) Set:'SQLCODE SQLCODE=tCode} Else {&sql(DECLARE C4 CURSOR FORSelect ID Into :%tID From Ens.MessageHeader Where TimeCreated < :%tDoNotDeleteDate)&sql(OPEN C4) For { &sql(FETCH C4) Quit:SQLCODESet %ROWCOUNT=0&sql(Delete From Ens.MessageHeader Where ID=:%tID)Set pDeletedCount=pDeletedCount+%ROWCOUNTIf SQLCODE {Set tMsg = "Failed to purge message header "_%tID_": SQLCODE="_SQLCODE Set:$G(%msg)'="" tMsg = tMsg_", %msg="_%msgSet tSC = ..PurgeSetTemp(1,tMsg,tSC,$$$ERROR($$$EnsErrGeneral,tMsg))}} Set tCode=SQLCODE &sql(CLOSE C4) Set:'SQLCODE SQLCODE=tCode}}Set:SQLCODE&&(SQLCODE'=100) tSC=$$$ADDSC(tSC,$$$ERROR($$$EnsErrGeneral,"Purge error at ID "_$G(%tID)_"; SQLCODE = "_SQLCODE))Set tBitmapSC = ..PurgeBitmaps(pBitmapChunkLimit,.tDeletedChunks)Merge pDeletedCount("bitmaps") = tDeletedChunksQuit $$$ADDSC(tSC,tBitmapSC)}/// Log a warning in the Event Log; add to tSC status; set error/warning in a temp global as below: <br>/// Total error count is in subscript 0. <br>/// Errors while deleting message headers are in subscript 1. <br>/// Errors while deleting message bodies are in subscript 2. <br>/// Errors while deleting search table entries are in subscript 3.Method PurgeSetTemp(pType As %Integer, pMsg As %String, pSC As %Status, pSC2 As %Status) As %Status{$$$LOGWARNING(pMsg)Set tCount0 = $I(^CacheTemp.EnsPurgeMessage($Job,0))Set tCount = $I(^CacheTemp.EnsPurgeMessage($Job,pType))Set:$G(%tID)'="" ^CacheTemp.EnsPurgeMessage($Job,pType,%tID) = pMsgIf tCount0<11 {Set pSC = $$$ADDSC(pSC,pSC2)}ElseIf tCount0=11 {Set pSC = $$$ADDSC(pSC,$$$ERROR($$$GeneralError,"There are more errors and/or warnings, see the Ensemble Event Log and ^CacheTemp.EnsPurgeMessage("_$Job_") for the full list"))}Else {// do not put into pSC more than 10 messages}Quit pSC}}
go to post Jeff Morgan · Mar 7, 2018 Ewan,A simple example of how to count DocType's in the EnsLib_HL7.Message table is below:SELECT Count(*) FROM EnsLib_HL7.MessageWHERE Name = 'ADT_A12'If you need to compare Ens.MessageHeader properties too, you can do a join on the two tables:select Count(*) from EnsLib_HL7.Message As BodyLEFT JOIN Ens.MessageHeader As HeaderON Body.ID = Header.MessageBodyIdWHERE Header.ID > 1 AND Body.Name = 'ADT_A12'Hope this helps.
go to post Jeff Morgan · Feb 14, 2018 This behavior matches a known limitation with using Extended Criteria on Ensemble/HealthShare 2017.1.1.Please contact InterSystems Support in order to continue investigating the behavior.
go to post Jeff Morgan · Oct 4, 2017 Clark is more or less correct. In order for a Business Operation FTP Passthrough component to send a file downstream, the file has to have been saved somewhere local to Ensemble/HealthShare. This can be one of two default locations. 1. The Caché database using GlobalCharacterStreams(initial default)2. The local file system using FileCharacterStreams This is controlled by the setting in the Business Service FTP Passthrough, "Use FileStream". There is no default way of preventing the Business Service from saving the file. If space is your primary concern, then I would suggest an aggressive Message Purge strategy (using BodiesToo and KeepIntegrity set to true) as this can be a standard solution for keeping your persistent data low even if you have large data flow in your Passthrough interface.
go to post Jeff Morgan · Jul 18, 2017 Alexi,It appears though you have already resolved your issue by reinstalling the instance.I encourage you to reach out to our Worldwide Response Center if you run into issues starting InterSystems applications in the future. We can assist in identifying the root cause of what is preventing a normal startup.You can find information about the WRC here:http://www.intersystems.com/services-support/worldwide-response-center/
go to post Jeff Morgan · Oct 5, 2016 Paul,I have successfully used the Parenthesis () Syntax with the Contains functions in a routing rule condition. Your first example should be working as you expect it to.I would like to work with you on this issue in a WRC case as I believe that it will take some investigation in order to identify what is causing the rule not to fire.I will email you shortly with a WRC number and a couple questions.Jeff MorganEnsemble Support
go to post Jeff Morgan · Jun 8, 2016 If a database is missing from an installation, aka if you remove your old SAMPLES cache.dat, and THEN do the reinstall, that will populate a new, fresh SAMPLES database back to it's original state.