New post
AllAccepted answers
Stuart Byrne · Dec 18, 2019

Thanks Jeffrey.

I've spoken to our Caché ODBA and he concurs there are orphaned records in the Ens.Message* globals.

We're on 2017.1 currently, but have been live since Caché 2012, which may explain todays issues.

Looking at the purge tasks we have records with no timestamp, which would escape the purging process.

Our Caché ODBA is going to open a WRC to confirm the best course of action for a one off purge of these Ens.Message* globals.

Thank you so much for your advice as it has helped immensely identifying this issue.

Stuart Byrne · Dec 17, 2019

Please find my method below which is used to drive a Business Operation.  It's activated by a schedule on a daily basis at 6am:

Method OnInit(Output pResponse As Ens.Response) As %Status
{
//set the DST variable
set DST = $SYSTEM.Util.IsDST()
// set pre-DST edited timestamps for the email header
set emailSummaryStartDAT = $ZDATE($H-1,3)_" "_(..SummaryStartTime)
set emailSummaryFinishDAT = $ZDATE($H,3)_" "_(..SummaryFinishTime)
// ..SummaryStartTime DST handling IF DST =1 adjust SummaryStartTime & SummaryFinishTime Property's H-1
IF DST = 1 {
set ..SummaryStartTime = ($P(..SummaryStartTime,":",1)-1)_":"_$P(..SummaryStartTime,":",2,3)
set ..SummaryFinishTime = ($P(..SummaryFinishTime,":",1)-1)_":"_$P(..SummaryFinishTime,":",2,3)
$$$LOGINFO("Daylight Savings is being applied to timestamps")
}
// IF the hour part of timestamp is <10 then add a leading 0 to make a full UTC timestamp
IF ($P(..SummaryStartTime,":",1)<10){
set ..SummaryStartTime = "0"_..SummaryStartTime
}
IF ($P(..SummaryFinishTime,":",1)<10){
set ..SummaryFinishTime = "0"_..SummaryFinishTime
}
//Set Search Start Date/Time of yesterday with SummaryStartTime
set SummaryStartDAT = $ZDATE($H-1,3)_" "_(..SummaryStartTime)
//Set Search Finish Date/Time for today with SummaryFinishtime
set SummaryFinishDAT = $ZDATE($H,3)_" "_(..SummaryFinishTime) //Obtain the ID of the first and last message in the search timeframe
&sql(SELECT MAX(ID),MIN(ID),COUNT(ID)INTO :LastID,:FirstID,:ErrorCount FROM Ens.AlertRequest WHERE AlertTime BETWEEN :SummaryStartDAT AND :SummaryFinishDAT) /// Log Ens.AlertRequest ID range and the no. of errors
$$$LOGINFO("AlertRequest ID's identified between: "_FirstID_" & "_LastID_" No. of Errors is: "_ErrorCount)
  //build the email subject line
  Set Subject = "Automatic alert from AIE: "_$NAMESPACE_" Email Alert Summary between:"_emailSummaryStartDAT_" & "_emailSummaryFinishDAT
  //build the email Body
  Set emailBody = "This is a summary of alerts generated by the CUH Application Integration Engine (AIE)."
  set emailBody = emailBody_"<br><br>"
  set emailBody = emailBody_"This email covers alerts between: "_emailSummaryStartDAT_" & "_emailSummaryFinishDAT
  set emailBody = emailBody_"<br><br>"
  Set emailBody = emailBody_"Namespace: "_$NAMESPACE
  Set emailBody = emailBody_"<br><br>"
  set emailBody = emailBody_"No of errors is: "_ErrorCount
  Set emailBody = emailBody_"<br><br>"
  // Check if any errors have been identified
  //If errors identified create table
  IF ErrorCount '= 0 {
  // set table header
  set errortbl = "<head><style>table, th, td { border: 0px solid black;}</style></head><table style='width:100%'><tr><th>Alert ID</th><th>AlertText</th><th>AlertTime</th><th>SourceConfigName</th> </tr>"
  //Populate the error table using sql cursor query to loop through the SQL resultset.
  &sql(DECLARE C1 CURSOR FOR SELECT ID, AlertText, AlertTime, SourceConfigName INTO :ID, :AlertText, :AlertTime, :SourceConfigName  FROM Ens.AlertRequest WHERE AlertTime BETWEEN :SummaryStartDAT AND :SummaryFinishDAT ORDER BY SourceConfigName, AlertTime ASC)
  &sql(OPEN C1)
  IF SQLCODE '= 0 {
  $$$LOGERROR("SQL Error Code: "_SQLCODE)
  }
QUIT:(SQLCODE'=0)
&sql(FETCH C1)
//Loop through Alert fields from the time period specified.
While SQLCODE = 0 {
IF DST = 1 {
set AlertHour = $P(AlertTime," ",2)
set $P(AlertHour,":",1) = ($P(AlertHour,":",1)+1)
// Check if updated time is before 10:00am if so a "0" prefix needs adding
IF ($P(AlertHour,":",1)<10){
set $P(AlertHour,":",1) = "0"_$P(AlertHour,":",1)
}
set AlertTime = ($P(AlertTime," ",1)_" "_AlertHour)
  }
Set errortbl = errortbl_"<tr> <td>"_ID_"</td> <td>"_AlertText_"</td> <td>"_AlertTime_"</td>"
set errortbl = errortbl_"<td>"_SourceConfigName_"</td></tr>"
&sql(FETCH C1)
}
&sql(CLOSE C1)
//End of HTML Table
set errortbl = errortbl_"</table>"
Set errortbl = errortbl_"<br>"
set emailBody = emailBody_errortbl}
//handling for no errors. No table is created and comment below written
ELSE {set emailBody = emailBody_"No errors have been identified between: "_emailSummaryStartDAT_" & "_emailSummaryFinishDAT_"<br><br>"}
  // Set Email Footer
  set emailBody = emailBody_"Do NOT reply to this email."
  Set emailBody = emailBody_"<br>"
Set emailBody = emailBody_"<br>Application Integration Engine (AIE) | eHospital | "
Set emailBody = emailBody_"<br>Cambridge University Hospitals NHS Foundation Trust | Addenbrookes Hospital | Box 117 |"
Set emailBody = emailBody_"<br>"
Set emailBody = emailBody_"This email is confidential, see www.cuh.org.uk/email_disclaimer.html"
  // Build new mail structure.
$$$LOGINFO("Attempting to send Alert Summary email")
IF (..From = "") {
SET From = "noreply@addenbrookes.nhs.uk"
}
IF (..To = "") {
$$$LOGWARNING("Warning - No email provided => No email sent")
}
ELSE {
SET mail = ##class(%Net.MailMessage).%New()
// Define sender.
    SET mail.From = ..From
    // Define recipients.
    // Converts String into list of string.
    DO mail.To.InsertList($lfs(..To,";"))
// Define subject.
    SET mail.Subject = Subject
//Char set to "utf-8" to allow for html manipulation
SET mail.Charset = "utf-8"
SET mail.ContentType = "text/html"
SET mail.IsHTML = 1
// Define body.
DO mail.TextData.WriteLine(emailBody)
// Check if high priority is needed.
set Priority = 1
IF Priority = 1 {
DO mail.Headers.SetAt(1,"X-Priority")
  DO mail.Headers.SetAt("High","X-MSMail-Priority")
  DO mail.Headers.SetAt("High","Importance")
}
// Build the smtp server to send the email.
SET server=##class(%Net.SMTP).%New()
SET server.smtpserver=..SMTPServer
SET server.port=..SMTPPort
SET status=server.Send(mail)
   If $$$ISERR(status) {
QUIT $$$ISERR(status)
}
$$$LOGINFO("Alert Summary Email has been sent")
}
QUIT $$$OK
} }
Stuart Byrne · Apr 15, 2019

The object of part 2 is to compare the strings from the same input as part 1 and identify which 2 strings differ by 1 character in the same position.

Use the links above to find the exercise. 

My code goes:

ClassMethod Part2SQL()
{
//initialising varibles
 set rowcount = ""
 set line = ""
  // Get the highest row number in the table WDA
 &sql(SELECT TOP 1 ID INTO :rowcount FROM AOC2018.Day2 ORDER BY ID DESC)
 Write !, "Row count is: "_rowcount
 // Loop through each row in the table
 FOR i=1:1:rowcount{
 // initialise looping variables
 set mismatchcount = "0"
 set complinelength = ""
 set compline = ""
 set compchar = ""
 // run SQL query to look at last number for row loop.
 &sql(SELECT WholeLine INTO :line FROM AOC2018.Day2 WHERE ID = :i)
 set linelength = $LENGTH(line)
 set complinelength = linelength
 //set currrentrow = ##class(AOC2018.Day2).%OpenId(i)
 FOR comprow = i+1:1:rowcount{
 set mismatchchar1 = ""
 set mismatchchar2 = ""
 set mismatchpos = ""
 //IF comprow = rowcount+1 {QUIT}
 &sql(SELECT WholeLine INTO :compline FROM AOC2018.Day2 WHERE ID = :comprow)
 IF compline = "" {QUIT}
 set mismatchcount = "0"
 For char=1:1:linelength{
 /// get current character being looked for
 set currentchar = $EXTRACT(line,char,char)
 set compchar = $EXTRACT(compline,char,char)
 IF compchar '= currentchar{
 set mismatchcount = $INCREMENT(mismatchcount)
 set mismatchchar1 = currentchar
 set mismatchchar2 = compchar
 set mismatchpos = char
 }
 QUIT:(mismatchcount >1)
 }
 QUIT:(mismatchcount =1)
 }
 QUIT:(mismatchcount =1)
 }
 write !, "The lines with only one mismatch are:"
 write !, "1st line: "_line_" On row: "_i
 write !, "2nd Line: "_compline_" On row: "_comprow
 write !, "The char in the mismatch in line: "_i_". The char is: "_mismatchchar1
 write !, "The char in the mismatch in line: "_comprow_". The char is: "_mismatchchar2
 write !, "With character position: "_mismatchpos
 set answerfirsthalf = $EXTRACT(line,1,mismatchpos-1)
 set answersecondhalf = $EXTRACT(line,mismatchpos+1,linelength)
 write !, "The matching text is: "_answerfirsthalf_answersecondhalf
}
Stuart Byrne · Mar 2, 2019

AOC 2018 Day1 Part 2

As promised in the post above, I have posted part 2 in the comments.

The summary of part 2 is to find the first frequency that repeats twice.

In this published method rather than looping through a string, I looped through the first set of sequence changes (990) via a sql query.

This version was much quicker than parsing the string of frequency changes through each loop.  I added a kill global to the first method, so I could use this to generate the first loop of values before testing/debugging this script.

In hindsight I didn't need to do this and could do something inside this method.

Let me know what you think:

 ClassMethod FreqPart2(Input As %String)
{
//This needs to be run after ImportFreqPI method as expects Global to be populated
// set previous record counter to 0 
    set prevrecord = 0 
    //Find the last record to append to in AOC2018.Day1 global
    &sql(SELECT TOP 1 ID INTO :prevrecord FROM AOC2018.Day1 ORDER BY ID DESC)
    //Set Prev frequency to 0 then get value via SQL query
    set prevfreq = 0
    &sql(SELECT CurrentFrequency INTO :prevfreq FROM AOC2018.Day1 WHERE ID = :prevrecord)
    write prevfreq
    set end = 990
   
PILOOP
    // Loop through list
    FOR i=1:1:end{
    //create new object to save to Global
    set newrecord = ##class(AOC2018.Day1).%New()
    // Set Freqency counter to 0, so when used later is only value from current loop
    set FreqCount = 0
    //set freqcount variable to newrecord.Freqcount so can be saved correctly .
    set FreqCount = newrecord.FreqCount
    //set current value in list
    &sql(SELECT FrequencyChange INTO :insertitem FROM AOC2018.Day1 WHERE ID = :i )
    //write list value
    //write insertitem
    //set Frequency change value = to insertitem variable
    set newrecord.FrequencyChange = insertitem
    //set current frequency by adding prevfreq to newrecord.FrequencyChange
    set newrecord.CurrentFrequency = prevfreq + newrecord.FrequencyChange
    //set prev freq variable to newrecord.CurrentFrequency
    // so can be used to calculate newrecord.CurrentFrequency in next loop
    set prevfreq = newrecord.CurrentFrequency
    //save record
    set status = newrecord.%Save()
    // Check if there is any entry in the global already for the frequency and input into FreqCount variable
    &sql(SELECT COUNT (CurrentFrequency) INTO :FreqCount FROM AOC2018.Day1 WHERE CurrentFrequency = :newrecord.CurrentFrequency)
    //write FreqCount
    set newrecord.FreqCount = FreqCount 
    set status = newrecord.%Save()
    //write !,"Frequency Change Is:"_insertitem_" Current Frequency is:"_newrecord.FrequencyChange
    //write !,"Frequency counter IS: "_FreqCount
    //write status
    IF FreqCount > 1 {
        write "First repeat = "_newrecord.CurrentFrequency
        QUIT}
        set prevrecord = prevrecord+1
        }
    IF FreqCount < 2 {
    write !, "Relooping" 
    GOTO PILOOP
            }

    return status
    }
Stuart Byrne · Oct 23, 2017

You can also test Data transformations from within the Data Transformation Editor, if you have the message as text.

There is no direct way (without writing something) to get 1 message through.

An indirect way would be use the throttle delay option for the Business Service, Business Process(es)/Router(s) & Business Operations in Ensemble via the Settings tab in the Additional Settings section.

It works in the Milliseconds (1000 milliseconds = 1 second) , so to get one message through with enough time to stop the feed you would set it to 60,000 milliseconds.

We used this to assurance test each of our feeds on the day when upgrading to Healthshare 2017.1

I hope this helps you.

PS you would need to remember to remove this throttle after testing, otherwise in a production environment you would have queues galore.

Stuart Byrne · Oct 16, 2017

Thank you again Eduard.

I added the set and found a new error:

ERROR <Ens>ErrException: <INVALID OREF>zCopyFrom+2 ^%Library.FileCharacterStream.1 -- logged as '-'
number - @''


Logged: 2017-10-16 12:46:52.131

Source: HL7-2-Email

Session: 1680693

Job: 8932

Class: CUH.Oper.HL7Email

Method: MessageHeaderHandler

Trace: (none)

Stack: 
•$$^zMessageHeaderHandler+198 ^CUH.Oper.HL7Email.1 +2
•$$^zOnTask+42^Ens.Host.1 +1
•DO^zStart+62^Ens.Job.1 +2
 
I can't see what not's being referenced at this point.  I've had a look at the class and believe to be referencing everything correctly.

Stuart Byrne · Oct 15, 2017

Hi Eduard,

Thank you for your suggestion. I am having trouble getting the OutputToIOStream method to work.

The code I have used in addition to the above is;

 set AttachementStream = pRequest.OutputToIOStream(pIOStream,..Separators,"",1)
 Set tSC = mail.AttachStream(AttachementStream,..Filename,1,"iso-8859-1") If $$$ISERR(tSC) Quit tSC

When I try and pass a HL7 message to the operation I get the following error:

ERROR <Ens>ErrException: <UNDEFINED>zOnMessage+5 ^CUH.Oper.HL7Email.1 *pIOStream -- logged as '-'
number - @'
set AttachementStream = pRequest.OutputToIOStream(pIOStream,..Separators,"",1)'

I am new to handling streams and the documentation isn't helping it make things clearer.

Stuart Byrne · Sep 4, 2017

I am having trouble implementing this in the DTL.  Were currently on  2012.2.5.

Would this version have an impact on this class from functioning?

I created the class as is, but renamed to:  CUH.Other.StripDelimiterfunction

The DTL calling on the class is: 

<assign value='##class(CUH.Other.StripDelimiterfunction).ConversionScrub(target.{})' property='target.{}' action='set' />

When I run a test message I get the following error:

ERROR <Ens>ErrException: <INVALID OREF>zConversionScrub+1^CUH.Other.StripDelimiterfunction.1 -- logged as '-' number - @' set tSegCount = pHL7.SegCount'

')+'

'; content += ''; return content; }, "modalShow": function() { // add ensExceptionModalGroup to class for floating div var modalGroup = EnsException.modalGroup; if (modalGroup) { var div = modalGroup.getFloatingDiv(); if (div) div.className += ' ensExceptionModalGroup'; } // override default behaviour -- user must make a choice var mouseTrap = document.getElementById('zenMouseTrap'); if (mouseTrap) mouseTrap.onmouseup = null; }, "modalDelete": function() { // clean up floating div var modalGroup = EnsException.modalGroup; if (modalGroup) { var div = modalGroup.getFloatingDiv(); if (div && div.parentNode) { div.parentNode.removeChild(div); } } } } window.zenUserExceptionHandler = EnsException.exceptionHandler;
 
Stuart Byrne · Jun 29, 2017

After some investigation I found that the method would work with the following:

I found a quirk in the testing service, where the rule would not work using the testing service.  

I created a business Operation that feeds our ADT Business service and the same messages now all work.

I'm currently testing with our eMR (Epic's) DEV platform ADT messages and this is also working.

It looks like the testing service might works a bit funny when running from the router.

Thank you all for your reads and contributions.