New post
Question
· Mar 31, 2017

Troubleshooting Disconnects

#Business Service #Ensemble

One of our Departments are claiming that we are loosing HL7 messages that are sent from their Vendor's system to Ensemble.

I know interfaces 101 if we have no record of the message then we never received it, however they are insisting they are sending it.

I asked them to provide the ACK's if they sent the messages but Ensemble had no corresponding Message Control ID (MSH.10) .

Currently I have the following settings...

Job Per Connection
 
Allowed IP Addresses
OS Accept Connection Queue Size
Stay Connected
Read Timeout
SSL Configuration
Local Interface
Framing
Schedule
Pool Size
Search Table Class
Local Facility Application
Ack Mode
Use ACK Commit Codes
 
Ignore Inbound ACK
 
Add NACK ERR
 
NACK Error Code
Clear TCP Buffer Before ACK
Batch Handling
Default Char Encoding
DocType Resolution
Save Replies

Stay Connected is set to 1 because they tend to drop after each message.  We have asked them to change their interface to stay connected all the time.

I am at a loss after 13 years of working with Interfaces for a vendor to be telling me that its our fault in loosing messages but not having a record of it.

Does anyone have any other ideas on how we can figure out where these messages are going if the vendor says they are sending them to us, but I have no record of it?

Thanks

Scott Roth
Sr. Applications Development Analyst
Information Technology Integration - Interfaces
The Ohio State University Wexner Medical Center
Scott.Roth@osumc.edu

Discussion (2)2
Log in or sign up to continue
Tom Fitzgibbon · Mar 31, 2017

Scott -

We've had this problem many times with HL7 messages being "lost" after the sender claims they sent messages to our receiver software. Most of the time we found the sender actually did not send out the HL7 message or sent out a "malformed" message that our receiver rejected. Plus most HL7 sender software has filters that will not send particular messages for a variety of conditions.

Since HL7 is somewhat predictable text, it's pretty easy to parse. You can ask the sender for the missing message number and search for it in your capture stream, for example.

Receiving data from another vendor often creates arguments about "missing" data.

We troubleshot problems as below:

1) Examine any audit information given by Ensemble for errors, rejections, retransmissions and the like.

2) Set up a passive listener that records all data (not just HL7 messages) and examine with tools or your own diagnostic software.

3) Set up additional HL7 receiver software which takes the incoming messages and creates files on your platform (while doing the ACK/NACK with the sender). Then you can examine the files and later process with Ensemble.

Tom Fitzgibbon | 347-464-8531 | gototomAtG...l.com

0 0
Sean Connelly · Mar 31, 2017

Hi Scott,

Sounds like classic teapotism from the vendor.

Typically at this stage I would put Wireshark on the TCP port so that I have absolute truth as to what's going on at the TCP level.

If you see no evidence of these messages in Wireshark then you can bounce the problem back to the vendor with the Wireshark logs.

If you see evidence of messages, then you will have something more to go on.

One thing to look out for is if the HL7 messages are correctly wrapped. If you don't see evidence of ending 1c 0d hex values then the message will get stuck in the buffer. If they are dropping the connection then this can get discarded. You might see warnings relating to this, something like "discarding TCP buffer".

The fact that they think they are getting HL7 level ACK's back is a bit odd. Again with Wireshark you will be able to prove or disprove their observations. There is a scenario where by a timed out connection can collect the previous messages ACK, again it would be obvious once you look at the Wireshark logs.

If you need help with Wireshark then I have some notes digging around that might help.

Sean.

0 0