Imagine the Future of Medical Triage: MediCoPilot

 

Current triage systems often rely on the experience of admitting physicians. This can lead to delays in care for some patients, especially when faced with inexperienced residents or non-critical symptoms. Additionally, it can result in unnecessary hospital admissions, straining resources and increasing healthcare costs.

We focused our project on pregnant women and conducted a survey with friends of ours who work at a large hospital in São Paulo, Brazil, specifically in the area of monitoring and caring for pregnant women.

We discovered that a significant problem occurs when an inexperienced doctor, still a resident, admits a patient with symptoms that are not considered risky, occupying a bed that could be allocated to a patient with more severe symptoms or at risk.

We believe that AI-powered triage systems can objectively analyze patient data, ensuring faster and more efficient care for all. This empowers residents and novice physicians by providing guidance and reducing the burden of initial assessment. Additionally, by optimizing bed allocation, we can promote better resource utilization and a reduction in financial waste.

Interoperability

Inspired by Evgeny Shvarov's post Making your own Chat with Open AI ChatGPT in Telegram, (highly recommended reading!), we embarked on this project leveraging a similar approach using InterSystems interoperability.

 

The first component we created was a BusinessService utilizing the Telegram Adapter by Nikolay Solovyev. This BusinessService acts as a central hub for communication with the Telegram bot. When it receives a message from a patient, it routes it through a dedicated Rule (Message.Route).

The Message.Route that identifies if the conversation with the bot is starting, using the /start command, and returns a welcome message.

In the future, at this stage, we plan to check if the patient is already registered and connect it with FHIR(Fast Healthcare Interoperability Resources), a standardized healthcare data format.

This feature is planned for a future version and not for this contest.

The Message.Route Rule also differentiates between text and voice messages. For text messages, the Rule seamlessly routes the information to the MediCopilot.Process component. This component, which we'll explore in detail later, is responsible for analyzing the patient's text-based symptoms.

Speech to Text

Thinking of a way to speed up the service, or in cases where the patient might not be able to type out their symptoms, the best approach would be to send a voice message. If a voice message is received, it should be converted to text before being sent to MediCopilot.Process.

For this, we used the IRIS Open-AI adapter created by Kurro Lopez, also mentioned in @Evgeny Shvarov 's post, however, we adopted a specific method within the adapter – Audio Transcription.

This method accepts an audio file (in formats like mp3, ogg, or wav) encoded in base64 format and returns the transcribed text.

While the Audio Transcription method offered an efficient solution, we encountered a slight challenge.

The Telegram API returns only the voice file ID. The solution was to extend the Telegram.OutboundAdapter class to fetch the voice file.

Determined to bridge this gap, we explored alternative approaches. The Telegram Adapter's private API property made direct modifications challenging. However, after the contest, I plan to submit a Pull Request to the Telegram Adapter to formally introduce voice file retrieval functionality.

In the meantime, my idea was to use embedded Python to overcome this obstacle. By reading the Telegram API documentation, I found that with the voice file ID, I needed to find the path of this file by querying the getFile endpoint. After this, I had to query another endpoint to fetch the file.

To manage this process efficiently, we created a dedicated Business Operation named "VoiceFile.BusinessOperation." This operation includes two methods written in ePython:

• GetVoiceFilePath: This method queries the "getFile" endpoint using the voice file ID, retrieving the path where the audio file resides.
• GetVoiceEncodedData: This method utilizes the retrieved path to download the audio file from the Telegram server. It then encodes the downloaded file and encoded in base64 format, which is the format expected by the OpenAI Audio Transcription method.

With the file in base64, it can be sent to the OpenAI.Operation to get the transcription. I created a Business Process called TelegramVoice.Process to make these calls synchronously and, after receiving the transcription, send the text to MediCopilot.Process.

MediCopilot Adapter

With the transcription of the audio file or the text message, it's time to send all this to the core of our project, the engine that analyzes patient-reported symptoms.

 @José Pereira wrote an incredible article explaining how this mechanism works in detail, so if you haven't read it yet, which I strongly recommend since it will be worth your time and give you a better perspective of our project

I won't delve into this part of the project, but we created an adapter to execute this Python class that utilizes LangChain and LangChain-Iris by Dmitry Maslennikov. This powerful combination allows us to perform Vector Search within InterSystems IRIS, a high-performance database platform.

Based on the research we conducted with our friends working in the medical field, we identified several common symptoms in pregnant women. We categorized these into non-risk symptoms, risk symptoms, and potential diseases or conditions that may require hospitalization.

To train our system in identifying potential risks, we employed OpenAI technology. By simulating patient conversations, we were able to build a synthetic database that effectively mirrored real-world scenarios. However, our ultimate goal is to transition to a database built upon real medical data, including anonymized anamneses (patient histories) and diagnoses.

Thus, we send the patient's report received via Telegram and return a possible diagnosis. The idea is to add another rule to identify the severity of the risk and either refer the patient or alert a doctor.

Conclusion

Our project aims to reduce hospitalization time and financial waste in hospitals while improving the speed of patient care. By focusing on pregnant women and leveraging interoperability, we developed a system that can perform preliminary triage and assist patients efficiently. Using components like the Telegram Adapter, IRIS Open-AI adapter, and MediCopilot Adapter, we created a seamless workflow from receiving a patient's message to delivering a potential diagnosis.

We would love for you to vote for our tool, but even more importantly, we would like to hear from you about how we can improve it. What features could be added? Your comments are welcome, and we are happy to respond.