d[IA]gnosis: developing RAG applications with IRIS for Health

With the introduction of vector data types and the Vector Search functionality in IRIS, a whole world of possibilities opens up for the development of applications and an example of these applications is the one that I recently saw published in a public contest by the Ministry of Health from Valencia in which they requested a tool to assist in ICD-10 coding using AI models.

How could we implement an application similar to the one requested? Let's see what we would need:

1. List of ICD-10 codes, which we will use as context for our RAG application to search for diagnoses within the plain texts.
2. A trained model that vectorizes the texts in which we are going to look for equivalences in the ICD-10 codes.
3. The Python libraries necessary for the ingestion and vectorization of ICD-10 codes and texts.
4. A friendly front-end that supports texts on which we look for possible diagnoses.
5. Orchestration of requests received from the front-end.

What does IRIS provide us to cover the above needs?

1. CSV import, either using the RecordMapper functionality or directly using Embedded Python.
2. Embedded Python allows us to implement the Python code necessary to generate the vectors using the selected model.
3. Publish REST APIs that will be invoked from the front-end application.
4. Interoperability productions that allow tracking of information within IRIS.

Well, we only have to see the developed example:

d[IA]gnosis

Associated with this article you have access to the developed application, in the next articles we will see in detail how we implement each of the functionalities, from the use of the model, the storage of the vectors and the use of vector searches.

Let's review the application:

Importing ICD-10 codes

From the configuration screen we are told the format that the CSV file must comply with the ICD-10 codes that we are going to import. The loading and vectorization process consumes a lot of time and resources, which is why the deployment of the Docker container configures not only the RAM memory usable by Docker but also the disk memory in case the requirements exceed the allocated RAM:

  # iris
  iris:
    init: true
    container_name: iris
    build:
      context: .
      dockerfile: iris/Dockerfile
    ports:
      - 52774:52773
      - 51774:1972
    volumes:
    - ./shared:/shared
    environment:
    - ISC_DATA_DIRECTORY=/shared/durable
    command: --check-caps false --ISCAgent false
    mem_limit: 30G
    memswap_limit: 32G

The file with the ICD-10 codes is available in the project path /shared/cie10/icd10.csv, once 100% is reached the application will be ready to be used.

In our application we have defined two different functionalities for diagnostic coding, one based on HL7 messages received in the system and another based on plain texts.

Diagnostic capture from HL7

The project contains some HL7 messages prepared for testing, it is only necessary to copy the /shared/hl7/messagesa01_en.hl7 file to the /shared/HL7In folder and the associated production will be responsible for extracting the diagnosis from it to display it in the web application:

From the diagnosis requests screen we can see all the diagnoses received via HL7 messaging. To code them to ICD-10 we only need to click on the magnifying glass to show a list of those ICD-10 codes closest to the diagnosis received:

Once selected, we will see the diagnosis and its associated ICD-10 code in the list. By clicking on the button with the envelope icon, a message will be generated using the original and including the new one selected within the diagnosis segment:

MSH|^~\&|HIS|HULP|EMPI||||ADT^A08|592956|P|2.5.1
EVN|A01|
PID|||1556655212^^^SERMAS^SN~922210^^^HULP^PI||GARCÍA PÉREZ^JUAN^^^||20150403|M|||PASEO PEDRO ÁLVAREZ 195 1 CENTRO^^LEGANÉS^MADRID^28379^SPAIN||555283055^PRN^^JUAN.GARCIA@YAHOO.COM|||||||||||||||||N|
PV1||N
DG1|1||O10.91^Unspecified pre-existing hypertension complicating pregnancy^CIE10-ES|Gestational hypertension||A||

This message can be found in the path /shared/HL7Out

Screenshots of diagnoses in plaintext

From the Text Analyzer option, the user can include plain text on which an analysis process will be carried out. The application will search in tuples of 3 lemmatized words (eliminating articles, pronouns and other less relevant words). Once analyzed, the system will show us the relevant underlined text and the possible diagnoses located:

Once the analysis has been carried out, it can be consulted at any time from the analysis history.

Analysis history

All analyzes carried out are recorded and can be consulted at any time, being able to view all possible ICD-10 codes available:

In the next article...

We will see how, using Embedded Python, we use a specific LLM model for the vectorization of both the ICD-10 codes that we will use as context and the free texts.

If you have any questions or suggestions, do not hesitate to write a comment on the article.