Prompt Engineering for ollama-ai-iris

My use case is I want AI to extract six pieces of key information from medical visit notes.

It works okay with notes from Epic patient portals. I am struggling with one of my own visit Summaries. So far I have not been able to instruct ollama to give me a JSON response where it gave me what I am asking for.

My original medical_progress_notes_prompt was:

Medical progress note:
---
(document)
---

You are an expert in analyzing medical progress notes. Please carefully read the provided progress note above and extract the following key information:

Extract these six (6) properties from the progress note:
- Title
- Date: The date of the medical encounter
- Provider: The full name of the provider
- Patient: The full name of the patient
- Chief Complaint: The reason for the medical encounter
- Assessment: The plan of the provider

Guidelines:
- The extracted information should be factual and accurate to the document.
- Be concise
- If any property is missing from the note, please leave the field empty rather than guessing.

Answer in JSON format. The JSON should contain 6 keys: "Title", "Date", "Provider", "Patient", "Chief Complaint", and "Assessment".

One of the documents I downloaded from a Patient Portal contained this text: (Patient name was masked)

Name: Merle K Bxxxxxx | DOB: 3/21/1935 | MRN: 50313076-7 | PCP: IVAN LEVIN, MD | Legal Name: Merle K Bxxxxxx
Appointment Details
Pt calls requesting PA for Rifaximin be submitted.
 
Forwarded to PA Team
Electronically signed by Eridana H, RN at 9/9/2024  4:01 PMTelephone Encounter
Eridana H, RN at 9/9/2024  4:00 PM
MyChart® licensed from Epic Systems Corporation © 1999 - 2025 DH9079POC

The first time I processed it using BplSendChat the response from ollama was:

['Here', ' is', ' the', ' extracted', ' information', ' in', ' JSON', ' format', ':\n\n', '``', '`\n', '{\n', ' ', ' "', 'Title', '":', ' "",\n', ' ', ' "', 'Date', '":', ' "', '9', '/', '9', '/', '202', '4', '",\n', ' ', ' "', 'Provider', '":', ' "', 'E', 'rid', 'ana', ' H', ',', ' RN', '",\n', ' ', ' "', 'Patient', '":', ' "', 'Mer', 'le', ' J', ' Bush', 'kin', '",\n', ' ', ' "', 'Chief', ' Complaint', '":', ' "",\n', ' ', ' "', 'Ass', 'essment', '":', ' ""\n', '}\n', '``', '`\n\n', 'Note', ' that', ' some', ' fields', ' were', ' unable', ' to', ' be', ' extracted', ' from', ' the', ' provided', ' progress', ' note', ',', ' and', ' are', ' therefore', ' left', ' blank', '.', '']

This was before I added classMethod DecodeTokenArray(). I processed the same text 30 minutes later and the response from ollama was:

Here is the extracted information in JSON format:\n\n```\n{\n  "Title": "",\n  "Date": "",\n  "Provider": "Ivan Levin, MD",\n  "Patient": "Merle K Bxxxxxx",\n  "Chief Complaint": "",\n  "Assessment": ""\n}\n```\n\nThe following fields were missing from the progress note and were left empty: Title, Date.

I am pleased that the response became much easier to read for human eyes. I noticed that the second response left the date empty. I am not sure why.

I updated my question prompt slightly to this: 

Medical notes:
---
(document)
---

You are an expert in analyzing medical notes downloaded from patient portals. Please carefully read the provided note and extract the following key information:

Extract these six (6) properties from the medical note:
- Title
- Date: The date the medical encounter took place
- Provider: The full name of the provider
- Patient: The full name of the patient
- Chief Complaint: The reason for the medical encounter
- Assessment: The plan of the provider

Guidelines:
- The extracted information should be factual and accurate to the document.
- Be concise
- If any property is missing from the note, please leave the field empty rather than guessing.

Answer in JSON format. The JSON should contain 6 keys: "Title", "Date", "Provider", "Patient", "Chief Complaint", and "Assessment".

The response from ollama with the modified prompt was:

```\n{\n  "Title": "",\n  "Date": "9/9/2024",\n  "Provider": "Eridana H, RN",\n  "Patient": "Merle K Bxxxxxx",\n  "Chief Complaint": "",\n  "Assessment": ""\n}\n```

The original prompt did not yield the expected response from my visitSummary.txt

This is a comprehensive patient chart in electronic format, likely created using the eClinicalWorks system. The chart contains various sections that provide detailed information about the patient's medical history, current health status, diagnoses, medications, test results, and more.\n\nHere are some key points from the chart:\n\n**Demographics**\n\n* Patient name: Oliver Wilms\n* Date of birth: February 28, 1964\n* Accurate identifier (Acc No): 164557\n\n**Medical History**\n\nThe patient has a history of mixed hyperlipidemia (E78.2) and essential hypertension (I10). The patient is also taking medication for these conditions.\n\n**Current Health Status**\n\n* The patient's vital signs include:\n\t+ Temperature: 97.8°F\n\t+ Blood pressure: systolic 142 mmHg, diastolic 84 mmHg\n\t+ Heart rate: 71/min\n\t+ Respiratory rate: 16/min\n* The patient's weight is 189.8 lbs.\n* The patient's BMI is 25.38 kg/m2.\n\n**Medications**\n\nThe patient is taking:\n* Losartan potassium (25 mg) orally once a day for 90 days\n\n**Lab Results and Test Results**\n\nNo laboratory results or test results are listed in the chart.\n\n**Encounters and Procedures**\n\nThe patient has had an encounter with Valora Medical - Frisco on October 18, 2023, for a general adult medical examination without abnormal findings. No procedures were performed during this visit.\n\n**Assessments and Treatment Plans**\n\n* The patient's assessment is categorized as "Encounter for general adult medical examination without abnormal findings (ICD-10 - Z00.00)".\n* The treatment plan includes medication, with Losartan potassium prescribed for 90 days.\n\n**Insurance and Contact Information**\n\nThe patient is insured by United Healthcare, with a subscriber number and group number listed.\n\nOverall, this chart provides a comprehensive overview of the patient's medical history, current health status, diagnoses, medications, and other relevant information

When I ran it with the modified prompt, the ollama response was:

Based on the provided information, here is a summary of Mr. Wilms' medical record:\n\n**Demographics**\n\n* Date of Birth: February 28, 1964\n* Accurate Medical Record Number: 164557\n* Insurance Provider: United Healthcare\n* Address: 2444 Falcon Point Dr, Frisco, TX 75035-0000\n\n**Medical History**\n\n* Mixed hyperlipidemia (E78.2) - Active confirmed\n* Essential hypertension (59621000) - Essential (primary) hypertension (I10) - Active confirmed\n* No other significant medical history noted.\n\n**Vital Signs**\n\n* Temperature: 97.8°F (normal)\n* Blood pressure systolic: 142 mm Hg (elevated)\n* Blood pressure diastolic: 84 mm Hg (elevated)\n* Heart Rate: 71 /min (normal)\n* Respiratory Rate: 16 /min (normal)\n* Height: 72.5 in (within normal range)\n* Weight: 189.8 lbs (obese)\n* BMI: 25.38 kg/m2 (overweight)\n\n**Medications**\n\n* Losartan Potassium 25 MG - Oral, once a day for 90 days\n* Started on August 19, 2022\n\n**Lab Results and Tests**\n\n* No other lab results or test results are provided in this summary.\n\n**Imaging and Procedures**\n\n* None mentioned.\n\n**Physical Examination**\n\n* General appearance: Alert, no distress\n* Head: Normal\n* Eyes: Extraocular movement intact (EOMI), pupils equal, round, reactive to light and accommodation\n* Ears: Normal, tympanic membranes intact, clear, Bilateral\n* Throat: Clear\n* Neck/Thyroid: No lymphadenopathy, no masses, no thyromegaly\n* Heart: RRR, No M/G/R\n* Lungs: CTA (clear)\n* Abdomen: S/NT/ND BS + (no abnormalities noted)\n* Neurologic: Cranial nerves 2-12 grossly intact\n* Skin: Normal\n* Extremities: No clubbing, cyanosis, or edema\n\n**History of Present Illness (HPI)**\n\n* Mr. Wilms is here for a general adult medical examination without abnormal findings.\n* He denies any problems or side effects from his medication and states that he is taking it consistently.\n* He expresses no concerns about his health.\n\n**Care Team and Next Steps**\n\n* Care team includes Bradley Friedman, MD\n* Follow-up appointment scheduled in 6 months\n\nThis summary provides a general overview of Mr. Wilms' medical record, highlighting his current diagnoses, medications, lab results, physical examination findings, and next steps in his care.

I still did not get the answer in JSON format. I waited 315 seconds for the response. I modified the prompt again:

Medical notes:
---
(document)
---

You are an expert in analyzing medical notes downloaded from patient portals. Please carefully read the provided note and extract the following key information:

Extract these six (6) properties from the medical note:
- Title
- Date: The date the medical encounter took place
- Provider: The full name of the provider
- Patient: The full name of the patient
- Chief Complaint: The reason for the medical encounter
- Assessment: The plan of the provider

Guidelines:
- The extracted information should be factual and accurate to the document.
- Please do not summarize the note, but extract only what is asked for.
- If any property is missing from the note, please leave the field empty rather than guessing.

Answer in JSON format. The JSON should contain 6 keys: "Title", "Date", "Provider", "Patient", "Chief Complaint", and "Assessment".

This time I waited 301 seconds and then I got this response:

This is a comprehensive medical record for WILMS, Oliver, a 61-year-old male patient with the following information:\n\n**Demographics**\n\n* Date of birth: 02/28/1964\n* Address: 2444 Falcon Point Dr, Frisco, TX 75035-0000\n* Contact information:\n\t+ Phone: 972-261-6717 (guarantor)\n\t+ Phone: 469-744-0422 (emergency contact)\n* Insurance: United Healthcare\n\n**Medical History**\n\n* Mixed hyperlipidemia (E78.2) - Active confirmed\n* Essential hypertension (59621000) - Essential primary hypertension (I10) - Active confirmed\n* No other significant medical conditions or diagnoses mentioned.\n\n**Vital Signs**\n\n* Temperature: 97.8 degrees Fahrenheit\n* Blood pressure systolic: 142 mm Hg\n* Blood pressure diastolic: 84 mm Hg\n* Heart rate: 71 /min\n* Respiratory rate: 16 /min\n\n**Physical Examination**\n\n* General appearance: Alert, in no distress\n* Head: Normal\n* Eyes: Extraocular movement intact (EOMI), pupils equal, round, reactive to light and accommodation\n* Ears: Normal, tympanic membranes intact, clear, Bilateral\n* Throat: Clear\n* Neck/Thyroid: Neck supple, no lymphadenopathy, no masses, no thyromegaly\n* Heart: RRR (normal heart sounds)\n* Lungs: CTA (clear to auscultation bilaterally)\n* Abdomen: S/NT/ND (normal examination of abdomen)\n* Neurologic: Cranial nerves 2-12 grossly intact\n* Skin: Normal\n* Extremities: No clubbing, cyanosis, or edema\n* Oral cavity: Mucosa moist\n\n**History and Physical Notes**\n\n* The patient is here for a general adult medical examination without abnormal findings.\n* The patient denies any problems or side effects from their medication and has been taking it consistently.\n* The patient feels that the medication is working appropriately.\n\n**Care Team**\n\n* Emergency contact: Rose Wilms\n* Guarantor: Oliver Wilms\n\n**Next Appointment**\n\n* Scheduled for 6 months from the current date, with no specific reason mentioned.\n\nThis medical record provides a comprehensive overview of WILMS' current medical status, including his diagnoses, vital signs, physical examination findings, and medication regimen. It also includes information about his insurance coverage and contact details.

My prompting is not getting the desired result. Let me try again:

Medical notes:
---
(document)
---

You are an expert in analyzing medical notes downloaded from patient portals. 
Please carefully read the provided note, answer six questions, and place the answers inside a JSON object:
- (Title:) What is the title or type of note?
- (Date:) What is the date the medical encounter took place?
- (Provider:) What is the full name of the provider?
- (Patient:) What is the full name of the patient
- (Chief Complaint:) Why did the patient have this encounter?
- (Assessment:) What is the treatment plan of the provider?

Guidelines:
- Please do not summarize the note. You should answer the six questions and put the answer into the corresponding JSON property.
- If any property is missing from the note, please leave the field empty rather than guessing.
- Answer in JSON format. The JSON should contain 6 keys: "Title", "Date", "Provider", "Patient", "Chief Complaint", and "Assessment".

Based on the medical history and physical examination, here are some key findings:\n\n**Medical History:**\n\n* The patient has a history of mixed hyperlipidemia and essential hypertension.\n* He is taking Losartan Potassium 25mg orally once a day for 90 days.\n\n**Physical Examination:**\n\n* General appearance: alert, in no distress\n* Head: normal EOMI (extraocular movement intact), pupils equal, round, reactive to light and accommodation\n* Eyes: extraocular movement intact\n* Ears: normal, tympanic membranes intact, clear, Bilateral\n* Throat: clear\n* Neck/Thyroid: neck supple, no lymphadenopathy, no masses, no thyromegaly\n* Heart: RRR (normal heart sounds), No M/G/R (no murmurs or gallop rhythm)\n* Lungs: CTA (clear chest X-ray)\n* Abdomen: S/NT/ND BS+ (smooth, normal, and non-tender)\n* Neurologic: cranial nerves 2-12 grossly intact\n* Skin: normal\n* Extremities: no clubbing, cyanosis, or edema\n* Oral cavity: mucosa moist\n\n**History of Present Illness (HPI):**\n\n* The patient is here for a general adult medical examination and medication follow-up.\n* He denies any problems or side effects with his medication.\n* He feels that the medication is working appropriately.\n\n**Assessment:**\n\n* The patient is found to be generally healthy, with no abnormal findings on physical examination.\n* His blood pressure is 142/84 mmHg, which is within normal limits for an adult.\n* His lipid profile and thyroid function tests are not provided in this document.\n\n**Plan of Treatment:**\n\n* Continue Losartan Potassium 25mg orally once a day as prescribed.\n* Schedule follow-up appointments every 6 months to monitor blood pressure and lipid profiles.\n\n**Next Appointments:**\n\n* Follow-up appointment: 6 months from today\n* Reason for follow-up: Goals

Medical notes:
---
(document)
---

You are an expert in analyzing medical notes downloaded from patient portals. Please follow these directions. Please do not summarize the note. 
You should answer the six questions below, and put the question and answer into a JSON array.
If the answer cannot be found in the note, please leave it empty rather than guessing.  
The JSON array should look like this:
[ "insights": {"question": "What is the title or type of note?", "answer": "?"},
  {"question": "What is the date the medical encounter took place?", "answer": "?"},
  {"question": "What is the full name of the provider?", "answer": "?"},
  {"question": "What is the full name of the patient?", "answer": "?"},
  {"question": "Why did the patient have this encounter?", "answer": "?"},
  {"question": "What is the treatment plan of the provider?", "answer": "?"}]