Incredible CSV TO FHIR TO SQL TO JUPYTER - FHIR contest V2
Hello everyone, this is with great pleasure that I announce the V2 of my application 'Contest-FHIR'.
In this new version, I used new tools and techniques I discovered at the EUROPEAN HEALTHCARE HACKATHON in which I was invited by InterSystems as a guest and as a mentor to display the multiple projects I did in my intership back in April 2022.
Today I present to you the V2 of my application, it can now transform CSV to FHIR to SQL to JUPYTER notebook.
This is for me a really huge step in InterSystems technologies and I think this tool will be used for many other purpuse and cases.
Visit my GitHub for more information, and for those of you that don't want to lose time, check out the part 5. Walkthrough
pf my GitHub for quick information and screens !!
Here is the Walkthrough for you ;)
5. Walkthrough
Complete walkthrough of the Python IRIS production.
5.1. CSV TO FHIR
Here you must use the ProductionCSV and change the parameters if needed to transform any CSV file added to the data/in
folder to your FHIR server.
This link will help you load the ProductionCSV, change the parameters if needed and start it :http://localhost:33783/csp/healthshare/fhirserver/EnsPortal.ProductionConfig.zen?$NAMESPACE=FHIRSERVER&$NAMESPACE=FHIRSERVER&
Connect using :
username : superuser
password : SYS
Go to Action
in the far right menu, then click Open
then choose Python
and ProducitonCSV
!
Now START the production.
Just by loading it and starting it, the production should automatically load a really simple csv file containing information on some organizations to our FHIR server.
Note :
If you want to do it with others fhir resources you need to create a new Data Transfomation and create a new message type.
5.1.1. FHIR ?
We now have a FHIR server ( That was pre-filled with random generated Organization and Patient ) that also contains our 2 new organizations !!
All that from a simple click in the Production.
5.2. FHIR TO SQL
Here we will use a powerful InterSystems tool that allows us to transform any FHIR server to an SQL server and perform some more transformation on the data if needed !
For that follow :http://localhost:33783/csp/fhirsql/index.csp/
Connect using :
username : superuser
password : SYS
We will now create an analysis of our FHIR repo, then a projection, and finally we will convert it to SQL.
5.2.1. FHIR ANALYSIS
Here, enter :
Name : Local_FHIR
Host : localhost
Port : 52773
SSL conf : don't touch
Credentials : Press new and add a new credentials like
name : superuser
username : superuser
password : SYS
FHIR Repository URL : /fhir/r4
And press OK.
Now enter 100 in Selectivity Percentage
and press Launch Analysis Task
.
Note
It's possible to use any FHIR server here, and the configuration given in this GitHub is just for our InterSystems local FHIR server ( Note that you could use also a cloud InterSystems FHIR server )
5.2.2. FHIR PROJECTION
We have prepared a simple and easy projection from FHIR to SQL.
Click import
and select Contest-FHIR/misc/ExportFHIRtoSQL.json
.
Select :
Name : T1
Analysis : Local_FHIR
Press Import
.
5.2.3. PROJECTION TO SQL
Then press Launch Projection
5.2.4. SQL ?
We now have an SQL server ( That was pre-filled with random generated Organization and Patient ) that also contains our 2 new organizations !!
All that from simples steps using the FHIR SQL BUILDER from InterSystems.
You can access the SQL server following this link :http://localhost:33783/csp/sys/exp/%25CSP.UI.Portal.SQL.Home.zen?$NAMESPACE=FHIRSERVER
See also :
You can clearly see the generated information presented here but also our two Organization added in the beginning.
5.3. SQL TO JUPYTER
Now that we have done CSV to FHIR to SQL, we need to gather the information from this SQL server to our Jupyter NoteBook.
For that follow this link :http://localhost:8888/notebooks/SqlAlchemy.ipynb
From here, you can, using the incredible sqlalchemy
tool, plug into our SQL IRIS DataBase and 'play' with our data while having everything protected and stored in FHIR, the future of the Health industry storage.
You can easily imagine plugging behind this a Machine Learning model, or a deep analysis of our data using all the wonderful Python tools.
6. Conclusion
I hope you are all convinced by now of the power of this tool.
If you want to sell InterSystems to your clients this the demo you need !
Fast data integration to an IRIS FHIR server, fast and realiable conversion from FHIR to SQL then all the data inside a Python NoteBook in a Pandas DataFrame, ready to be worked on by our best data analyst or to be used a fuel to your best Machine learning models !!!
Special thanks to Guillaume Rongier, Evgeny Shvarov, Dmitry Zasypkin and Dmitry Maslennikov as I'm using tools from them like the FHIR SQL BUILDER, the SQLALCHEMY module in Python or the incredible interoperability-embedded-python from Guillaume Rongier. Without this last tool nothing of what I've did would work since I know almost nothing in ObjectScript and this allowed me to continue participating in the Dev Community.
Thanks again everyone.