Using Embedded Python to generate data streamflow

Article

Jimmy Xu · Sep 26, 2022 3m read

#Embedded Python #InterSystems IRIS #InterSystems IRIS for Health

Hi Developers,

Python has a large and powerful ecosystem that contains thousands of libraries and packages available, especially in data science.

Therefore, I wanted to have a first try in using a recent feature of IRIS called Embedded Python, to simply import a python library called datetime, generate data with a timestamp component and persist it in InterSystems IRIS for Health Data Platform. The same will work on IRIS Data Platform as well.

I have broken down this small project into 2 main pieces:

Set up the Embedded Python environment following the IRIS for Health Data Platform documentation
Create the timestamp data as an example using Embedded Python and persist to IRIS, using the datetime package

Setup Embedded Python on InterSystems IRIS for Health Data Platform 2022.1

On Windows, open CMD terminal:

C:\InterSystems\IRISHealth\bin>irispip install --target C:\InterSystems\IRISHealth\mgr\python numpy

Note: This is the step for checking the Embedded Python environment, because I find out when I use IRIS 2021.0 eairly the irispip is not working

Test Python functionality by importing a Python math package

Open IRIS terminal and execute the following in the USER namespace: set pymath = ##class(%SYS.Python).Import("math")

Then execute write pymath.pi, You can see you have successfully called the python package, with the output as follows:

USER>write pymath.pi

3.141592653589793116

And we can also start the Python shell by opening from the IRIS terminal:

do ##class(%SYS.Python).Shell()

Class User.PythonFirstTry Extends %RegisteredObject
{

ClassMethod pyHello() As %Status
{
set pythonBuiltins = ##class(%SYS.Python).Builtins()
do pythonBuiltins.print("Hello World!")
}

ClassMethod pyForLoop() [ Language = python ]
{
for i in range(5):
print("Python")
}

}

I have written two class methods that you can try to put in a .cls file, compile them and see the output.

Execute the methods as below:

USER>do ##class(User.PythonFirstTry).pyHello()

Hello World!

USER>do ##class(User. PythonFirstTry).pyForLoop()

Python

Use python library to generate dataflow and persist to InterSystems IRIS

Python Library needed for data flow generation:

Import datetime package (this do not require extra installation using CLI)

In addition to the datetime package, I tried using other libraries and packages however some of them are not present natively with Embedded Python. To install these, open CMD terminal (for Windows) then run the following (XX is to be replaced with the package name):

C:\InterSystems\IRISHealth\bin>irispip install XX

Here are the steps followed to generate data and persist into IRIS.

Create %Persistent class
Create Property
Set a classmethod (e.g. GetData) and specify [Language = python]
Generate data by calling datetime python library, using the following code snippet:

Note: timeNow needs to be converted into a String datatype

Execute the method GetData() via IRIS Terminal, execute SQL query

Note: You could use a simple for loop to control the volume of data that is generated by this method.