Initializing InterSystems IRIS Docker Image with Python

While starting the development with IRIS we have a distribution kit or in case of Docker we are pulling the docker image and then often we need to initialize it and setup the development environment. We might need to create databases, namespaces, turn on/off some services, create resources. We often need to import code and data into IRIS instance and run some custom code to init the solution.

And there plenty of templates on Open Exchange where we suggest how to init REST, Interoperability, Analytics, Fullstack and many other templates with ObjectScript. What if we want to use only Python to setup the development environment for Embedded Python project with IRIS?

So, the recent release of Embedded Python template is the pure python boilerplate that could be a starting point for developers that build python projects with no need to use and learn ObjectScript. This article expresses how this template could be used to initialize IRIS. Here we go!

Ok!

Creating Database and Namespace

Often we need to create database and namespace. This could be done with merge cpf functionality of IRIS, and here is the merge file that does the thing:

[Actions]
CreateResource:Name=%DB_IRISAPP_DATA,Description="IRISAPP_DATA database"
CreateDatabase:Name=IRISAPP_DATA,Directory=/usr/irissys/mgr/IRISAPP_DATA,Resource=%DB_IRISAPP_DATA
CreateResource:Name=%DB_IRISAPP_CODE,Description="IRISAPP_CODE database"
CreateDatabase:Name=IRISAPP_CODE,Directory=/usr/irissys/mgr/IRISAPP_CODE,Resource=%DB_IRISAPP_CODE
CreateNamespace:Name=IRISAPP,Globals=IRISAPP_DATA,Routines=IRISAPP_CODE,Interop=1
ModifyService:Name=%Service_CallIn,Enabled=1,AutheEnabled=48
ModifyUser:Name=SuperUser,PasswordHash=a31d24aecc0bfe560a7e45bd913ad27c667dc25a75cbfd358c451bb595b6bd52bd25c82cafaa23ca1dd30b3b4947d12d3bb0ffb2a717df29912b743a281f97c1,0a4c463a2fa1e7542b61aa48800091ab688eb0a14bebf536638f411f5454c9343b9aa6402b4694f0a89b624407a5f43f0a38fc35216bb18aab7dc41ef9f056b1,10000,SHA512

Merge cpf contains Actions which in this case create dabasases IRISAPP_DATA and IRISAPP_CODE for data and code respectively, the related IRISAPP namespace to access it and the related resources %IRISAPP_DATA and %IRISAPP_CODE" to manage the access.

This Merge cpf also enables Callin service to make Embedded python work via ModifyService action:

ModifyService:Name=%Service_CallIn,Enabled=1,AutheEnabled=48

The last action is an example how you can securely set user's password, using ModifyUser action:

ModifyUser:Name=SuperUser,PasswordHash=a31d24aecc0bfe560a7e45bd913ad27c667dc25a75cbfd358c451bb595b6bd52bd25c82cafaa23ca1dd30b3b4947d12d3bb0ffb2a717df29912b743a281f97c1,0a4c463a2fa1e7542b61aa48800091ab688eb0a14bebf536638f411f5454c9343b9aa6402b4694f0a89b624407a5f43f0a38fc35216bb18aab7dc41ef9f056b1,10000,SHA512

The password hash here can be generated via another docker image from InterSystems, e.g. with the following line called in a terminal (learn more about passwordhash):

docker run --rm -it containers.intersystems.com/intersystems/passwordhash:1.1 -algorithm SHA512 -workfactor 10000

merge command is could be called in Dockerfile like this:

iris merge IRIS merge.cpf && \

Other types of merge actions and different settings available for IRIS could be found in documentation.

Running an Arbitrary Init Code

Another popular use case is to run an arbitrary custom code to load data, init variables and load code from the repository e.g. in a form of IPM package. Usually in IRIS templates there is a special file iris.script that contains such a code in form of ObjectScript. I kept the iris.script file in the repository just to demo how it have been used with ObjectScript. 

In this python template you can find the example of such init code in a iris_script.py file. Let's see what's there:

import glob

import os

import iris

import pandas as pd

from sqlalchemy import create_engine

from iris import ipm

# switch namespace to the %SYS namespace
iris.system.Process.SetNamespace("%SYS")

# set credentials to not expire
iris.cls('Security.Users').UnExpireUserPasswords("*")

# switch namespace to IRISAPP built by merge.cpf
iris.system.Process.SetNamespace("IRISAPP")

# load ipm package listed in module.xml
#iris.cls('%ZPM.PackageManager').Shell("load /home/irisowner/dev -v")
assert ipm('load /home/irisowner/dev -v')

# load demo data
engine = create_engine('iris+emb:///')

# list all csv files in the demo data folder
for files in glob.glob('/home/irisowner/dev/data/*.csv'):

# get the file name without the extension
table_name = os.path.splitext(os.path.basename(files))[0]

# load the csv file into a pandas dataframe
df = pd.read_csv(files)

# write the dataframe to IRIS
df.to_sql(table_name, engine, if_exists='replace', index=False, schema='dc_demo')

The code is rather self-explanatory and it has the demo of changing namespaces, loading repo code in form of IPM module, importing csv files that contained in /data folder of the repo using sqlalchemy lib.

What Does the Template do?

Basically this template provides examples how can  Embedded Python by used with IRIS. 

1. Calling Python From ObjectScript

To demo it we have several ObjectScript classes in the repo.

To test the approach open IRIS terminal and run:

IRISAPP>d ##class(dc.python.test).Hello()

World

this test demoes how you can use functions in sample.py code in IRIS using Embedded Python.

2. Refer to IRIS from python

To test this you can run irisapp.py that performs different operations with IRIS database. To run all the test start terminal session in docker:

# attach to the running IRIS container

docker-compose exec iris bash

# run the script

$ irispython ./python/irisapp.py

3. Building Python Flask REST API that works with IRIS

There is also an example of REST api in python made with Flask that you could use in your applications that you can find in /python/flask/app.py. 

To start Flask REST api do:

$ docker-compose exec iris bash

# irispython /python/flask/app.py

Then you can test the server at http://localhost:55030/.

And here are two examples on how to add a new record in IRIS and read a record:

POST http://localhost:55030/persistentclass

Content-Type: application/json

Accept: application/json

{

"test": "toto"

}

Result:

json

{

"id": 1,

"test": "toto"

}


GET http://localhost:55030/persistentclass/1

Accept: application/json

Result:

json

{

"id": 1,

"test": "toto"

}

4. Deployment

The deployment of the solution to IRIS systems can be performed via IPM. For that purpose we have an IPM module.xml that will deliver ObjectScript and Python files to a target IRIS system from an IPM Registry you will publish module at.

Thanks a lot to @Guillaume Rongier and @Dmitry Maslennikov who helped a lot to create the template and make "Pure Python" approach possible.

There are more "radical" approaches in organizing such templates - here is the "no dockerfile" and "no objedctscript" sample by @Dmitry Maslennikov ,

here is "embedded python" template by @Guillaume Rongier.