· Dec 17, 2019 3m read

Python Gateway IV: Interoperability Adapter

This series of articles would cover Python Gateway for InterSystems Data Platforms. Execute Python code and more from InterSystems IRIS. This project brings you the power of Python right into your InterSystems IRIS environment:

  • Execute arbitrary Python code
  • Seamlessly transfer data from InterSystems IRIS into Python
  • Build intelligent Interoperability business processes with Python Interoperability Adapter
  • Save, examine, modify and restore Python context from InterSystems IRIS

Other articles

The plan for the series so far (subject to change).


You now have tried Python Gateway from a terminal, time to start using it via Interoperability Productons.
In this article, I would cover the main Interoperability interface to Python - It allows us to:

  • Execute Python code and return requested variables (string/stream)
  • Save/Restore context
  • Load data into Python

Generally speaking, it's an Interoperability wrapper over Interoperability adapter offers the ability to interact with the Python process from Interoperability productions. Currently, five requests are supported:

  • Execute Python code via Returns with requested variable values
  • Execute Python code via Returns with requested variable values. Same as above, but accepts and returns streams instead of strings.
  • Set dataset from SQL Query with Returns Ens.Response.
  • Set dataset faster from Global/Class/Table with Returns Ens.Response.
  • Save the Python context via Returns Ens.StringResponse with context id.
  • Restore Python context via has but two settings:

    • Initializer - select a class implementing It can be used to load functions, modules, classes and so on. It would be executed at process start.
    • PythonLib - (Linux only) if you see loading errors set it to or even to a full path to the shared library.

Writing Business Process

There are two utility classes to ease BP development:
- allows annotation fetching with variable substitution
- allows easy testing of Python Interoperability business processes. It can execute a business process (python parts) in a current job.

Variable substitution

All business processes inheriting from can use GetAnnotation(name) method to get value of activity annotation by activity name. Activity annotation can contain variables which would be calculated on ObjectScript side before being passed to Python. This is the syntax for variable substitution:

  • ${class:method:arg1:...:argN} - execute method
  • #{expr} - execute ObjectScript code

Check test business process for example in Correlation Matrix: Graph activity: f.savefig(r'#{process.WorkDirectory}SHOWCASE${%PopulateUtils:Integer:1:100}.png')

In this example:
- #{process.WorkDirectory} returns WorkDirectory property of process object which is an instance of class and current business process.
- ${%PopulateUtils:Integer:1:100} calls Integer method of %PopulateUtils class passing arguments 1 and 100, returning random integer in range 1...100.

Test Business Process

Test Interoperability Production and test Business Process are available by default as a part of the Python Gateway. To use them:

  1. In OS bash execute pip install pandas matplotlib seaborn.
  2. Execute: do ##class( to populate test data.
  3. Start production.
  4. Send empty Ens.Request message to the

Let's see how it all works together. Open in BPL editor (or Studio):

Code execution call

Here's the most important call - to execute Python code:

Request is and the properties are:
- Code - Python code to execute.
- SeparateLines - Separate incoming message into lines for execution. $c(10) (\n) is used for line separation. Note that it's NOT recommended to process the whole message at once, this feature is only for def and similar multi-line expressions processing. Defaults to 0.
- Variables - Comma-separated list of variables to get in response message.
- Serialization - How to serialize variables we want to return with Str, Repr, JSON, Pickle and Dill options, defaulting to Str.

In our case, we're only setting Code property so all other properties are defaults. We set it by calling process.GetAnnotation("Import pandas") which at runtime returns annotation after performing variable substitution. Eventually, import pandas as pd string would be passed to Python. GetAnnotation can be useful to set up multiline python scripts, but there's no restriction on it. You can set Code property any way you like.

Variable retrieval call

Another interesting call using is Correlation Matrix: Tabular:

It calculates Correlation Matrix on a Python side and retrieves corrmat baack into InterSystems IRIS in a JSON format, by setting request properties:
- Variables: "corrmat"
- Serialization: "JSON"

We can see results in a visual trace:

And if we need it down the line in BP it can be saved with: callresponse.Variables.GetAt("corrmat")

Data transfer call

Next, let's talk about InterSystems IRIS -> Python data transfer, all data transfer requests extend which supplies the following common properties:
- Variable - Python variable to set.
- Type - Variable type (Currently supported: dataframe (pandas dataframe) and list.
- Namespace - Namespace in which we get the data. '' package must be available in this namespace.

Building on that are 4 concrete classes:
- - set Query property to transfer SQL resultset.
- - set Class property to transfer class data.
- - set Table property to transfer a whole table.
- - set Global property to transfer a global.

In the test process check RAW call to see in action.

Save/Restore Python context calls

Finally, we can persist Python context to InterSystems IRIS, to do that send with:
- Mask - Only variables that satisfy Mask are saved. Wildcards * and ? are accepted. Example: "Data*,Figure?". Defaults to *.
- MaxLength - Maximum length of the saved variable. If variable serialization is longer than that, it would be ignored. Set to 0 to get them all. Defaults to $$$MaxStringLength.
- Name - Context name (optional).
- Description - Extended context info (optional).

Check Save Context call in the test process for example. Returns Ens.StringResponse with Id of a saved context.

Corresponding loads context from InterSystems IRIS to Python:
- ContextId - Context identifier to restore.
- Clear - Clear context before the restore.


Python Gateway allows seamless integration between InterSystems IRIS and Python. Use it to add Python functionality to your Interoperability productions.


Illustrated guide

There's also an illustrated guide in ML Toolkit user group. ML Toolkit user group is a private GitHub repository set up as part of the InterSystems corporate GitHub organization. It is addressed to the external users that are installing, learning or are already using ML Toolkit components including Python Gateway. To join ML Toolkit user group, please send a short e-mail at the following address: and indicate in your e-mail the following details (needed for the group members to get to know and identify you during discussions):

  • GitHub username
  • Full Name (your first name followed by your last name in Latin script)
  • Organization (you are working for, or you study at, or your home office)
  • Position (your actual position in your organization, or “Student”, or “Independent”)
  • Country (you are based in)
Discussion (0)0
Log in or sign up to continue