Article
· Feb 5 20m read

Feedback : Using embedded python daily for more than 2 years

I have been using embedded python for more than 2 years now on a daily basis.
May be it's time to share some feedback about this journey.

Why write this feedback? Because, I guess, I'm like most of the people here, an ObjectScript developer, and I think that the community would benefit from this feedback and could better understand the pros & cons of chosing embedded python for developing stuff in IRIS. And also avoid some pitfalls.

image

Introduction

I'm a developer since 2010, and I have been working with ObjectScript since 2013.

So roughly 10 years of experience with ObjectScript.

Since 2021 and the release of Embedded Python in IRIS, I put my self a challenge :

  • Learn Python
  • Do as much as possible everything in Python

When I started this journey, I had no idea of what Python was. So I started with the basics, and I'm still learning every day.

Starting with Python

The good thing with Python is that it's easy to learn. It's even easier when you already know ObjectScript.

Why ? They have a lot in common.

ObjectScript Python
Untyped Untyped
Scripting language Scripting language
Object Oriented Object Oriented
Interpreted Interpreted
Easy C integration Easy C integration

So, if you know ObjectScript, you already know a lot about Python.

But, there are some differences, and some of them are not easy to understand.

Python is not ObjectScript

To keep it simple, I will focus on the main differences between ObjectScript and Python.

For me there are mainly 3 differences :

  • Pep8
  • Modules
  • Dunders

Pep8

What the hell is Pep8 ?

It's a set of rules to write Python code.

pep8.org

Few of them are :

  • naming convention
    • variable names
      • snake_case
    • class names
      • CamelCase
  • indentation
  • line length
  • etc.

Why is it important ?

Because it's the way to write Python code. And if you don't follow these rules, you will have a hard time to read other people's code, and they will have a hard time to read your code.

As ObjectScript developers, we also have some rules to follow, but they are not as strict as Pep8.

I learned Pep8 the hard way.

For the story, I'm a sales engineer at InterSystems, and I'm doing a lot of demos. And one day, I was doing a demo of Embedded Python to a customer, this customer was a Python developer, and the conversation turned short when he saw my code. He told me that my code was not Pythonic at all (he was right) I was coding in python like I was coding in ObjectScript. And because of that, he told me that he was not interested in Embedded Python anymore. I was shocked, and I decided to learn Python the right way.

So, if you want to learn Python, learn Pep8 first.

Modules

Modules are something that we don't have in ObjectScript.

Usually, in object oriented languages, you have classes, and packages. In Python, you have classes, packages, and modules.

What is a module ?

It's a file with a .py extension. And it's the way to organize your code.

You didn't understand ? Me neither at the beginning. So let's take an example.

Usually, when you want to create a class in ObjectScript, you create a .cls file, and you put your class in it. And if you want to create another class, you create another .cls file. And if you want to create a package, you create a folder, and you put your .cls files in it.

In Python, it's the same, but Python bring the ability to have multiple classes in a single file. And this file is called a module.
FYI, It's Pythonic to have multiple classes in a single file.

So plan head how you will organize your code, and how you will name your modules to not end up like me with a lot of modules with the same name as your classes.

A bad example :

MyClass.py

class MyClass:
    def __init__(self):
        pass

    def my_method(self):
        pass

To instantiate this class, you will do :

import MyClass.MyClass # weird right ?

my_class = MyClass()

Weird right ?

Dunders

Dunders are special methods in Python. They are called dunder because they start and end with double underscores.

They are kind of our % methods in ObjectScript.

They are used for :

  • constructor
  • operator overloading
  • object representation
  • etc.

Example :

class MyClass:
    def __init__(self):
        pass

    def __repr__(self):
        return "MyClass"

    def __add__(self, other):
        return self + other

Here we have 3 dunder methods :
* __init__ : constructor
* __repr__ : object representation
* __add__ : operator overloading

Dunders methods are everywhere in Python. It's a major part of the language, but don't worry, you will learn them quickly.

Conclusion

Python is not ObjectScript, and you will have to learn it. But it's not that hard, and you will learn it quickly.
Just keep in mind that you will have to learn Pep8, and how to organize your code with modules and dunder methods.

Good sites to learn Python :


Embedded Python

Now that you know a little bit more about Python, let's talk about Embedded Python.

What is Embedded Python ?

Embedded Python is a way to execute Python code in IRIS. It's a new feature of IRIS 2021.2+.
This means that your python code will be executed in the same process as IRIS.
For the more, every ObjectScript class is a Python class, same for methods and attributes and vice versa. 🥳
This is neat !

How to use Embedded Python ?

There are 3 main ways to use Embedded Python :

  • Using the language tag in ObjectScript
    • Method Foo() As %String [ Language = python ]
  • Using the ##class(%SYS.Python).Import() function
  • Using the python interpreter
    • python3 -c "import iris; print(iris.system.Version.GetVersion())"

But if you want to be serious about Embedded Python, you will have to avoid using the language tag.

image

Why ?

  • Because it's not Pythonic
  • Because it's not ObjectScript either
  • Because you don't have a debugger
  • Because you don't have a linter
  • Because you don't have a formatter
  • Because you don't have a test framework
  • Because you don't have a package manager
  • Because you are mixing 2 languages in the same file
  • Because when you process crashes, you don't have a stack trace
  • Because you can't use virtual environments or conda environments
  • ...

Don't get me wrong, it works, it can be useful, if you want to test something quickly, but IMO it's not a good practice.

So, what did I learn from this 2 years of Embedded Python, and how to use it the right way ?

How I use Embedded Python

For me, you have two options :

  • Use Python libraries as they were ObjectScript classes
    • with ##class(%SYS.Python).Import() function
  • Use a python first approach

Use Python libraries and code as they were ObjectScript classes

You still want to use Python in your ObjectScript code, but you don't want to use the language tag. So what can you do ?

"Simply" use Python libraries and code as they were ObjectScript classes.

Let's take an example :

You want to use the requests library ( it's a library to make HTTP requests ) in your ObjectScript code.

With the language tag

ClassMethod Get() As %Status [ Language = python ]
{
    import requests

    url = "https://httpbin.org/get"
    # make a get request
    response = requests.get(url)
    # get the json data from the response
    data = response.json()
    # iterate over the data and print key-value pairs
    for key, value in data.items():
        print(key, ":", value)
}

Why I think it's not a good idea ?

Because you are mixing 2 languages in the same file, and you don't have a debugger, a linter, a formatter, etc.
If this code crashes, you will have a hard time to debug it.
You don't have a stack trace, and you don't know where the error comes from.
And you don't have auto-completion.

Without the language tag

ClassMethod Get() As %Status
{
    set status = $$$OK
    set url = "https://httpbin.org/get"
    // Import Python module "requests" as an ObjectScript class
    set request = ##class(%SYS.Python).Import("requests")
    // Call the get method of the request class
    set response = request.get(url)
    // Call the json method of the response class
    set data = response.json()
    // Here data is a Python dictionary
    // To iterate over a Python dictionary, you have to use the dunder method and items()
    // Import built-in Python module
    set builtins = ##class(%SYS.Python).Import("builtins")
    // Here we are using len from the builtins module to get the length of the dictionary
    For i = 0:1:builtins.len(data)-1 {
        // Now we convert the items of the dictionary to a list, and we get the key and the value using the dunder method __getitem__
        Write builtins.list(data.items())."__getitem__"(i)."__getitem__"(0),": ",builtins.list(data.items())."__getitem__"(i)."__getitem__"(1),!
    }
    quit status
}

Why I think it's a good idea ?

Because you are using Python as it was ObjectScript. You are importing the requests library as an ObjectScript class, and you are using it as an ObjectScript class.
All the logic is in ObjectScript, and you are using Python as a library.
Even for maintenance, it's easier to read and understand, any ObjectScript developer can understand this code.
The drawback is that you have to know how to use duners methods, and how to use Python as it was ObjectScript.

Conclusion

Belive me, this way you will end up with a more robust code, and you will be able to debug it easily.
At first, it's seems hard, but you will find the benefits of learning Python faster than you think.

Use a python first approach

This is the way I prefer to use Embedded Python.

I have built a lot of tools using this approach, and I'm very happy with it.

Few examples :

So, what is a python first approach ?

There is only one rule : Python code must be in .py files, ObjectScript code must be in .cls files

How to achieve this ?

The whole idea is to create ObjectScript wrappers classes to call Python code.


Let's take the example of iris-fhir-python-strategy :

Example : iris-fhir-python-strategy

First of all, we have to understand how IRIS FHIR Server works.

Every IRIS FHIR Server implements a Strategy.

A Strategy is a set of two classes :

Superclass Subclass Parameters
HS.FHIRServer.API.InteractionsStrategy StrategyKey — Specifies a unique identifier for the InteractionsStrategy.
InteractionsClass — Specifies the name of your Interactions subclass.
HS.FHIRServer.API.RepoManager StrategyClass — Specifies the name of your InteractionsStrategy subclass.
StrategyKey — Specifies a unique identifier for the InteractionsStrategy. Must match the StrategyKey parameter in the InteractionsStrategy subclass.

Both classes are Abstract classes.

  • HS.FHIRServer.API.InteractionsStrategy is an Abstract class that must be implemented to customize the behavior of the FHIR Server.
  • HS.FHIRServer.API.RepoManager is an Abstract class that must be implemented to customize the storage of the FHIR Server.

Remarks

For our example, we will only focus on the HS.FHIRServer.API.InteractionsStrategy class even if the HS.FHIRServer.API.RepoManager class is also implemented and mandatory to customize the FHIR Server.
The HS.FHIRServer.API.RepoManager class is implemented by HS.FHIRServer.Storage.Json.RepoManager class, which is the default implementation of the FHIR Server.

Where to find the code

All source code can be found in this repository : iris-fhir-python-strategy
The src folder contains the following folders :

  • python : contains the python code
  • cls : contains the ObjectScript code that is used to call the python code

How to implement a Strategy

In this proof of concept, we will only be interested in how to implement a Strategy in Python, not how to implement a RepoManager.

To implement a Strategy you need to create at least two classes :

  • A class that inherits from HS.FHIRServer.API.InteractionsStrategy class
  • A class that inherits from HS.FHIRServer.API.Interactions class

Implementation of InteractionsStrategy

HS.FHIRServer.API.InteractionsStrategy class aim to customize the behavior of the FHIR Server by overriding the following methods :

  • GetMetadataResource : called to get the metadata of the FHIR Server
    • this is the only method we will override in this proof of concept

HS.FHIRServer.API.InteractionsStrategy has also two parameters :

  • StrategyKey : a unique identifier for the InteractionsStrategy
  • InteractionsClass : the name of your Interactions subclass

Implementation of Interactions

HS.FHIRServer.API.Interactions class aim to customize the behavior of the FHIR Server by overriding the following methods :

  • OnBeforeRequest : called before the request is sent to the server
  • OnAfterRequest : called after the request is sent to the server
  • PostProcessRead : called after the read operation is done
  • PostProcessSearch : called after the search operation is done
  • Read : called to read a resource
  • Add : called to add a resource
  • Update : called to update a resource
  • Delete : called to delete a resource
  • and many more...

We implement HS.FHIRServer.API.Interactions class in the src/cls/FHIR/Python/Interactions.cls class.

 

Spoiler

 

The FHIR.Python.Interactions class inherits from HS.FHIRServer.Storage.Json.Interactions class and FHIR.Python.Helper class.

The HS.FHIRServer.Storage.Json.Interactions class is the default implementation of the FHIR Server.

The FHIR.Python.Helper class aim to help to call Python code from ObjectScript.

The FHIR.Python.Interactions class overrides the following methods :

  • %OnNew : called when the object is created
    • we use this method to set the python path, python class name and python module name from environment variables
    • if the environment variables are not set, we use default values
    • we also set the python class
    • we call the %OnNew method of the parent class
Method %OnNew(pStrategy As HS.FHIRServer.Storage.Json.InteractionsStrategy) As %Status
{
    // First set the python path from an env var
    set ..PythonPath = $system.Util.GetEnviron("INTERACTION_PATH")
    // Then set the python class name from the env var
    set ..PythonClassname = $system.Util.GetEnviron("INTERACTION_CLASS")
    // Then set the python module name from the env var
    set ..PythonModule = $system.Util.GetEnviron("INTERACTION_MODULE")

    if (..PythonPath = "") || (..PythonClassname = "") || (..PythonModule = "") {
        // use default values
        set ..PythonPath = "/irisdev/app/src/python/"
        set ..PythonClassname = "CustomInteraction"
        set ..PythonModule = "custom"
    }

    // Then set the python class
    do ..SetPythonPath(..PythonPath)
    set ..PythonClass = ..GetPythonInstance(..PythonModule, ..PythonClassname)

    quit ##super(pStrategy)
}
  • OnBeforeRequest : called before the request is sent to the server
    • we call the on_before_request method of the python class
    • we pass the HS.FHIRServer.API.Service object, the HS.FHIRServer.API.Data.Request object, the body of the request and the timeout
Method OnBeforeRequest(
    pFHIRService As HS.FHIRServer.API.Service,
    pFHIRRequest As HS.FHIRServer.API.Data.Request,
    pTimeout As %Integer)
{
    // OnBeforeRequest is called before each request is processed.
    if $ISOBJECT(..PythonClass) {
        set body = ##class(%SYS.Python).None()
        if pFHIRRequest.Json '= "" {
            set jsonLib = ##class(%SYS.Python).Import("json")
            set body = jsonLib.loads(pFHIRRequest.Json.%ToJSON())
        }
        do ..PythonClass."on_before_request"(pFHIRService, pFHIRRequest, body, pTimeout)
    }
}
  • OnAfterRequest : called after the request is sent to the server
    • we call the on_after_request method of the python class
    • we pass the HS.FHIRServer.API.Service object, the HS.FHIRServer.API.Data.Request object, the HS.FHIRServer.API.Data.Response object and the body of the response
Method OnAfterRequest(
    pFHIRService As HS.FHIRServer.API.Service,
    pFHIRRequest As HS.FHIRServer.API.Data.Request,
    pFHIRResponse As HS.FHIRServer.API.Data.Response)
{
    // OnAfterRequest is called after each request is processed.
    if $ISOBJECT(..PythonClass) {
        set body = ##class(%SYS.Python).None()
        if pFHIRResponse.Json '= "" {
            set jsonLib = ##class(%SYS.Python).Import("json")
            set body = jsonLib.loads(pFHIRResponse.Json.%ToJSON())
        }
        do ..PythonClass."on_after_request"(pFHIRService, pFHIRRequest, pFHIRResponse, body)
    }
}
  • And so on...

Interactions in Python

FHIR.Python.Interactions class calls the on_before_request, on_after_request, ... methods of the python class.

Here is the abstract python class :

import abc
import iris

class Interaction(object):
    __metaclass__ = abc.ABCMeta

    @abc.abstractmethod
    def on_before_request(self, 
                          fhir_service:'iris.HS.FHIRServer.API.Service',
                          fhir_request:'iris.HS.FHIRServer.API.Data.Request',
                          body:dict,
                          timeout:int):
        """
        on_before_request is called before the request is sent to the server.
        param fhir_service: the fhir service object iris.HS.FHIRServer.API.Service
        param fhir_request: the fhir request object iris.FHIRServer.API.Data.Request
        param timeout: the timeout in seconds
        return: None
        """


    @abc.abstractmethod
    def on_after_request(self,
                         fhir_service:'iris.HS.FHIRServer.API.Service',
                         fhir_request:'iris.HS.FHIRServer.API.Data.Request',
                         fhir_response:'iris.HS.FHIRServer.API.Data.Response',
                         body:dict):
        """
        on_after_request is called after the request is sent to the server.
        param fhir_service: the fhir service object iris.HS.FHIRServer.API.Service
        param fhir_request: the fhir request object iris.FHIRServer.API.Data.Request
        param fhir_response: the fhir response object iris.FHIRServer.API.Data.Response
        return: None
        """


    @abc.abstractmethod
    def post_process_read(self,
                          fhir_object:dict) -> bool:
        """
        post_process_read is called after the read operation is done.
        param fhir_object: the fhir object
        return: True the resource should be returned to the client, False otherwise
        """


    @abc.abstractmethod
    def post_process_search(self,
                            rs:'iris.HS.FHIRServer.Util.SearchResult',
                            resource_type:str):
        """
        post_process_search is called after the search operation is done.
        param rs: the search result iris.HS.FHIRServer.Util.SearchResult
        param resource_type: the resource type
        return: None
        """

Implementation of the abstract python class

from FhirInteraction import Interaction

class CustomInteraction(Interaction):

    def on_before_request(self, fhir_service, fhir_request, body, timeout):
        #Extract the user and roles for this request
        #so consent can be evaluated.
        self.requesting_user = fhir_request.Username
        self.requesting_roles = fhir_request.Roles

    def on_after_request(self, fhir_service, fhir_request, fhir_response, body):
        #Clear the user and roles between requests.
        self.requesting_user = ""
        self.requesting_roles = ""

    def post_process_read(self, fhir_object):
        #Evaluate consent based on the resource and user/roles.
        #Returning 0 indicates this resource shouldn't be displayed - a 404 Not Found
        #will be returned to the user.
        return self.consent(fhir_object['resourceType'],
                        self.requesting_user,
                        self.requesting_roles)

    def post_process_search(self, rs, resource_type):
        #Iterate through each resource in the search set and evaluate
        #consent based on the resource and user/roles.
        #Each row marked as deleted and saved will be excluded from the Bundle.
        rs._SetIterator(0)
        while rs._Next():
            if not self.consent(rs.ResourceType,
                            self.requesting_user,
                            self.requesting_roles):
                #Mark the row as deleted and save it.
                rs.MarkAsDeleted()
                rs._SaveRow()

    def consent(self, resource_type, user, roles):
        #Example consent logic - only allow users with the role '%All' to see
        #Observation resources.
        if resource_type == 'Observation':
            if '%All' in roles:
                return True
            else:
                return False
        else:
            return True

Too long, do a summary

The FHIR.Python.Interactions class is a wrapper to call the python class.

IRIS abstracts classes are implemented to wrap python abstract classes 🥳.

That help us to keep python code and ObjectScript code separated and for so benefit from the best of both worlds.

Discussion (10)2
Log in or sign up to continue

Great article!

I've gained a clearer understanding after reading it. However, I'm struggling a bit with your example. It seems a bit complex, especially for those of us who are new to integrating ObjectScript with custom Python classes and routines. A more straightforward example would be really helpful.

Your code particularly draws attention to the use of this class:

ClassMethod GetPythonInstance(
	pModule,
	pRemoteClassname) As %SYS.Python
{
    set importlib = ##class(%SYS.Python).Import("importlib")
    set builtins = ##class(%SYS.Python).Import("builtins")
    set sys = ##class(%SYS.Python).Import("sys")

    set module = importlib."import_module"(pModule)
    do importlib."reload"(module)

    set class = builtins.getattr(module, pRemoteClassname)
    return class."__new__"(class)
}
 

Could you elaborate on the rationale behind this specific structure, particularly the inclusion of builtins, importlib, and sys? Is there a simpler alternative you might suggest?

In my own projects, I've been using embedded Python with the Python tag, which usually is enough due to its simplicity and speed. However, as you mentioned, it lacks certain features. Eventually, I end up with numerous embedded functions, repeatedly importing the same libraries for each method, which looks cluttered. See:
 

ClassMethod moveFile() [ Language = python ]
{
    import boto3,iris
    # do stuff
}

ClassMethod getFile() [ Language = python ]
{
    import boto3,iris,json,datetime
    # do stuff
}


At this juncture, adopting your strategy to transfer all code to a .py file (not as a class) seems like a viable option, then accessing these methods via ObjectScript. I'm inclined to avoid Python classes due to their complexity and the need for instantiation. I prefer using functions for their simplicity and ease of understanding. What would you recommend in this scenario? How would calling code looks like? 

 

Thanks @Mario Sanchez Macias for your feedback. I understand your concerns and will address them in this response.

The example I provided is indeed complex, and I appreciate your patience. I will provide a simpler example to illustrate the integration of Python with ObjectScript.

First, I will give you a quick tip on how to simplify your code without the need of importing the same libraries repeatedly.

In ObjectScript, you create a class like this:

Class dc.boto3 Extends %RegisteredObject
{

XData %import [ MimeType = application/python ]
{
import boto3
}

ClassMethod moveFile() [ Language = python ]
{
    boto3.client('s3').download_file('bucket', 'key', 'filename')
    # do stuff
}

ClassMethod getFile() [ Language = python ]
{
    boto3.client('s3').upload_file('filename', 'bucket', 'key')
    # do stuff
}

}

In this example, I've created a class called dc.boto3 that extends %RegisteredObject. I've added an XData block to import the boto3 library. This way, you don't need to import the library in each method.

But in this example you are still mixing Python and ObjectScript code.

You can also try to stick to only to ObjectScript and use the ##class(%SYS.Python).Import method to import the Python library and use it in your ObjectScript code.

Here's an example of how you can use it to import the boto3 library and use it in your ObjectScript code:

Class dc.boto3 Extends %RegisteredObject
{

ClassMethod moveFile()
{
    set boto3 = ##class(%SYS.Python).Import("boto3")
    set s3 = boto3.client('s3')
    do s3."download_file"('bucket', 'key', 'filename')
    # do stuff
}

ClassMethod getFile()
{
    set boto3 = ##class(%SYS.Python).Import("boto3")
    set s3 = boto3.client('s3')
    do s3."upload_file"('filename', 'bucket', 'key')
    # do stuff
}

}

Now, to move to a python only approach, this will depend on your application architecture and the complexity of your Python code.

  • First, do you use the Interoperability framework of Iris ?

If yes, it will be quite easy have a look at IoP, it will allow you to create Services, Processes, Operations in Python Only and call them from a Production.

  • Your entry point is %CSP.Rest application ?

You can start to create .py files and import them in your ObjectScript code. with the ##class(%SYS.Python).Import method.

  • Create an ObjectScript abstract class that will be used to call your Python code.

This is my favorite approach, but the most complex one. This is the one I try to explain in the article.

If we take the example of the boto3 library.

First create the abstract class in ObjectScript:

Class dc.boto3 Extends %RegisteredObject [ Abstract ]
{

ClassMethod moveFile()
{
    Quit $$$ERROR($$$NOTIMPLEMENTED)
}

ClassMethod getFile()
{
    Quit $$$ERROR($$$NOTIMPLEMENTED)
}

}

The great thing about this approach is that you can create a class that extends this abstract class and implement the methods in Python, ObjectScript or any other language.

Then create the implementation of this class in ObjectScript:

Class dc.boto3Python Extends dc.boto3
{

Property PythonPath As %String;
Property PythonClassname As %String;
Property PythonModule As %String;
Property PythonClass As %SYS.Python;

Method %OnNew() As %Status
{
    // %OnNew is called when the object is created.
    set ..PythonPath = $system.Util.GetEnviron("PYTHON_BOTO_PATH")
    // Then set the python class name from the env var
    set ..PythonClassname = $system.Util.GetEnviron("PYTHON_BOTO_CLASS")
    // Then set the python module name from the env var
    set ..PythonModule = $system.Util.GetEnviron("PYTHON_BOTO_MODULE")

    if (..PythonPath = "") || (..PythonClassname = "") || (..PythonModule = "") {
        //quit ##super(pStrategy)
        set ..PythonPath = "/irisdev/app/src/python/"
        set ..PythonClassname = "CustomPythonBoto"
        set ..PythonModule = "custom"
    }


    // Then set the python class
    do ..SetPythonPath(..PythonPath)
    set ..PythonClass = ##class(FHIR.Python.Interactions).GetPythonInstance(..PythonModule, ..PythonClassname)

    quit ##super(pStrategy)
}

ClassMethod moveFile()
{
    do ..PythonClass."moveFile"()
}

ClassMethod getFile()
{
    do ..PythonClass."getFile"()
}

}

Now you can create a Python class that is used to implement the methods of the dc.boto3Python class.

import boto3

class CustomPythonBoto:
    def moveFile(self):
        s3 = boto3.client('s3')
        s3.download_file('bucket', 'key', 'filename')
        # do stuff

    def getFile(self):
        s3 = boto3.client('s3')
        s3.upload_file('filename', 'bucket', 'key')
        # do stuff

Now anone in your company can use the dc.boto3Python class as it's an ObjectScript class. But in fact, it's proxying the Python class CustomPythonBoto.

I hope this helps. Let me know if you have any further questions.

Thanks! And nice tips and lovely explanation,  now everything is making more sense. 

Regarding the pure Python approach, I am still struggling with how to use it in a much simpler way. Your last example is still too complex for a very simple scenario of one or two python files. 

Let me explain. I have this class in the file hello_world.py

class HelloWorld:
    def sayhello(self):
        print("Hello, World!")
        return "you did it!"

Then I would like to use it doing something like: 

Class dc.boto Extends %RegisteredObject
{

ClassMethod test1() [ Language = python ]
{
    from hello_world import HelloWorld
    # Create an instance of HelloWorld
    greeting = HelloWorld()

    # Call the say_hello method
    ret=greeting.sayhello()
    print(ret)
}

ClassMethod test2()
{
    Set hello = ##class(%SYS.Python).Import("/iris-shared/python/hello_world.py")
    Write hello.sayhello()
}

}

But this is throwing ModuleNotFoundError errors. 

I don't know if I have to create a packet and import it with pip... if so, it will be still complex. 😌
Is this possible? Do you see my point?

Ok, I see your point and I have a better understanding of your first question.

The issue here is that the Python interpreter is not able to find the module hello_world because it is not in the Python path. The Python path is a list of directories that the interpreter searches to find the modules that you import. By default, the Python path in embedded python is /<install dir>/lib/python.

To solve this issue, you can add the directory containing the hello_world module to the Python path. You can do this by adding the directory to the sys.path list in Python. Here is an example of how you can do this:

Class dc.PythonProxy Extends %RegisteredObject
{

Property PythonPath As %String;
Property PythonModule As %String;
Property PythonClassname As %String;
Property PythonClass As %SYS.Python;

ClassMethod SetPythonPath(pClasspaths)
{
    set sys = ##class(%SYS.Python).Import("sys")

    // avoid adding the same path multiple times
    for i=0:1:(sys.path."__len__"()-1) {
        Try {
            if sys.path."__getitem__"(i) = pClasspaths {
                do sys.path."__delitem__"(i)
            }
        }
        Catch ex {
            // do nothing
        }

    }
    do sys.path.insert(0, pClasspaths)
}

ClassMethod GetPythonInstance(
    pModule,
    pRemoteClassname) As %SYS.Python
{
    set importlib = ##class(%SYS.Python).Import("importlib")
    set builtins = ##class(%SYS.Python).Import("builtins")

    set module = importlib."import_module"(pModule)
    do importlib."reload"(module)

    set class = builtins.getattr(module, pRemoteClassname)
    return class."__new__"(class)
}


Method %OnNew(pPythonPath,pPythonModule,pPythonClassname) As %Status
{
    // set variables
    set ..PythonPath = pPythonPath
    set ..PythonModule = pPythonModule
    set ..PythonClassname = pPythonClassname

    // Then set the python class
    do ..SetPythonPath(..PythonPath)
    set ..PythonClass = ..GetPythonInstance(..PythonModule, ..PythonClassname)

    quit $$$OK
}

}

With this class you have a generic way to import any python module and class. You can use it like this:

Set pythonProxy = ##class(dc.PythonProxy).%New("/iris-shared/python", "hello_world", "HelloWorld")
Write pythonProxy.PythonClass.sayhello()

This will add the /iris-shared/python directory to the Python path and then import the hello_world module and create an instance of the HelloWorld class.

With the language tag you can do something like this:

Class dc.boto Extends %RegisteredObject
{

ClassMethod test1() [ Language = python ]
{
    # set path
    import sys

    # check if the path is already in the sys.path
    if "/iris-shared/python" not in sys.path:
        sys.path.insert(0, "/iris-shared/python")

    # import module
    import hello_world
    greeting = hello_world.HelloWorld()

    # Call the say_hello method
    ret=greeting.sayhello()
    print(ret)
}

}

I hope this helps! Let me know if you have any other questions.

Very nice article. Its exiting to see that Intersystems developers are taking advantage of the (great) Embedded Python feature in IRIS.

If you want to see a "real life" use case (we are using it for more than 2 years in our production environment), check this article: IRIS Embedded Python with Azure Service Bus (ASB) use case | InterSystems
(it also won the 1st place in an Intersystems article competition in 2022)
 

Thank you for your kind words, I'm glad you liked the article. I've read your article and it's a great use case of Embedded Python in IRIS.

I noticed that you are making an extensive use of the language tags ( [ language = python ])in your article.

In this article, I try to explain how I try to move away from the language tags and instead stick with only ObjectScript code and use the ##class(%SYS.Python).Import() method to call Python code from ObjectScript or even have a python first approach.

Since you are using the language tags, I'm curious to know if you have tried the ##class(%SYS.Python).Import() method or a python first approach and if so, what are the advantages and disadvantages of using the language tags over the ##class(%SYS.Python).Import() method and the python first approach?

Are you agree with me on all the drawbacks of using the language tags ( it's not Pythonic, it's not ObjectScript either, you don't have a debugger, you don't have a linter, you mixing two language in the same file, when you process crashes, you don't have a stack trace, ... ) ?

Hi Guillaume,

thank you for the post and the straightforward recommendations. I have been using embedded Python for a while and have found that the hybrid between ObjectScript and python code is not always easy to maintain and refactor.

That's why I have now consistently outsourced the python code to .py files and imported my modules via

##class(%SYS.Python).Import("mymodule")

after reading your post. So far I'm very happy with how things are going, except I've had problems when applying the pythonic naming conventions for method names.

def validate_header():

caused errors when I tried to call it this way:

$$$ThrowOnError(mymodule.validate_header())

I had to rename the method to

def validateHeader():

which isn't particularly bad, but I thought I'd mention it here in case anyone is stuck in a similar situation