#Embedded Python

Hi Contestansts!

Here are the results of bonuses gathered by applications in InterSystems Python Programming Contest 2024!

In the previous article we presented the d[IA]gnosis application developed to support the coding of diagnoses in ICD-10. In this article we will see how InterSystems IRIS for Health provides us with the necessary tools for the generation of vectors from the ICD-10 code list using a pre-trained language model, its storage and the subsequent search for similarities on all these generated vectors.

Introduction

One of the main features that have emerged with the development of AI models is what we know as RAG (Retrieval-Augmented Generation) that allows us to improve the results of LLM models

Hi Community,

We are pleased to invite you to the next InterSystems online programming contest, which is focused on Python!

🏆 InterSystems Python Contest 🏆

Duration: July 15 - August 4, 2024

Prize pool: $14,000

The invention and popularization of Large Language Models (such as OpenAI's GPT-4) has launched a wave of innovative solutions that can leverage large volumes of unstructured data that was impractical or even impossible to process manually until recently. Such applications may include data retrieval (see Don Woodlock's ML301 course for a great intro to Retrieval Augmented Generation), sentiment analysis, and even fully-autonomous AI agents, just to name a few!

In this article, I want to demonstrate how the Embedded Python feature of IRIS can be used to directly interface with the Python OpenA

Hi Developers!

Here are the technology bonuses for the InterSystems Python Contest 2024 that will give you extra points in the voting:

• IRIS Vector Search usage - 3
• Python Pex Interoperability - 3
• Python in BPL - 2
• WSGI Web Apps - 2
• Python libs: sqlalchemy and dbt - 2
• LLM AI or LangChain usage: Chat GPT, Bard and others - 3
• NoObjectScriptLine - 3
• Hugginface - 2
• Docker container usage - 2 
• ZPM Package deployment - 2
• Online Demo - 2
• Implement InterSystems Community Idea - 4
• Find a bug in Embedded Python - 2
• First Article on Developer Community - 2
• Second Article On DC - 1
• First Time Contribution - 3
• Video on YouTube - 3
• YouTube Short - 1

See the details below.

I have been using embedded python for more than 2 years now on a daily basis. May be it's time to share some feedback about this journey.

Why write this feedback? Because, I guess, I'm like most of the people here, an ObjectScript developer, and I think that the community would benefit from this feedback and could better understand the pros & cons of chosing embedded python for developing stuff in IRIS. And also avoid some pitfalls.

• Feedback : Using embedded python daily for 2 years
• Introduction
• Starting with Python
  • Python is not ObjectScript
    • Pep8
    • Modules
    • Dunders
  • Conclusion
• Embedded Python
  • What is Embedded Python ?
  • How to use Embedded Python ?
  • How I use Embedded Python
    • Use Python libraries and code as they were ObjectScript classes
      • With the language tag
      • Without the language tag
      • Conclusion
    • Use a python first approach
      • Example : iris-fhir-python-strategy
      • Remarks
      • Where to find the code
        • How to implement a Strategy
        • Implementation of InteractionsStrategy
        • Implementation of Interactions
        • Interactions in Python
        • Implementation of the abstract python class
      • Too long, do a summary

This is an attempt to run a vector search demo completely in IRIS
There are no external tools and all you need is a Terminal / Console and the management portal.
Special thanks to Alvin Ryanputra as his package iris-vector-search that was the base
of inspiration and the source for test data.
My package is based on IRIS 2024.1 release and requires attention to your processor capabilities.

I attempted to write the demo in pure ObjectScript.
Only the calculation of the description_vectoris done in embedded Python
Calculation of a vector with 384 dimensions over 2247 records takes time.
In my Docker con

I try to get a vector from calling GetEmbedding, but i failed to convert it into a vector 

Here is a simplyfied sample class: 

Class User.myclass Extends %Persistent
{Property myVECTOR As %Vector(CAPTION = "Vector");

Property myProperty As %String(MAXLEN = 40) [ Required ];

}

here the GetEmbedding part from User.mymethods:

...
ClassMethod GetEmbedding(sentences As %String) As %String [ Language = python ]
{
  import sentence_transformers   model = sentence_transformers.SentenceTransformer('C:/InterSystems/IRIS/lib/python/Lib/site-packages/sentence_transformers/models/all-MiniLM-L6-v2')
  emb

Hello everyone,

Recently, I've been working on a Business Process that processes a large JSON FHIR message containing up to 50k requests in an array within the JSON.

Currently, the code imports the JSON as a dynamic object from the original message stream, obtains an iterator from it, and processes each request one at a time in a loop.

The performance meets the requirements, even with much larger requests than the one exposed above. However, I am interested in learning the best practices for handling large JSON in ObjectScript (and why not, Embedded Python) to achieve better performance for

Pandas is not just a popular software library. It is a cornerstone in the Python data analysis landscape. Renowned for its simplicity and power, it offers a variety of data structures and functions that are instrumental in transforming the complexity of data preparation and analysis into a more manageable form. It is particularly relevant in such specialized environments as ObjectScript for Key Performance Indicators (KPIs) and reporting, especially within the framework of the InterSystems IRIS platform, a leading data management and analysis solution.
 
In the realm of data handling and analysi

Hi Community
In this article, I will introduce my application irisChatGPT which is built on LangChain Framework.
First of all, let us have a brief overview of the framework.

The entire world is talking about ChatGPT and how Large Language Models(LLMs) have become so powerful and has been performing beyond expectations, giving human-like conversations. This is just the beginning of how this can be applied to every enterprise and every domain! 

We have a yummy dataset with recipes written by multiple Reddit users, however most of the information is free text as the title or description of a post. Let's find out how we can very easily load the dataset, extract some features and analyze it using features from OpenAI large language model within Embedded Python and the Langchain framework.

Loading the dataset

First things first, we need to load the dataset or can we just connect to it?

There are different ways you can achieve this: for instance CSV Record Mapper you can use in an interoperability production or even nice OpenExchange appl

Current triage systems often rely on the experience of admitting physicians. This can lead to delays in care for some patients, especially when faced with inexperienced residents or non-critical symptoms. Additionally, it can result in unnecessary hospital admissions, straining resources and increasing healthcare costs.

We focused our project on pregnant women and conducted a survey with friends of ours who work at a large hospital in São Paulo, Brazil, specifically in the area of monitoring and caring for pregnant women.

We discovered that a significant problem occurs when an inexperien

Introduction

With the rise of Gen AI, we believe that now users should be able to access unstructured data in a much simpler fashion. Most people have many emails that they cannot often keep track of. For example, in investment/trading strategies, professionals rely on quick decisions leveraging as much information as possible. Similarly, senior employees in a startup dealing with many teams and disciplines might find it difficult to organize all the emails that they receive. These common problems can be solved using GenAI and help make their lives easier and more organized. The possibilit

ChatIRIS Health Coach, a GPT-4 based agent that leverages the Health Belief Model as a psychological framework to craft empathetic replies. This article elaborates on the backend architecture and its components, focusing on how InterSystems IRIS supports the system's functionality.

ChatIRIS Health Coach, a GPT-4 based agent that leverages the Health Belief Model (Hochbaum, Rosenstock, & Kegels, 1952) as a psychological framework to craft empathetic replies.

Problem

Do you resonate with this - A capability and impact of a technology being truly discovered when it's packaged in a right way to it's audience. Finest example would be, how the Generative AI took off when ChatGPT was put in the public for easy access and not when Transformers/RAG's capabilities were identified. At least a much higher usage came in, when the audience were empowered to explore the possibilities.  

Motivation

Recently I got to participate in MIT Grand Hack, Boston where during my conversation with other participants, I noticed immense interest from physicians and

Hi, I need to use some pythonic library from cos.
To use them I need a python dict with some python object in it
Ex in python:

obj = pythonObject("value1")
dict = {object : obj  ,key : "value2"}
result = pythonFunc(dict)

To do that I first tried to pass by dynamic object, to later convert them in dict from Json. But unfortunately the dynamic object doesn't accept python object inside it. And my pythonic function need to have an instance of my python object.
So I tried with the python built-in function dict() to create a python dictionary, but I'm not sure how to use it as in the python docume

Hi Community,

In this article, I will introduce my application iris-image-vector-search.
The image vector retrieval demo uses IRIS Embedded Python and OpenAI CLIP model to convert images into 512 dimensional vector data. Through the new feature of Vector Search, VECTOR-COSINE is used to calculate similarity and display high similarity images.

Application direction of image retrieval  

Image retrieval has important application scenarios in the medical field, and using image retrieval can greatly improve work efficiency. Image retrieval can also be applied in the following fields, such as:

Hi Community,
In this article, I will introduce my application iris-VectorLab along with step by step guide to performing vector operations.

IRIS-VectorLab is a web application that demonstrates the functionality of Vector Search with the help of embedded python. It leverages the functionality of the Python framework SentenceTransformers for state-of-the-art sentence embeddings.

Application Features

• Text to Embeddings Translation.
• VECTOR-typed Data Insertion.
• View Vector Data
• Perform Vector Search by using VECTOR_DOT_PRODUCT and VECTOR_COSINE functions.
• Demonstrate the difference between normal and vector search
• HuggingFace Text generation with the help of GPT2 LLM (Large Language Model) model and Hugging Face pipeline

Principle: After dividing the article uploaded by the user into sentences using Python, the embedded value is obtained and stored in the Iris database. Then, the similarity between sentences is compared through Iris vector search, and finally displayed on the front-end page.

The installation steps can be viewed in the readme file. It should be noted that the BERT model used in the example has some memory requirements. If there is a long-term stuck situation during the testing process, other models such as MiniLM (which is used in the online demo) can be considered. Note that if using other mo

Hi Developers,

Join us at the upcoming Developer Roundtable on April 25th at 9 am ET | 3 pm CET. 📍
We will have 2 topics covered by the invited experts and open discussion as always.

Tech Talks:
➡ Practical Usage of Embedded Python - by Stefan Wittmann Product Manager, InterSystems

▶ Recording: 

Introduction

Accessing Amazon S3 (Simple Storage Service) buckets programmatically is a common requirement for many applications. However, setting up and managing AWS accounts is daunting and expensive, especially for small-scale projects or local development environments. In this article, we'll explore how to overcome this hurdle by using Localstack to simulate AWS services. Localstack mimics most AWS services, meaning one can develop and test applications without incurring any costs or relying on an internet connection, which can be incredibly useful for rapid development and debugging. We used ObjectScript with embedded Python to communicate with Intersystems IRIS and AWS simultaneously.Before beginning, ensure you have Python and Docker installed on your system. When Localstack is set up and running, the bucket can be created and used. 

With the advent of Embedded Python, a myriad of use cases are now possible from within IRIS directly using Python libraries for more complex operations. One such operation is the use of natural language processing tools such as textual similarity comparison.

Setting up Embedded Python to Use the Sentence Transformers Library

Note: For this article, I will be using a Linux system with IRIS installed. Some of the processes for using Embedded Python with Windows, such as installing libraries, may be a bit different from Linux to Windows so please refer to the IRIS documentation for the prope

Hi Developers!

Here're the technology bonuses for the InterSystems Vector Search, GenAI, and ML contest 2024 that will give you extra points in the voting:

• Vector Search usage - 5
• IntegratedML usage - 3
• Embedded Python - 3
• LLM AI or LangChain usage: Chat GPT, Bard, and others - 3
• Questionnaire - 2
• Docker container usage - 2 
• ZPM Package deployment - 2
• Online Demo - 2
• Implement InterSystems Community Idea - 4
• Find a bug in Vector Search, or Integrated ML, or Embedded Python - 2
• First Article on Developer Community - 2
• Second Article On DC - 1
• First Time Contribution - 3
• Video on YouTube - 3
• Suggest a new idea - 1

See the details below.

Hello everybody, 

I've been experimenting with Embedded Python and have been following the steps outlined in this documentation: https://docs.intersystems.com/irislatest/csp/docbook/DocBook.UI.Page.cl…

I'm trying to convert a python dictionary into an objectscript array but there is an issue with the 'arrayref' function, that is not working as in the linked example.

This is a snapshoot of my IRIS terminal: 

USER>do##class(%SYS.Python).Shell()

Python 3.10.12 (main, Nov 202023, 15:14:05) [GCC 11.4.0] on linux
Type quit() or C

Hi,

I am using embeded python to utilize some pythonic library but i got a problem on my hand.
One of the python function i am using return multiple values

in python you would do something like that :

val1, val2, val3, = function(params)

In COS I got something like that :

lib = ##class(%SYS.Python).Import("lib")
val1 = lib.function(params)

And I don't know how to get the second and third values.
Is there a way to get them?

I recently had the need to monitor from HealthConnect the records present in a NoSQL database in the Cloud, more specifically Cloud Firestore, deployed in Firebase. With a quick glance I could see how easy it would be to create an ad-hoc Adapter to make the connection taking advantage of the capabilities of Embedded Python, so I got to work.

Preparing the environment

To start, we need an instance of the database on which we can perform the tests. By accessing the Firebase console, we have created a new project to which we have added the Firestore database.

Next we create a collection on o

As you have seen in the latest community publications, InterSystems IRIS has included since version 2024.1 the possibility of including vector data types in its database and based on this type of data vector searches have been implemented. Well, these new features reminded me of the article I published a while ago that was based on facial recognition using Embedded Python.

Introduction

For those of you who don't remember what that article was about, it is linked at the end of this article. The way the application worked was to recognize the faces present in any image and subsequently compare

Hi, I am trying to use embedded python in a cache class, but I can only get it to work in the source code namespace.

We map our client namespaces to our source code namespaces using Default Database for Routines under System > Configuration > Namespaces > Edit Namespace in the management portal.

In the source code namespace:

SOURCENEW>w ##class(EF.helloWorld).helloWorldPython()

Hello World!

SOURCENEW>ZN "EVEXAMPLE"

In the client namespace:

EVEXAMPLE>w ##class(EF.helloWorld).helloWorldPython()

W ##CLASS(EF.helloWorld).helloWorldPython()

^

<OBJECT DISPATCH> *python object not found

W