Article
· Oct 22, 2025 2m read
Tips on handling Large data

Hello community,

I wanted to share my experience about working on Large Data projects. Over the years, I have had the opportunity to handle massive patient data, payor data and transactional logs while working in an hospital industry. I have had the chance to build huge reports which had to be written using advanced logics fetching data across multiple tables whose indexing was not helping me write efficient code.

Here is what I have learned about managing large data efficiently.

Choosing the right data access method.

As we all here in the community are aware of, IRIS provides multiple ways to access data. Choosing the right method, depends on the requirement.

  • Direct Global Access: Fastest for bulk read/write operations. For example, if i have to traverse through indexes and fetch patient data, I can loop through the globals to process millions of records. This will save a lot of time.
Set ToDate=+H
Set FromDate=+$H-1 For  Set FromDate=$O(^PatientD("Date",FromDate)) Quit:FromDate>ToDate  Do
. Set PatId="" For  Set PatId=$Order(^PatientD("Date",FromDate,PatID)) Quit:PatId=""  Do
. . Write $Get(^PatientD("Date",FromDate,PatID)),!
  • Using SQL: Useful for reporting or analytical requirements, though slower for huge data sets.

3 6
1 173
InterSystems Developer Community is a community of 25,998 amazing developers
We're a place where InterSystems IRIS programmers learn and share, stay up-to-date, grow together and have fun!
Learn more

img

In this section, we will explore how to use Python as the primary language in IRIS, allowing you to write your application logic in Python while still leveraging the power of IRIS.

Open Exchange app
8 0
8 293

I am regularly contacted by customers about memory sizing when they get alerts that free memory is below a threshold, or they observe that free memory has dropped suddenly. Is there a problem? Will their application stop working because it has run out of memory for running system and application processes? Nearly always, the answer is no, there is nothing to worry about. But that simple answer is usually not enough. What's going on?

13 2
7 252

We recently changed the 'UserID" property in a "User" class from type of %String to be %Library.Username. This is for better consistency across our codebase regarding MAXLEN limit.

%Library.Username is a system wrapper datatype which extends %String and has a MAXLEN of 160. This change should have minimal/no impact on code behavior. However, we found that some SQL query cannot return expected rows after the change. Query will return empty values even if the entry is in the table.

5 3
0 231

Window functions in InterSystems IRIS let you perform powerful analytics — like running totals, rankings, and moving averages — directly in SQL.
They operate over a "window" of rows related to the current row, without collapsing results like GROUP BY.
This means you can write cleaner, faster, and more maintainable queries — no loops, no joins, no temp tables.

In this article let's understand the mechanics of window functions by addressing some common data analisys tasks.

7 0
3 289

Introduction

InterSystems IRIS allows you to build REST APIs using ObjectScript classes and the %CSP.REST framework. This enables the development of modern services to expose data for web apps, mobile apps, or system integrations.

In this article, you'll learn how to create a basic REST API in InterSystems IRIS, including:

4 3
3 229

Our goal on the Developer Community is to foster a high-quality, trustworthy, and original body of technical content created by and for developers. While AI tools like ChatGPT can be helpful in the writing process, we expect all published content to reflect authentic expertise and personal understanding. If you have any questions or wish to discuss your coding problem with an AI, consider using our Developer Community AI chat.

7 4
0 208

I am receiving the garbled text due to incorrect encoding or decoding. I tried to use the $zconvert function to convert it into the normal text but failed to do that. Can anybody suggest what I have to use to convert that into normal text?

Example: Garbled text that I am getting is "canââ¬â¢t , theyââ¬â¢re".

0 6
0 282

Hi Community!

We’re excited to announce that several winners of the InterSystems AI Programming Contest have been invited to showcase their projects at the Tech Exchange during InterSystems Ready 2025!

Join us on Wednesday, June 25, to explore innovative, real-world solutions built with InterSystems IRIS, AI, LLMs, and intelligent agent technologies — directly from the developers who created them:

2 4
0 206

When using standard SQL or the object layer in InterSystems IRIS, metadata consistency is usually maintained through built-in validation and type enforcement. However, legacy systems that bypass these layers—directly accessing globals—can introduce subtle and serious inconsistencies.

Open Exchange app
1 3
0 225

Hello InterSystems EHR community,

InterSystems Certification is currently developing a certification exam for InterSystems EHR Reports specialists, and if you match the exam candidate description given below, we would like you to beta test the exam! The exam will be available for beta testing starting June 30, 2025.

2 1
2 265

Hi all,

Let's do some more work about the testing data generation and export the result by REST API.😁

Here, I would like to reuse the datagen.restservice class which built in the pervious article Writing a REST api service for exporting the generated patient data in .csv

This time, we are planning to generate a FHIR bundle include multiple resources for testing the FHIR repository.

3 6
0 162

In this article I'll show you how to set up in your laptop, very quickly, a cluster of IRIS nodes in sharding. It's not the goal of this article neither to talk about sharding in detail nor define a deployment of a production ready architecture, but to show how to set up quickly, in your own machine, a cluster of IRIS instances configured as shard nodes, with which you'll able to play and test this functionality. If you're insterested in knowing more about sharding in IRIS, take a look at the documentation clicking here.

First and foremost, I want to remark that IRIS sharding will allow us 2 things:

  • Define, load and query shard tables, which data will be distributed transparently between the cluster's nodes
  • Define federated tables, which offer a global and composed view of data belonging to different tables that will be physically stored in different distributed nodes

So, as I said, we let for other article playing with shard or federated tables, and just focus now in the previous step, that is, setting up the cluster of shard nodes.

9 1
3 262
Article
· May 2, 2025 3m read
Minify XML in IRIS

In a project I'm working on we need to store some arbitrary XML in the database. This XML does not have any corresponding class in IRIS, we just need to store it as a string (it's relatively small and can fit in a string).
Since there are MANY (millions!) of records in the database I decided to reduce as much as possible the size without compressing. I know that some XML to be stored is indented, some not, it varies.

4 3
2 220
Article
· Sep 4, 2025 4m read
IRIS in Docker for beginners

The article was motivated by the 2025 September Article Bounty
***************************************************************

The principle of Docker is just convincing to me.

  • Get a sandbox where you play and try whatever you want/need to do
  • Once done. You drop it without leaving traces in your working environment

This was the technical base for me to run about 700 reviews in OEX
with almost no side effects (except those caused by myself).

12 3
5 217

Hi Community,

In the first part of this series, we examined the fundamentals of Interoperability on Python (IoP), specifically how it enables us to construct such interoperability elements as business services, processes, and operations using pure Python.

Now, we are ready to take things a step further. Real-world integration scenarios extend beyond simple message handoffs.They involve scheduled polling, custom message structures, decision logic, filtering, and configuration handling.In this article, we will delve into these more advanced IoP capabilities and demonstrate how to create and run a more complex interoperability flow using only Python.

To make it practical, we will build a comprehensive example: The Reddit Post Analyzer Production. The concept is straightforward: continuously retrieving the latest submissions from a chosen subreddit, filtering them based on popularity, adding extra tags to them, and sending them off for storage or further analysis.

The ultimate goal here is a reliable, self-running data ingestion pipeline. All major parts (the Business Service, Business Process, and Business Operation) are implemented in Python, showcasing how to use IoP as a Python-first integration methodology.

5 2
1 234

Hey Community! New rewards have arrived on Global Masters 🎁

With our July update, we’ve added some fresh new options for you to choose from! The rewards catalog is now packed with even more exciting ways to exchange your points for something truly useful, , and well-deserved If you’ve been saving up your points, now’s the perfect time to treat yourself!



➡️ Enter Global Masters to check out available prizes*.

🧩 Want to learn how to earn points faster? You can find tips and guidance here

7 6
1 151