#Data Import and Export

1 Follower · 121 Posts

This topic unites posts which describe approaches, tools, and solutions to import and export data into InterSystems IRIS and other InterSystems Data platform products: CSV, JSON, SQL, flat files, binary files, globals, streams, etc.

InterSystems staff + admins Hide everywhere
Hidden post for admin
Article Robbie Luman · Aug 15, 2025 7m read

Dynamic Entities (objects and arrays) in IRIS are incredibly useful in situations where you are having to transform JSON data into an Object Model for storage to the database, such as in REST API endpoints hosted within IRIS. This is because these dynamic objects and arrays can easily serve as a point of conversion from one data structure to the other.

4
3 277
Article Pietro Di Leo · Dec 9, 2025 4m read

Hello Community! 👋
Welcome to the second part of the IRIS IO Utility series. This extension represents my submission for the InterSystems "Bringing Ideas to Reality" Contest 2025 and offers you an intuitive and powerful interface for importing and exporting data directly inside VS Code.

If you find this extension useful, please consider voting for me at the contest!


2
0 110
Article John Murray · Dec 2, 2025 1m read

I'm pleased to announce the publication of gj :: dataLoader, a new VS Code extension that simplifies the task of loading data from local CSV files into SQL tables on your InterSystems IRIS servers.

Here's an introductory video:

gj :: dataLoader is now available on Marketplace for direct installation into VS Code. It is my entry in the "Bringing Ideas to Reality" Contest 2025 and implements this idea - https://ideas.intersystems.com/ideas/DPI-I-667

Feedback is welcome.

1
0 98
Article Pietro Di Leo · Dec 8, 2025 4m read

Hello Developers! 👋
I’m excited to introduce IRIS IO Utility, my submission for the InterSystems "Bringing Ideas to Reality" Contest 2025. This VS Code extension provides you an intuitive and powerful interface for importing and exporting data without leaving your IDE. The extension implements and expand this idea submitted on the Ideas Portal turning a common developer need into a real tool: seamless IO operations with any IRIS instance directly inside VS Code.

0
0 96
Article Corentin Blondeau · Feb 24, 2025 4m read

Hello
This article follows up on the question I had asked the community UDP Adapter not working
In this article, I will present to you
1) What is "UDP"?
2) The current state of Iris with UDP
3) My solution with the UDP adapter


1) What is "UDP"?

UDP stands for User Datagram Protocol. It is one of the core protocols of the Internet Protocol (IP) suite, used for transmitting data over a network. Here are some key features of UDP:

  1. Connectionless: UDP does not establish a connection before sending data, which means it can send messages (datagrams) without prior handshaking.
4
4 343
Article Kate Lau · Oct 9, 2025 6m read

Hi,

 

It's me again😁, recently I am working on generating some fake patient data for testing purpose with the help of Chat-GPT by using Python. And, at the same time I would like to share my learning curve.😑

1st of all for building a custom REST api service is easy by extending the %CSP.REST

Creating a REST Service Manually

Let's Start !😂

1. Create a class datagen.restservice which extends  %CSP.REST 

Class datagen.restservice Extends %CSP.REST
{
Parameter CONTENTTYPE = "application/json";
}

 

2.

3
1 129
Article Evgeny Shvarov · Aug 27, 2025 3m read

Hi folks!

It is very easy to import CSV data into IRIS. But what if we want to preserve the original IDs in CSV?

Recently I came across with the situation when I needed to import two csv's into IRIS which were linked by one column referencing  to another csv's col: a typical Foreign Key and Primary Key situation, where csv1 contains this column as Primary Key, and csv2 as Foreign key with id's related to csv1.

The image is generated by ChatGPT so don't blame it - it tried its best to generate countries as primary keys with countries.csv-cities.csv relationship :)

0
1 128
Article Raj Singh · Mar 27, 2024 2m read

In recent versions of IRIS, a powerful new data loading command has been introduced to SQL: LOAD DATA. This feature has been highly optimized to import data into IRIS extremely fast, allowing hundreds of gigabytes of data to be inserted in seconds instead of hours or days. 

This is a very exciting improvement. However, a big problem in the data loading experience still exists. Namely, the time and hassle it takes to:

  1. Define the schema for the table in which you want to load data.
  2. Figure out the syntax for the LOAD DATA command.
7
1 1079
Article Landon Minor · May 30, 2025 2m read

Profiling CCD Documents with LEAD North’s CCD Data Profiler
Ever opened a CCD and been greeted by a wall of tangled XML? You’re not alone. Despite being a core format for clinical data exchange, CCD's are notoriously dense, verbose, and unfriendly to the human eye. For developers and analysts trying to validate their structure or extract meaningful insights, navigating these documents can feel more like archaeology than engineering.


0
0 229
Article Yuri Marx · Feb 13, 2025 4m read

FHIR repositories, applications and servers typically serve clinical data in small quantities, whether to return data about a patient, their medications, vaccines, allergies, among other information. However, it is common for a large amount of data in FHIR/JSON format to be requested to be used to load into Data Lakes, identifying study cohorts, population health, or transferring data from one EHR to another. To meet these business scenarios that require large extractions and loads of data, it is recommended to use the FHIR Bulk Data Access feature provided by HL7 institution.

0
6 531
Article Luis Angel Pérez Ramos · Nov 4, 2023 5m read

I recently had the need to monitor from HealthConnect the records present in a NoSQL database in the Cloud, more specifically Cloud Firestore, deployed in Firebase. With a quick glance I could see how easy it would be to create an ad-hoc Adapter to make the connection taking advantage of the capabilities of Embedded Python, so I got to work.

Preparing the environment

To start, we need an instance of the database on which we can perform the tests. By accessing the Firebase console, we have created a new project to which we have added the Firestore database.

2
1 534
Article Robert Cemper · Feb 24, 2024 2m read

I want to address the nasty problems about reading a flat text in ASCII, UTF*
explicitly excluding HTML, EBCDIC, and other encoding.
According to Wikipedia there are at least 8 variations of control characters.

  • CR+LF is typical for Windows
  • LF is typical for the Linux/UNIX world
  • CR is Mac's favorite

As you can deduct from the names the inspiration comes from mechanical typewriters.

In IRIS* similar as in Caché or Ensemble or .

0
2 556
Article Anssi Kauppi · Jun 30, 2020 3m read
Many organisations implement centralised log management systems to separate and centralise the log data in order to e.g. automate threat detection (and response) and to comply with regulatory requirements. The primary systems of interest are the various user facing applications, but increasingly also other kinds of systems including integration platforms.

What comes to the data platform (not the user facing application) most of the the events of interest/required are available as predefined system audit events in audit log - out of the box. They just need to be enabled.

2
2 526
Article Ikram Shah · Jul 6, 2023 6m read

Idea sourced from InterSystems Ideas Portal

Previous Posts:

Using AI to Simplify Clinical Documents Storage, Retrieval, and Search

Doctor-Patient Conversations: AI-Powered Transcription and Summarization

Introduction

The healthcare industry is continuously evolving, and the need for efficient document management and patient data management is more critical than ever. In this article, we will focus on the specific aspects of integrating Google Docs and Google Sheets with FHIR data in the context of healthcare data interoperability.

0
1 252
Article Jude Mukkadayil · Mar 29, 2023 4m read

WIN SQL is the normal editor used by most of the users .But we can't download large amount of data using winsql . So I have written a tutorial how to connect with a new Java based editor called Squirrel SQL which can easily download or export data in excel or any other formats. Also I included a Java JCBC connection program to connect with the IRIS database especially a mirroring/failover server.

5
0 742
Article Philipp Bonin · Feb 13, 2023 3m read
Logo

In today's fast-paced and highly competitive manufacturing industry, efficient machine communication and data exchange is essential to maximize productivity and minimize downtime. That's where MTConnect comes in. MTConnect is an open, royalty-free standard that provides a common language for communication between machines, devices, and software applications in a manufacturing environment.

0
0 400
Article Piyush Adhikari · Oct 19, 2022 3m read

The capacity of taking numerous records every second while also facilitating real-time queries simultaneously in real time is called Hybrid Transactional Analytical Processing (HTAP). It is also called Transactional analytics or Transanalytics or Translytics and is a very useful element in scenarios where there is constant flow of real time data coming from IIOT sensors or data on fluctuations in stock market, and supporting the need for querying these data sets in real-time or near real-time.

4
0 959
Article Aya Heshmat · Oct 11, 2022 3m read

Watch the session here: HealthShare Health Connect: Tips & Tricks

Enhanced HL7 V2 Productions

1. Review of Interface Settings: Ensure these are set purposefully! (Documentation linked)

 

Review of Recent Product Features 

(Documentation Linked)

1. HL7 V2 Schema Editor

  • Drag & Drop tooling for faster schema editing!
  • Note: Add-on segments may be made optional if starting from a generic schema.

2.

0
0 321
Article Lucas Enard · Aug 17, 2022 8m read

In this GitHub we gather information from a csv, use a DataTransformation to make it into a FHIR object and then, save that information to a FHIR server all that using only Python.

The objective is to show how easy it is to manipulate data into the output we want, here a FHIR Bundle, in the IRIS full Python framework.

1. Fhir-orga-dt

3
0 659
Article Robert Cemper · Jan 2, 2022 3m read

Thanks to @Yuri Marx we have seen a very nice example for DB migration from Postgres to IRIS.
My personal problem is the use of DBeaver as a migration tool.
Especially as one of the strengths of IRIS ( and also Caché) before is the availability of the
SQLgateways that allow access to any external Db as long as for them an access usinig 
JDBC or ODBC is available. So I extended the package to demonstrate this.

3
2 914
Article Evgeny Shvarov · Sep 20, 2021 3m read

Hi folks!

Sometimes we need to import data into InterSystems IRIS from CSV. It can be done e.g. via csvgen tool that generates a class and imports all the data into it.

But what if you already have your own class and want to import data from CSV into your existing table?

There are numerous ways to do that but you can use csvgen (or csvgen-ui) again! I prepared and and example and happy to share. Here we go!

2
0 1259
Article Yuri Marx · Mar 15, 2022 4m read

This is the fourth part of the series of articles on migrating from the main databases on the market to InterSystems IRIS. In this part, the procedures for migrating from Microsoft SQL Server will be detailed. As described in previous articles, there are currently a few options to do the migration, but the two most popular options include the use of DBeaver (https://openexchange.intersystems.com/package/DBeaver) or SQLGateway. The first one will be demonstrated in this article, and the second one in an excellent article by Robert Cemper, DB Migration using SQLgateway (https://community.

0
3 891
Article Henrique Dias · Jan 13, 2022 4m read

Hey community! How are you doing?

I hope to find everyone well, and a happy 2022 to all of you!

Over the years, I've been working on a lot of different projects, and I've been able to find a lot of interesting data.

But, most of the time, the dataset that I used to work with was the customer data. When I started to join the contest in the past couple of years, I began to look for specific web datasets.

I've curated a few data by myself, but I was thinking, "This dataset is enough to help others?"

So, discussing the ideas for this contest with @José.

4
0 483
Article Yuri Marx · Mar 7, 2022 4m read

This is the third part of the series of articles on migrating from the main databases on the market to InterSystems IRIS. In this part, the procedures for migrating from DB2 will be detailed.As described in previous articles, there are currently a few options to do the migration. However, the two most popular options include the usage of DBeaver (https://openexchange.intersystems.com/package/DBeaver) or SQLGateway. The first one will be demonstrated in this article, and the second one is presented in an excellent article by Robert Cemper, DB Migration using SQLgateway (https://community.

0
1 677
Article Yuri Marx · Feb 2, 2022 4m read

This article is a follow-up to the previous one on how to migrate from popular databases (like PostgreSQL and MySQL) to IRIS.
We will use the same procedures utilized to migrate from PostgreSQL. However, you will see that it is even easier since the data types in MySQL are very similar to IRIS. That is why we will not need to create transformation rules in the columns.

Get the sample data to the migration process

In GitHub it is possible to download a docker-compose project to build and run 2 databases:

  • Source Database: MySQL database Docker instance with a sample database.
0
0 821
Article Yuri Marx · Jan 25, 2022 4m read

Sometimes it is necessary to transfer or migrate data and data schema from Postgres to IRIS. There are currently a few options for doing this, but the two most popular options are using DBeaver (https://openexchange.intersystems.com/package/DBeaver) or SQLGateway. The first will be demonstrated in this article and the second is presented in an excellent article by Robert Cemper, DB Migration using SQLgateway (https://community.intersystems.

1
1 1176
Article Yuri Marx · Jan 5, 2022 2m read

According to the WHO, The top global causes of death, in order of total number of lives lost, are associated with three broad topics (source: https://www.who.int/news-room/fact-sheets/detail/the-top-10-causes-of-death):

  1. Cardiovascular (ischaemic heart disease, stroke),
  2. Respiratory (chronic obstructive pulmonary disease, lower respiratory infections) and
  3. Neonatal conditions – which include birth asphyxia and birth trauma, neonatal sepsis and infections, and preterm birth complications.
2
3 402