New post

#Data Import and Export

1

This topic unites posts which describe approaches, tools, and solutions to import and export data into InterSystems IRIS and other InterSystems Data platform products: CSV, JSON, SQL, flat files, binary files, globals, streams, etc.

Hello everyone! This is my first post into the developer community and one that I hope is fairly simple to answer. In our environment we currently have 3 different Test environments for testing before migrating code to our Prod environment. Currently we are working on establishing a source control method using Git in house which has been a bit of a struggle. We have also had developers that had used different test and migration methods in the past which has caused some issues with keeping the different test environment in sync.

0 7
0 73

Prefer not to read? Check out the demo video I created:

https://www.youtube.com/embed/-OwOAHC5b3s
[This is an embedded link, but you cannot view embedded content directly on the site because you have declined the cookies necessary to access it. To view embedded content, you would need to accept all cookies in your Cookies Settings]

Open Exchange app
3 2
1 97

The latest "Bringing Ideas to Reality" InterSystems competition saw me trawling through the ideas portal for UI problems to have a go at.

https://www.youtube.com/embed/zw51X1JQhQ0
[This is an embedded link, but you cannot view embedded content directly on the site because you have declined the cookies necessary to access it. To view embedded content, you would need to accept all cookies in your Cookies Settings]

Open Exchange app
3 0
0 101

In recent versions of IRIS, a powerful new data loading command has been introduced to SQL: LOAD DATA. This feature has been highly optimized to import data into IRIS extremely fast, allowing hundreds of gigabytes of data to be inserted in seconds instead of hours or days.

This is a very exciting improvement. However, a big problem in the data loading experience still exists. Namely, the time and hassle it takes to:

Open Exchange app
10 6
0 802

when doing ETL process from cache database to mysql database(ods) and to another mysql database(dwd).

There are lots of DR columns in one table.

In the first step we need two databases to be totally consistent(ods=cache database)

However, the DR column in another database lost its meaning and become to normal number.

In the last step we do the data governance in ODS to DWD. My question is how to write sql without many `join` in this step.

0 0
0 49

Hi everyone,

We have successfully configured the Patient Index and are now looking to bring it to upper environments. Our plan is to export the Linkage Definition and import it into higher environments to avoid any manual work. However, I am having trouble finding an export option in the Linkage Definition Designer and locating the globals that hold this setup.

I would greatly appreciate any tips or guidance on how to export/import the Patient Index configuration between environments.

Thank you in advance!

0 4
0 93

I recently had the need to monitor from HealthConnect the records present in a NoSQL database in the Cloud, more specifically Cloud Firestore, deployed in Firebase. With a quick glance I could see how easy it would be to create an ad-hoc Adapter to make the connection taking advantage of the capabilities of Embedded Python, so I got to work.

Open Exchange app
4 2
1 384

Hi,

I have an odd problem that occurs when I'm trying to import lookup tables in the XML format into Studio.

I store my files in in C:\CodeRepository.

I can manually import these lookup tables using "Import local..." option in the InterSystems Studio.

However, when I'm trying to import them using command in terminal: do $system.OBJ.LoadDir("C:\CodeRepository\","ck")

I'm getting the following error:

0 1
0 179
Article
· Feb 24, 2024 2m read
The bewitched line terminator

I want to address the nasty problems about reading a flat text in ASCII, UTF*
explicitly excluding HTML, EBCDIC, and other encoding.
According to Wikipedia there are at least 8 variations of control characters.

  • CR+LF is typical for Windows
  • LF is typical for the Linux/UNIX world
  • CR is Mac's favorite

As you can deduct from the names the inspiration comes from mechanical typewriters.

25 0
1 411

I am exporting data from Cache using the SQL export wizard. This is on a docker image of Cache hosted on a Linux server.

I select my database, schema, and table, make sure that all columns are being exported, check other settings, and then finish the export.

All server settings look correct and permissions are rwe on the directory. I have tried various directories, including a system level directory. I tried creating the file before I do the export in case there was an issue with creating the files, and I also made sure the created files had rwe permisisons.

0 0
0 115

Hi everyone,

Does anyone know how to export projects via VSC?

I opened the project through the "InterSystems Tools" plugin (command is "Edit Code in Project") and I can correctly work on it.

However, when I try using the "ObjectScript" plugin to export the project (right click on the project -> "Export Project Contents")

This message appears and it is not possible to export the project:

0 3
0 2.3K
Article
· Jun 30, 2020 3m read
Replicating Audit Log Near Real Time
Many organisations implement centralised log management systems to separate and centralise the log data in order to e.g. automate threat detection (and response) and to comply with regulatory requirements. The primary systems of interest are the various user facing applications, but increasingly also other kinds of systems including integration platforms.
1 2
2 427

Is it planned that LOAD DATA takes into account several DATE/DATETIME formats with, for example, a parameter indicating the format used in the source data?

example :

LOAD DATA .../...
USING
{
  "from": {
    "file": {
       "dateformat": "DD/MM/YYYY"
    }
  }
}
2 3
0 209

We are retiring a hosted application for an electronic health care records (EHR) system which stored the data on Cache for UNIX (Red Hat Enterprise Linux for x86-64) 2017.2.2 (Build 867_4_20245) Thu Oct 8 2020 16:58:40 EDT. The hosting company is providing me with a single CBK file. I need to install a database system to restore the database and provide occasional SQL access for reports when necessary. I'll need to maintain access to the data for an approximately 10 year retention period. Not sure how to approach restoring this old of a database and eventually upgrading it to a newer re

0 36
1 1K

Hi,

I am trying to load all the data tables from one iris server to a client server but some of the tables data failing to load all the time. But I can load around 100 tables successfully but 8 to 10 tables are failing all the time. I made an IRIS odbc connection using odbc driver to load the data from tables.

Also I can see read server loop error message on the iris server side as the same time the table loading fails.

Please find the screen shot attached which shows the error on client server.

0 15
0 1.1K

Hey Developers,

Enjoy watching the new video on InterSystems Developers YouTube:

⏯ Projecting Data into InterSystems IRIS with Foreign Tables

https://www.youtube.com/embed/FnSPT1U9u9A
[This is an embedded link, but you cannot view embedded content directly on the site because you have declined the cookies necessary to access it. To view embedded content, you would need to accept all cookies in your Cookies Settings]

2 0
0 283