#Data Import and Export

1 Follower · 121 Posts

This topic unites posts which describe approaches, tools, and solutions to import and export data into InterSystems IRIS and other InterSystems Data platform products: CSV, JSON, SQL, flat files, binary files, globals, streams, etc.

Question Vitaly Furman · Apr 5, 2022

Hello,

I am working on the new USCDI Requirement to store LOINC Codes/Descriptions for various types of Clinical Notes in the SDA.

According to all of the documentation I had been able to find, in order to store a coded value for Document Type, I have to use Custom Pairs:

#; Code System
        do ..write(cr_"<CustomPairs")
        do ..write(cr_"<NVPair>")
        do ..write(cr_"<Name>DocumentNameCodingStandard</Name>")
        do ..write(cr_"<Value>LN</Value>")
        do ..write(cr_"</NVPair>")
        #; Code
        do ..write(cr_"<NVPair>")

3
0 442
Question Roderick Julius · Mar 16, 2022

Hello,

Our team is looking for a way to export all of our Cache SQL tables into Microsoft SQL Server.  I have only found a method to export one table at a time into an ASCII file.  We have over 170 tables so this would be very tedious and time consuming.  Is there a way to directly export from Cache to SQL Server.  Alternatively is it possible to export the entire database in a single shot or even multiple tables to text files?

Thanks!

2
0 1002
Article Yuri Marx · Mar 15, 2022 4m read

This is the fourth part of the series of articles on migrating from the main databases on the market to InterSystems IRIS. In this part, the procedures for migrating from Microsoft SQL Server will be detailed. As described in previous articles, there are currently a few options to do the migration, but the two most popular options include the use of DBeaver (https://openexchange.intersystems.com/package/DBeaver) or SQLGateway. The first one will be demonstrated in this article, and the second one in an excellent article by Robert Cemper, DB Migration using SQLgateway (https://community.

0
3 897
Article Henrique Dias · Jan 13, 2022 4m read

Hey community! How are you doing?

I hope to find everyone well, and a happy 2022 to all of you!

Over the years, I've been working on a lot of different projects, and I've been able to find a lot of interesting data.

But, most of the time, the dataset that I used to work with was the customer data. When I started to join the contest in the past couple of years, I began to look for specific web datasets.

I've curated a few data by myself, but I was thinking, "This dataset is enough to help others?"

So, discussing the ideas for this contest with @José.

4
0 490
Article Yuri Marx · Mar 7, 2022 4m read

This is the third part of the series of articles on migrating from the main databases on the market to InterSystems IRIS. In this part, the procedures for migrating from DB2 will be detailed.As described in previous articles, there are currently a few options to do the migration. However, the two most popular options include the usage of DBeaver (https://openexchange.intersystems.com/package/DBeaver) or SQLGateway. The first one will be demonstrated in this article, and the second one is presented in an excellent article by Robert Cemper, DB Migration using SQLgateway (https://community.

0
1 682
Article Yuri Marx · Feb 2, 2022 4m read

This article is a follow-up to the previous one on how to migrate from popular databases (like PostgreSQL and MySQL) to IRIS.
We will use the same procedures utilized to migrate from PostgreSQL. However, you will see that it is even easier since the data types in MySQL are very similar to IRIS. That is why we will not need to create transformation rules in the columns.

Get the sample data to the migration process

In GitHub it is possible to download a docker-compose project to build and run 2 databases:

  • Source Database: MySQL database Docker instance with a sample database.
0
0 826
Article Yuri Marx · Jan 25, 2022 4m read

Sometimes it is necessary to transfer or migrate data and data schema from Postgres to IRIS. There are currently a few options for doing this, but the two most popular options are using DBeaver (https://openexchange.intersystems.com/package/DBeaver) or SQLGateway. The first will be demonstrated in this article and the second is presented in an excellent article by Robert Cemper, DB Migration using SQLgateway (https://community.intersystems.

1
1 1185
Announcement Evgeny Shvarov · Dec 28, 2021

Hi Developers!

Here're the technology bonuses for the InterSystems IRIS Datasets Contest 2021 that will give you extra points in the voting:

  • Dataset Usage Demo Repository - 4
  • LOAD DATA Usage - 3
  • Questionnaire  - 2
  • Unique Real Dataset - 4
  • Docker container usage - 2 
  • ZPM Package deployment - 3
  • Online Demo - 2
  • Code Quality pass - 1
  • First Article on Developer Community - 2
  • Second Article On DC - 1
  • Video on YouTube - 3

See the details below.<--break-><--break->

4
0 454
Article Yuri Marx · Jan 5, 2022 2m read

According to the WHO, The top global causes of death, in order of total number of lives lost, are associated with three broad topics (source: https://www.who.int/news-room/fact-sheets/detail/the-top-10-causes-of-death):

  1. Cardiovascular (ischaemic heart disease, stroke),
  2. Respiratory (chronic obstructive pulmonary disease, lower respiratory infections) and
  3. Neonatal conditions – which include birth asphyxia and birth trauma, neonatal sepsis and infections, and preterm birth complications.
2
3 406
Question Oliver Wilms · Jan 8, 2022

I have csv date file with date values like this "4/10/2021" for April 10, 2021. I defined a table with this property: Property TranDate As %Library.Date.

I capture error

[SQLCODE: <-104>:<Field validation failed in INSERT, or value failed to convert in DisplayToLogical or OdbcToLogical>] [Location: <ServerLoop>] [%msg: <Field 'dc_data_finance.transact.TranDate' (value '4/10/2021') failed validation Field ...

I do not really want to change TranDate to %String. How can I import "4/10/2021" into %Date property?

10
0 698
Question Scott Roth · Jan 6, 2022

We are migrating from AIX to Linux and part of our testing is trying to figure the best method to migrate the code. I am trying to export an entire Production, however I keep running into an error...

Error generating export list for production osuwmc.TestClin and all items may not be listed.
ERROR #5002: Cache error: <CLASS DOES NOT EXIST>zgetRecordandComplexMapClasses+34^Ens.Config.Production.1 *(No name)

I went through any Complex Record Maps, and recompiled them but I am still getting the same error

Does anyone know how I can track down this error and the culprit?

Thanks

Scott Roth

3
0 344
Article Yuri Marx · Jan 9, 2022 5m read

 

Hi Community,

In a hard work of data curation and data quality, the Health Dataset application deploy to you the above datasets.

These datasets can be used in your ML applications/models, AutoML and analytics projects. See more details here:

Installation

  1. Clone/git pull the repo into any local directory
$ git clone https://github.com/yurimarx/automl-heart.
0
0 512
Announcement Yuri Marx · Jan 6, 2022

The Health Dataset application now has Six real healthcare datasets (and more in this week!) to you!

  • Diabetes dataset: data to predict diabetes diagnosis
  • Heart Disease dataset: data to predict heart disease
  • Kidney Disease dataset: data to predict kidney disease
  • Breast Cancer dataset: data to predict breast cancer
  • Maternal Risk dataset (NEW): data to predict maternal risk level
  • Hospital Mortality dataset (NEW): data to predict hospital mortality

Get it now: https://openexchange.intersystems.com/package/Health-Dataset

0
0 226
Question Justin Richter · Jan 3, 2022

So I've been reviewing a lot of questions posted in the InterSystems community regarding NULL properties in JSON. I've also been reviewing the JSON documentation. None of these things have been able to help me so far.

1. We don't seem to have the %JSON.Adaptor class available for us to use in our system.

2. I'm not really confident enough to create JSON Type classes or backporting code, etc.

I created a dummy class and I need my JSON to look like this:
{
    "notanumber":"28001",
    "aboolean":true,
    "anumber":12345,
    "adecimal":1.

2
0 861
Announcement Anastasia Dyubaylo · Dec 21, 2021

Hi Community,

We are pleased to invite all the developers to the upcoming InterSystems Datasets Contest Kick-off Webinar! The topic of this webinar is dedicated to the Datasets contest.

In this webinar, we’ll do a quick tour of the new LOAD DATA feature, also chime in on packaging global data or file data with ZPM, and run a data-generation script as part of a method in the zpm install. 

As always, our experts will answer the questions on how to develop, build, and deploy datasets using InterSystems IRIS.

Date & Time: Tuesday, December 28 – 10:00 AM EDT

Speakers:  
🗣 @Benjamin De Boe, Product Manager, InterSystems  
🗣 @Evgeny Shvarov, InterSystems Developer Ecosystem Manager

3
0 464
Announcement Anastasia Dyubaylo · Oct 1, 2021

Hi Community,

Join the upcoming InterSystems Interoperability Contest Kick-off Webinar dedicated to the Interoperability Contest.

In this webinar, you'll get an overview of the interoperability capabilities of InterSystems IRIS, and we'll show you how to use the PEX. Also, we’ll discuss and answer your questions on how to build interoperability solutions using InterSystems IRIS and IRIS for Health.

Date & Time: Monday, October 4 — 12:00 AM EDT

Speakers:  
🗣 @Stefan Wittmann, InterSystems Product Manager 
🗣 @Bob Kuszewski, InterSystems Product Manager 
🗣 @Evgeny Shvarov, InterSystems Developer Ecosystem Manager


5
0 422
Question Bharathsimha Reddy Jakka · Sep 27, 2021

How to find a global's original namespace ? Potentially mapped from a different namespace .

I have a global ^Custom that exists in multiple namespaces but it could mapped from namespace Drone(A) to Launch(B)

Without access to Cache management portal how to find where is my global located using cache code ?

Like if ^Custom == ^[Drone]Custom ??

5
0 492
Question James Hipp · Sep 14, 2021

https://docs.intersystems.com/irislatest/csp/documatic/%25CSP.Documatic…

method Export(savertn As %Boolean = 0) [ Language = objectscript ]

I am looking to export table to a text file in a programmatic way. I am not familiar with this class and how to invoke a "savertn" as input here.

Essentially I would like to Export Table (SELECT * FROM Table) and save it as a text file locally on Linux server. Is there any examples out there of that? Perhaps I need to save the query output to a global and then pass that input to the export?

2
0 541
Question James Casazza · Aug 23, 2021

After linking in Oracle Table with Field Column's Data Types of NUMBER, my updates into these table fields are resulting in data that is being rounded to 2 decimal places. I insert a record with 1234.1234 and 1234.12 is stored.

It appears Cache xDBC might be manipulating my values prior to sending to Oracle. Is there a setting or system parameter that is controlling this? If so, is there a way to relax this from occurring so the values I send are being stored in Oracle with the same values?

10
0 837
Announcement Ben Spead · Jul 7, 2021

Hello Developers!

Have you ever had to convert HL7v2 messages to FHIR (Fast Healthcare Interoperability Resources) and found the process complicated and confusing? InterSystems is rolling out a new cloud based SaaS offering called InterSystems FHIR Transformation Service, which makes the process easy.  We are excited to announce an Early Access Preview Program for our new offering, and we would love to have you kick the tires and let us know what you think!  All you need is a free AWS account, with an S3 bucket to drop in your HL7v2 messages, and another S3 bucket to get your FHIR output. 

5
0 951
Question Stella Ticker · May 26, 2021

I am trying to populate a table using the sql Data Import Wizard. The input file is a tab delimited text file. But the import keeps failing with a 104 error showing validation for the columns which use %Library.TimeStamp and %Boolean datatypes is failing. Yet when I insert values into the table through a SQL insert command, the values get saved correctly in the table.

For the TimeStamp format in the wizard form, I am choosing YYYY-MM-DD-HH:MI:SS because  there was no option for this format: YYYY-MM-DD HH:MM:SS.

4
0 452