In recent versions of IRIS, a powerful new data loading command has been introduced to SQL: LOAD DATA. This feature has been highly optimized to import data into IRIS extremely fast, allowing hundreds of gigabytes of data to be inserted in seconds instead of hours or days.

This is a very exciting improvement. However, a big problem in the data loading experience still exists. Namely, the time and hassle it takes to:

10 6
0 507

We have a yummy dataset with recipes written by multiple Reddit users, however most of the information is free text as the title or description of a post. Let's find out how we can very easily load the dataset, extract some features and analyze it using features from OpenAI large language model within Embedded Python and the Langchain framework.

10 3
2 312

We have JSON type data in a Dynamic Object. Is there a simple way to export / dump that data to a delimited string or file?

e.g.

Results={"ClassA":{"ClassName":"ClassA","ACount":367191880,"BCount":367191880,"CurrentDiff":0,"PreviousDiff":0,"ReportDate":"2024-03-02 00:00:00"}
"ClassB":{"ClassName":"ClassB","ACount":5352149227,"BCount":5352149227,"CurrentDiff":0,"PreviousDiff":0,"ReportDate":"2024-03-02 00:00:00"}}

0 6
0 316

Hi folks!

Have a question for those who are masters of interoperability.

I have a basic task of having one CSV with some data. I need to transform one column in the initial dataset and get the new csv with the same form.

What's the best approach with Interoperability?

Should I user record mapper?

Should I use streams, objects?

What is the best practice?

0 12
0 623

The last days I've work with the great new feature: LOAD DATA With this post I would like to share my first experiences with you. The following points do not contain any order or other evaluation. These are only things that I noticed when using the LOAD DATA command. It should also be noted that these points are based on the IRIS Version 2021.2.0.617 which is a preview release. So it may be that my observations do not apply to newer IRIS versions.

6 5
2 1.1K

In this GitHub we gather information from a csv, use a DataTransformation to make it into a FHIR object and then, save that information to a FHIR server all that using only Python.

The objective is to show how easy it is to manipulate data into the output we want, here a FHIR Bundle, in the IRIS full Python framework.

3 3
0 515

Hi Developers!

Sometimes we need to import CSV data programmatically to InterSystems IRIS either from CSV or from URL. And we expect the class with proper datatypes to be created and the data to be imported.

I published a module csvgen on Open Exchange which does exactly that.

If you just need the CSV file be imported into IRIS you can do the following:

USER>do ##class(community.csvgen).Generate("/usr/data/titanic.csv",,"Data.Titanic")

Class name: Data.Titanic
Header: PassengerId INTEGER,Survived INTEGER,Pclass INTEGER,Name VARCHAR(250),Sex VARCHAR(250),Age INTEGER,SibSp INTEGER,Parch INTEGER,Ticket VARCHAR(250),Fare MONEY,Cabin VARCHAR(250),Embarked VARCHAR(250)
Records imported: 891
USER>

Or if you have the CSV on the internet, e.g. COVID-19 Data on Github you can get the data in the following way:

USER>d ##class(community.csvgen).GenerateFromURL("https://raw.githubusercontent.com/CSSEGISandData/COVID-19/master/csse_covid_19_data/csse_covid_19_daily_reports/05-29-2020.csv",",","Data.Covid19")

Class name: Data.Covid19
Header: FIPS INTEGER,Admin2 VARCHAR(250),Province_State VARCHAR(250),Country_Region VARCHAR(250),Last_Update DATE,Lat MONEY,Long_ DOUBLE,Confirmed INTEGER,Deaths INTEGER,Recovered INTEGER,Active INTEGER,Combined_Key VARCHAR(250),Incidence_Rate DOUBLE,Case-Fatality_Ratio DOUBLE
Records imported: 3522
USER>

7 16
1 1.3K

Hi folks!

Sometimes we need to import data into InterSystems IRIS from CSV. It can be done e.g. via csvgen tool that generates a class and imports all the data into it.

But what if you already have your own class and want to import data from CSV into your existing table?

There are numerous ways to do that but you can use csvgen (or csvgen-ui) again! I prepared and and example and happy to share. Here we go!

8 2
0 1.1K

I'm trying to create a new Record Map but I keep getting a ZEN Exception error. I actually go this to work the very first time I did it and have a record map. But I need to do more.

I'm working on my local machine so I can grab a CSV (or txt in this case as it's pipe delimited) from the local directory. I pick the file I want to use but when I click OK i get a popup and the following error:

0 1
0 354

I work as an Integration Engineer for United States Department of Veterans Affairs (VA). I work on a Health Connect production which processes many RecordMap files. I do not fully understand RecordMaps and I wanted to develop an application for the Interoperability contest where I could learn more about working with RecordMaps. I browsed InterSystems documentation for inspiration on how to start. I was happy to find CSV Record Wizard.

0 0
0 400

Hello

I made for fun a bash script that creates csv files from a %GOGEN exported file.
%GOGEN, the utility for exporting globals for Cache 5.xx

First I thought it would be very simply with bash, but ...
- if I want a header in the csv file
- if you export with GOGEN more globals together not only one
- if I want a csv file for every global/indexes has the global ..

The simply bash script goes up 100 lines of code. If I will start again with this, I would use Python or Go, not bash.

Example:

For a EXPORTED.GLO file

0 5
0 701
Announcement
· Oct 2, 2020
Csvgen-UI

Introduction

This is a csvgen UI frontend in Angular 8.
The aim of this project is to easily import csv file to Iris from a web ui.

Demo

Demo

Install with ZPM

It will automatically install the dependency of csvgen and sslclient

zpm "install csvgen-ui"

This will create two new endpoints :

  • One for the Rest API for uploading files

4 2
0 348

Hey Developers,

Enjoy watching this brief explanation of a new feature in IRIS RAD Studio – a new project by @José Pereira and @Henrique.GonçalvesDias:

Automated user management in IRIS RAD Studio

https://www.youtube.com/embed/-Y2kaKES8yg
[This is an embedded link, but you cannot view embedded content directly on the site because you have declined the cookies necessary to access it. To view embedded content, you would need to accept all cookies in your Cookies Settings]

6 0
0 289

Hi colleagues!

Every day Johns Hopkins University publishes new data on coronavirus COVID-19 pandemic status.

I built a simple InterSystems IRIS Analytics dashboard using InterSystems IRIS Community Edition in docker deployed on GCP Kubernetes which shows key measures of the disease outbreak.

This dashboard is an example of how information from CSV could be analyzed with IRIS Analytics and deployed to GCP Kubernetes in a form of InterSystems IRIS Community Edition.

Added the interactive map of the USA:

19 13
3 999
Question
· Jan 18, 2021
ERROR #5012:

Hello!

I have created a record map that I am going to use to parse a flat file coming into Ensemble. From there I am trying to send the parsed out data as emails to patients. I am getting this error when the Service tries to locate the file and send it through...it finds the file (as it states in the error message). So why does it say it does not exist?

0 2
0 377