#CSV

1

A comma-separated values (CSV) file is a delimited text file that uses a comma to separate values. Each line of the file is a data record. Each record consists of one or more fields, separated by commas. The use of the comma as a field separator is the source of the name for this file format. A CSV file typically stores tabular data (numbers and text) in plain text, in which case each line will have the same number of fields.

Hello community.

I have a request from a client who wants to do the following.
Currently, they have a data upload process via FTP in which they have implemented a RecordMap to store the information of the CSV files that are uploaded to the FTP directory.
Now they want to have another directory so that it is ONLY executed through an external instruction, not when the file is uploaded to the directory.

0 3
0 46

Hi folks!

It is very easy to import CSV data into IRIS. But what if we want to preserve the original IDs in CSV?

Recently I came across with the situation when I needed to import two csv's into IRIS which were linked by one column referencing to another csv's col: a typical Foreign Key and Primary Key situation, where csv1 contains this column as Primary Key, and csv2 as Foreign key with id's related to csv1.

The image is generated by ChatGPT so don't blame it - it tried its best to generate countries as primary keys with countries.csv-cities.csv relationship :)

Open Exchange app
3 0
1 52

In recent versions of IRIS, a powerful new data loading command has been introduced to SQL: LOAD DATA. This feature has been highly optimized to import data into IRIS extremely fast, allowing hundreds of gigabytes of data to be inserted in seconds instead of hours or days.

This is a very exciting improvement. However, a big problem in the data loading experience still exists. Namely, the time and hassle it takes to:

Open Exchange app
11 7
1 952
Article
· Jul 31 1m read
Import CSV into CACHÉ

Here's a practical example of how to import data from a CSV file into InterSystems CACHÉ using ObjectScript
Assuming your CSV file is simple (e.g., comma-separated, with headers), you can use %Stream.FileCharacter to read it line by line and parse the data.

ClassMethod ImportCSV(filePath As %String) As %Status {
Set stream = ##class(%Stream.FileCharacter).%New()
Set sc = stream.LinkToFile(filePath)
If 'sc Quit sc

2 0
0 50


Hi Community,

In this article, I will introduce my application iris-fhir-bridge

IRIS-FHIR-Bridge is a robust interoperability engine built on InterSystems IRIS for Health, designed to transform healthcare data across multiple formats into FHIR and vice versa. It leverages the InterSystems FHIR Object Model (HS.FHIRModel.R4.*) to enable smooth data standardization and exchange across modern and legacy healthcare systems.

Open Exchange app
4 2
1 117
Question
· Feb 10
CRLF to LF

For example we have a docker app that receives a .csv file from windows environment and then do some processing on it in docker linux environment.



Windows encode new line in CRLF format, but linux can understand only LF and throw an unexpected errors like

0 4
0 122

We have a yummy dataset with recipes written by multiple Reddit users, however most of the information is free text as the title or description of a post. Let's find out how we can very easily load the dataset, extract some features and analyze it using features from OpenAI large language model within Embedded Python and the Langchain framework.

10 3
2 390

We have JSON type data in a Dynamic Object. Is there a simple way to export / dump that data to a delimited string or file?

e.g.

Results={"ClassA":{"ClassName":"ClassA","ACount":367191880,"BCount":367191880,"CurrentDiff":0,"PreviousDiff":0,"ReportDate":"2024-03-02 00:00:00"}
"ClassB":{"ClassName":"ClassB","ACount":5352149227,"BCount":5352149227,"CurrentDiff":0,"PreviousDiff":0,"ReportDate":"2024-03-02 00:00:00"}}

0 6
0 427

Hi folks!

Have a question for those who are masters of interoperability.

I have a basic task of having one CSV with some data. I need to transform one column in the initial dataset and get the new csv with the same form.

What's the best approach with Interoperability?

Should I user record mapper?

Should I use streams, objects?

What is the best practice?

0 12
0 715

The last days I've work with the great new feature: LOAD DATA With this post I would like to share my first experiences with you. The following points do not contain any order or other evaluation. These are only things that I noticed when using the LOAD DATA command. It should also be noted that these points are based on the IRIS Version 2021.2.0.617 which is a preview release. So it may be that my observations do not apply to newer IRIS versions.

Open Exchange app
6 5
2 1.2K

In this GitHub we gather information from a csv, use a DataTransformation to make it into a FHIR object and then, save that information to a FHIR server all that using only Python.

The objective is to show how easy it is to manipulate data into the output we want, here a FHIR Bundle, in the IRIS full Python framework.

Open Exchange app
3 3
0 594

Hi Developers!

Sometimes we need to import CSV data programmatically to InterSystems IRIS either from CSV or from URL. And we expect the class with proper datatypes to be created and the data to be imported.

I published a module csvgen on Open Exchange which does exactly that.

If you just need the CSV file be imported into IRIS you can do the following:

USER>do ##class(community.csvgen).Generate("/usr/data/titanic.csv",,"Data.Titanic")

Class name: Data.Titanic
Header: PassengerId INTEGER,Survived INTEGER,Pclass INTEGER,Name VARCHAR(250),Sex VARCHAR(250),Age INTEGER,SibSp INTEGER,Parch INTEGER,Ticket VARCHAR(250),Fare MONEY,Cabin VARCHAR(250),Embarked VARCHAR(250)
Records imported: 891
USER>

Or if you have the CSV on the internet, e.g. COVID-19 Data on Github you can get the data in the following way:

USER>d ##class(community.csvgen).GenerateFromURL("https://raw.githubusercontent.com/CSSEGISandData/COVID-19/master/csse_covid_19_data/csse_covid_19_daily_reports/05-29-2020.csv",",","Data.Covid19")

Class name: Data.Covid19
Header: FIPS INTEGER,Admin2 VARCHAR(250),Province_State VARCHAR(250),Country_Region VARCHAR(250),Last_Update DATE,Lat MONEY,Long_ DOUBLE,Confirmed INTEGER,Deaths INTEGER,Recovered INTEGER,Active INTEGER,Combined_Key VARCHAR(250),Incidence_Rate DOUBLE,Case-Fatality_Ratio DOUBLE
Records imported: 3522
USER>

Open Exchange app
7 16
1 1.4K

Hi folks!

Sometimes we need to import data into InterSystems IRIS from CSV. It can be done e.g. via csvgen tool that generates a class and imports all the data into it.

But what if you already have your own class and want to import data from CSV into your existing table?

There are numerous ways to do that but you can use csvgen (or csvgen-ui) again! I prepared and and example and happy to share. Here we go!

8 2
0 1.2K

I'm trying to create a new Record Map but I keep getting a ZEN Exception error. I actually go this to work the very first time I did it and have a record map. But I need to do more.

I'm working on my local machine so I can grab a CSV (or txt in this case as it's pipe delimited) from the local directory. I pick the file I want to use but when I click OK i get a popup and the following error:

0 1
0 393

I work as an Integration Engineer for United States Department of Veterans Affairs (VA). I work on a Health Connect production which processes many RecordMap files. I do not fully understand RecordMaps and I wanted to develop an application for the Interoperability contest where I could learn more about working with RecordMaps. I browsed InterSystems documentation for inspiration on how to start. I was happy to find CSV Record Wizard.

Open Exchange app
0 0
0 457

Hello

I made for fun a bash script that creates csv files from a %GOGEN exported file.
%GOGEN, the utility for exporting globals for Cache 5.xx

First I thought it would be very simply with bash, but ...
- if I want a header in the csv file
- if you export with GOGEN more globals together not only one
- if I want a csv file for every global/indexes has the global ..

The simply bash script goes up 100 lines of code. If I will start again with this, I would use Python or Go, not bash.

Example:

For a EXPORTED.GLO file

0 5
0 774