This topic unites posts which describe approaches, tools, and solutions to import and export data into InterSystems IRIS and other InterSystems Data platform products: CSV, JSON, SQL, flat files, binary files, globals, streams, etc.
when doing ETL process from cache database to mysql database(ods) and to another mysql database(dwd).
There are lots of DR columns in one table.
In the first step we need two databases to be totally consistent(ods=cache database)
However, the DR column in another database lost its meaning and become to normal number.
In the last step we do the data governance in ODS to DWD. My question is how to write sql without many `join` in this step.
I am exporting data from Cache using the SQL export wizard. This is on a docker image of Cache hosted on a Linux server.
I select my database, schema, and table, make sure that all columns are being exported, check other settings, and then finish the export.
All server settings look correct and permissions are rwe on the directory. I have tried various directories, including a system level directory. I tried creating the file before I do the export in case there was an issue with creating the files, and I also made sure the created files had rwe permisisons.
In terms of general through-put design and long term support, I'm considering what would be a "best approach" for needing to create multiple batch files in a few different layouts from the same data-sets.
I need to iterate over a few very large tables to generate consistent batch outputs for different partner applications that need to be working on the same "version" of the source data.