Data warehouses and data lakes with data from IRIS

Discussion

Dmitry Maslennikov · Sep 1, 2023

#Big Data #SQL #InterSystems Ideas Portal #InterSystems IRIS

In today's landscape, enterprises have grown substantially in scale, amassing vast amounts of data. This data is collected from a plethora of sources including different applications, databases, and other channels. Given the diversity and volume of this data, it's only logical for these enterprises to seek a deeper understanding of what their data entails. Some of the data can be stored in IRIS, and it can be reasonable to be able to add this data to a data lake too.

The Internet now offers many different tools for such tasks, that do not yet support IRIS, but it's achievable.

What do you think about it, probably someone already needs something like that, or is still in search.

Discussion (7)6

Vadim Aniskin · Sep 1, 2023

@Dmitry Maslennikov
Thank you for creating this discussion and ideas on the Ideas Portal dedicated to this topic:

Implement support to Fivetran, data movement platform

Implement IRIS connector for Airbyte data

Developers, please vote for these ideas if you support them.

3 0

Dmitry Maslennikov · Sep 4, 2023

Obviously, I can implement those connectors, just wanted to get some feedback, or other suggestions or what should be implemented first

0 0

Ashok Kumar · Sep 2, 2023

You're absolutely right.It's crucial, We really need to build it for IRIS.

0 0

Luis Angel Pére... · Sep 2, 2023

Well, you hit the nail @Dmitry Maslennikov .

Here in Spain a public contest for the public health service of Asturias was published 1 month ago for a data lake and we missed the opportunity to participate.

0 0

Dmitry Maslennikov · Sep 2, 2023

So, we have time, to be prepared for the next time

1 0

Evgeny Shvarov · Sep 3, 2023

Great idea! Thank you @Dmitry Maslennikov

0 0

Raj Singh · Sep 12, 2023

Great discussion topic, @Dmitry Maslennikov. Having the IRIS Data Platform play a role in all kinds of data warehouse/lake architectures is certainly important. Especially since these architectures have been evolving recently to take advantage of the convergence of systems of record and systems of reference -- i.e. transactional and analytical databases. In short, one can easily make a strong argument that the IRIS Data Platform can be the foundation for most operations on data -- transactions, analytics and machine learning. On top of that, I see a strong opportunity for our developer community to create tools that offer better DataOps capabilities, like orchestrating data transformations with data pipelines, creating data catalogs, and augmenting all these with machine learning tools to automate much of this data engineering work.

0 0