HELP: get and prase Caché databases logs

Question

Question

liu yanjie · Jan 7

Hi all,
Does anyone have experience with obtaining logs from Caché databases and parsing them now? Please leave me a message.

I ask because my project involves some hospitals that used Caché when creating databases over 10 years ago. Currently, we cannot replace the database, but we need to check and parse the database logs from the Caché.

Thanks.

Discussion (12)2

Log in or sign up to continue

score 0 · Answer 1 · 2024-01-08T00:50:19-05:00

Dmitry Maslennikov · Jan 8

What do you mean by parsing logs, what do you expect to extract from it?

0 0

score 0 · Answer 2 · 2024-01-08T03:51:13-05:00

Hi there, thanks for your quick reply.

We are currently developing a Data Center to collect business data from hospital HIS systems. I realized that some hospitals use Caché databases. But we used Flickcdc to collect data which do not support Caché databases, such as Oracle and MySQL.

Is there a tool available to collect real-time? I mean, such as ogg provided by Oracle.

Many thanks.

score 0 · Answer 3 · 2024-01-08T06:37:59-05:00

There are no such tools at the moment, or anything similar to Oracle GoldenGate.

I suppose you meant Apache Flink, https://github.com/ververica/flink-cdc-connectors

Yeah, there is no support for Caché/IRIS. But it can be implemented, I already have experience with it

But, such an old system may not have adoption to SQL, and would require some additional work on the Caché side.

You may try using JDBC connection, to check if the data available through SQL. Then it could worth to implement flink CDC connector for Caché/IRIS.

Just in case you can use other options, I've already adapted Trino and Presto

score 0 · Answer 4 · 2024-01-08T21:03:49-05:00

What is the additional work you mentioned doing on the Caché side? I hope to be able to monitor and collect data in real-time when adding or modifying data, rather than just querying it.

score 0 · Answer 5 · 2024-01-09T00:15:48-05:00

There were multiple ways to build applications back then

Using globals to store data, no this way can be called NoSQL, this case will require additional work on the server side
and using persistent classes, which can be accessed via SQL

So, depends on how old this application is, and it has not updated for a long time

If you already know for certain, that the data you need can be accessed through SQL, then it's possible to make Flink working

score 0 · Answer 6 · 2024-01-09T01:42:58-05:00

liu yanjie · Jan 9

Is there a detailed method there？ Sorry we are not very familiar with cache databases.

0 0

score 0 · Answer 7 · 2024-01-09T02:02:25-05:00

Dmitry Maslennikov · Jan 9

You can try using DBeaver, connect to Caché, and see if there are any tables you interested

0 0

score 0 · Answer 8 · 2024-01-09T02:21:16-05:00

liu yanjie · Jan 9

Yes, it can connect to Caché, but I'd like to monitor the real-time datas and place them in Kafka.

0 0

score 0 · Answer 9 · 2024-01-08T04:04:26-05:00

Luis Angel Pére... · Jan 8

Have you checked Open Exchange?

https://openexchange.intersystems.com/?search=cache%20log&sort=r

0 0

score 0 · Answer 10 · 2024-01-08T04:27:02-05:00

Yes, thank you. But I'm not sure if there a tool that support Caché databases and can collect data in real-time, such as the ogg provided by Oracle.

score 0 · Answer 11 · 2024-01-08T21:12:38-05:00

Hi, I have checked the link, but it doesn't seem to be available.

I'd like to collect tables based on logs, parse the logs and place them in Kafka.

score 0 · Answer 12 · 2024-01-09T02:43:27-05:00

I'd like to collect tables based on logs

In general, such approach is far from optimal for Caché like databases because logs (which are usually called "journals" in Caché/M world) are being written on global (= lowest possible) level, disregard of data model used by an app (SQL, Persistent classes, etc). Reconstructing app level data from underlying globals can be a tricky task even for Caché guru.

That was one of the reasons why colleagues of mine took another approach for close to real-time data export from Caché to external system. In a few words, they used triggers to form application level data packets on Caché side and pipe them to the receiver. This approach saved CPU time preventing its waste for filtering out unnecessary journal records and minimized cross-system network traffic.