What is Journaling?

Journaling is a critical IRIS feature and a part of what makes IRIS a reliable database. While journaling is fundamental to IRIS, there are nuances, so I wrote this article to summarize (more briefly than our documentation which has all the details) what you need to know. I realize the irony of saying the 27 minute read is brief.

17 3
9 1.1K

Working in support, I usually get asked how many days I should keep journals. Should it be two days or after two backups? More? Less? Why two?

The correct answer (for most of the environments) is that you should keep the journals since the last validated Backup. I.e., until you don't check if a Backup is valid (restoring the file and checking with the Integrity utility), you can't be sure there is a good copy of your data and can't purge the journals safely.

13 7
2 777
Article
· Nov 24, 2023 4m read
Journal File Indexer

Hi community!

In this article, I'm excited to introduce my latest application, Journal File Indexer. The development of this application is rooted in the portal idea DPI-I-270.

In short, this application lets you load and index a log file in a database.

Why?

If you've ever used the log file search function in the management portal, you may have encountered a timeout error or even a blank page. This problem usually occurs when searching a large journal file. Journal File Indexer solves this problem by considerably increasing the speed of searches once the file has been loaded into the database.

Another problem arises during the restoration process. When searching for global entries in a log file to restore old or new values, the management portal doesn't have this specific functionality. A routine must therefore be coded to achieve this. Journal File Indexer comes to the rescue, incorporating a restore function!

6 3
1 312

Recently I needed to restore a version of a production class, which was overwritten by compilation and running UpdateProduction. As the correct version was unavailable in the source control, I used journals to restore the data. Journals store a plethora of information about what's happening in the system and are quite a powerful tool. This article explains how to work with journals to extract the data you require.

5 0
1 255

InterSystems FAQ rubric

If the system does not stop for 24 hours, old journal files will be deleted at 0:30 according to the "Journal file deletion settings".

A possible cause of journal files remaining that are older than the "Journal file deletion settings" is that there are transactions that remain open.

In that case, you will be able to delete the journal file by searching for processes executing transactions and finalizing the transactions.

5 3
2 269

In this article I'd like to share with you a phenomena that is best you avoid - something you should be aware of when designing your data model (or building your Business Processes) in Caché or in Ensemble (or older HealthShare Health Connect Ensemble-based versions).

3 2
0 401

InterSystems has corrected a defect that causes invalid database and journal records to be introduced when using a specific $LIST syntax. The likelihood of encountering this defect is very low but the operational impacts can be significant.

2 0
0 98

Hi,

We have Mirroring established between NODE 1 & Node 2 . We have set the "cachesys" database enabled for Journalling. But we dont see the User Accounts , Roles, Resources created on Node 1 ( favoured Primary) reflected on Node 2 . Is creating them manually again is the only option for this ? . Is there any way to sync them or would adding %SYS to MIRROR a possible solution. Would it be great if anyone has faced this as we have an issue that during failovers Team is locked out .

Best Regards,

Arun Madhan

1 10
1 896

Hi!

Currently, I'm working on a script using %SYS.Journal API.

My journal files are compressed, following the documentation the compression is zstd.

I tried to unzip a file with zstd tool, ex:

apt-get update
apt-get install zstd

zstd -d 20230806.001z -o 20230806.001

20230806.001z        : 92.03%   (909312 => 836859 bytes, 20230806.001) 

Unfortunately, the unzipped file is invalid:

Write ##class(%SYS.Journal.File).IsValidJournal("/usr/irissys/mgr/journal/20230806.001")

1 2
0 356

I'm purging a lot of management data from an Ensemble production, which is creating 100s of GBs of journals. Has anybody succeeded in disabling journaling on an Ensemble purge? The user interface doesn't have an option for this, but I'm thinking you might be able to identify the process and externally disable journaling on it.

0 3
0 934

Hi,

We are using CACHE 2017.2.1, I would like to retrieve data from Journal for killed global. Let say we have global name ^ EMP(123) with data and also have some child nodes and it has been killed by using cache kill command for some reason and we don't know who has executed this and when. My questions are below.

1) Can we get back the data of killed global from journal files,Is it possible or not ?

0 5
0 663

Currently, we are running 2014.1 on two different servers (OpenVMS, RHEL). The plan is to transition from OpenVMS to RHEL, but our Write Daemon is in a Troubled state on both servers.

On the OpenVMS server, we have a WIJ file that's 26G and can grow to 40G (size of database cache). Since it hasn't grown to 40G, we don't believe the size of the WIJ file to be the issue.

What else should we be looking at regarding the performance of the Write Daemon?

0 1
0 956

Hey everyone.

I have noticed that my backup mirror is warning that the MirrorDatabaseLatencyTime is having a bad time (time in ms is 3000, and warnvalue is 3000). While I look into what may be causing this latency between the two servers, I was considering if reducing the size of the journal files would improve this value in any way.

My assumption is that reducing the file size would mean that the frequency of the journal files being created would be increased, but the reduced size would mean that the transfer and application of each file would be reduced.

0 2
0 760

In System Administration | Configuration | System Configuration | Journal Settings there is a check box for Freeze on error.

From reading the documentation, it sounds like the choice to freeze on error is one of system availability vs system integrity.

Curious to know whether Ensemble users choose to freeze on error or not.

If you have a best practice recommendation based upon your experience or knowledge, that would be helpful information too.

0 3
0 360
Question
· Apr 23, 2021
Log (Journal) shipping

I am new to Cache, we are trying to move a 4 TB database over the internet, but this will be take too long to copy the single backup file. In Oracle and MS-SQL Server there is an option of doing log shipping. Is there such an option in Cache?

I want to copy the initial backup file one weekend, then keep sending the Journals (logs) the new location.

Thank you,

Eudoro

0 5
0 316

Hello, community!

I've stumbled on some unexpected behavior, and decided to check with you if this is normal. Basically, I'm rebuilding indices and the result is not journaling (which leads to missing indices at shadow server).

The $ZV is "Cache for UNIX (Red Hat Enterprise Linux for x86-64) 2015.2.1 (Build 705U) Mon Aug 31 2015 16:53:38 EDT"

I have an example class

Class tmp.A As %Persistent;

Index IP1 On P1;

Property P1 As %String;

for example there is one object which have P1 = 1, so

0 1
0 304