#Journaling

Working in support, I usually get asked how many days I should keep journals. Should it be two days or after two backups? More? Less? Why two?

The correct answer (for most of the environments) is that you should keep the journals since the last validated Backup. I.e., until you don't check if a Backup is valid (restoring the file and checking with the Integrity utility), you can't be sure there is a good copy of your data and can't purge the journals safely.

Following my previous post, some urged me to get to the point – ok, so I found my "star" journaling globals, the ones that take up the most space – but how do I avoid this? How do I minimize the journal's size?

[DISCLAIMER: Some might still be disappointed after this post as well but wait till the next one... ]

Hi community!

In this article, I'm excited to introduce my latest application, Journal File Indexer. The development of this application is rooted in the portal idea DPI-I-270.

In short, this application lets you load and index a log file in a database.

Why?

If you've ever used the log file search function in the management portal, you may have encountered a timeout error or even a blank page. This problem usually occurs when searching a large journal file. Journal File Indexer solves this problem by considerably increasing the speed of searches once the file has been loaded into the database.

Another problem arises during the restoration process. When searching for global entries in a log file to restore old or new values, the management portal doesn't have this specific functionality. A routine must therefore be coded to achieve this. Journal File Indexer comes to the rescue, incorporating a restore function!

Recently I needed to restore a version of a production class, which was overwritten by compilation and running UpdateProduction. As the correct version was unavailable in the source control, I used journals to restore the data. Journals store a plethora of information about what's happening in the system and are quite a powerful tool. This article explains how to work with journals to extract the data you require.

I am still working on a generic task where I need to apply journal file records to another database. Initially I didn't want to use Journal.Restore class methods as I need to perform some data transformation, and it seemed that the clearest way to achieve it was to read journal file record by record using %SYS.Journal.Record API.

In this article I'd like to share with you a phenomena that is best you avoid - something you should be aware of when designing your data model (or building your Business Processes) in Caché or in Ensemble (or older HealthShare Health Connect Ensemble-based versions).

I need to turn off Journalling for a particular database programmatically.

How can I make it happen?

My fantasy didn't go beyond periodically running %SYS.Journal.System:Progress() class query, while I'd prefer to use an event handler of some kind. Any ideas?

Hi!

Currently, I'm working on a script using %SYS.Journal API.

My journal files are compressed, following the documentation the compression is zstd.

I tried to unzip a file with zstd tool, ex:

apt-get update
apt-get install zstd

zstd -d 20230806.001z -o 20230806.001

20230806.001z        : 92.03%   (909312 => 836859 bytes, 20230806.001) 

Unfortunately, the unzipped file is invalid:

Write ##class(%SYS.Journal.File).IsValidJournal("/usr/irissys/mgr/journal/20230806.001")

I am new to Cache, we are trying to move a 4 TB database over the internet, but this will be take too long to copy the single backup file. In Oracle and MS-SQL Server there is an option of doing log shipping. Is there such an option in Cache?

I want to copy the initial backup file one weekend, then keep sending the Journals (logs) the new location.

Thank you,

Eudoro

I'm working on a task where I need to apply journal file records to another database. I can't use Journal.Restore class methods as I need to perform some data transformation, therefore I'm reading journal file record by record using %SYS.Journal.Record API.

It seems that there are only few journal records that I need to process, namely:

Hello, community!

I've stumbled on some unexpected behavior, and decided to check with you if this is normal. Basically, I'm rebuilding indices and the result is not journaling (which leads to missing indices at shadow server).

The $ZV is "Cache for UNIX (Red Hat Enterprise Linux for x86-64) 2015.2.1 (Build 705U) Mon Aug 31 2015 16:53:38 EDT"

I have an example class

Class tmp.A As %Persistent;

Index IP1 On P1;

Property P1 As %String;

for example there is one object which have P1 = 1, so

The steps are as follows：

1. create global

For I=1:1:200 Set ^ABC(I)=""
For I=1:1:200 Set ^XYZ(I)=""
For I=1:1:100 Kill ^ABC(I)

2. create ZJRNFILT

ZJRNFILT(jid,dir,glo,type,restmode,addr,time) /*Filter*/
Set restmode=1 /*Return 1 for restore*/
If glo["^ABC",type="K" Set restmode=0 /*except if a kill on ^ABC*/
Quit
;

3. restore

In the Windows Ressource Manager I can observe multiple parallel processes coming from cache.exe with read operations to journaling files.

All except one of these processes have the same reads(Byte/s). The processes point to different journal files and constantly read between 200 and 3000 Bytes/s.

The corresponding process via PID in the management portal of Caché shows the process %SYS.Monitor.Control.1. In 3 days of uptime on the server it has run 181.632.583 commands and modified 32.140.642 globals.