What is the simplest/quickest way to detect that Journaling was switched to a new file?

Question

Question

Alexey Maslov · Aug 7, 2019

My fantasy didn't go beyond periodically running %SYS.Journal.System:Progress() class query, while I'd prefer to use an event handler of some kind. Any ideas?

Discussion (13)4

Log in or sign up to continue

Alexey Maslov · Sep 16, 2019

Here is my solution. A couple of words as a preface. There are two tasks:
#1

Switches journal and fixes the name of new journal file (e.g., in @..#GtrlJ@("JrnFirst")).
Processes the globals of a namespace. The algorithm of processing doesn't matter here, it's usually some kind of data re-coding.

#2. This task occurred just because users' activity during the task #1 execution can introduce the changes in globals already processed by the task #1.

Wait for the next journal file available for processing (WaitForJrnSwitch());
Process the globals found in this journal using the algorithm similar to the task_#1's one.

The latter is a pseudo-code of WaitForJrnSwitch() method and GetJrnID(), its helper.

 /// If new jrn is available, set %JrnID=Jrn ID and return 1;
/// waiting by ..#TimeWait steps till ..#TimeLimit
ClassMethod WaitForJrnSwitch() As %Boolean
{
 set rc=0
 set nTimes = ..#TimeLimit \ ..#TimeWait
 for i=1:1:nTimes {
  $$$TOE(sc, ..GetJrnID(.JrnID)) // current journal
  if %JrnID="" {
    set JrnNext=@..#GtrlJ@("JrnFirst")
  } else {
    set JrnNext=%JrnID+1
  }
  if JrnNext<JrnID { // avoid extra journal switching ("by restore")
    set %JrnID=JrnNext
    set rc=1
    quit
  }  
  hang ..#TimeWait
 }
 quit rc
}


/// Get Jrn ID of the current journal file
/// Out:
/// returns %Status;
/// pJrnID - journal file name w/o prefix and "."
ClassMethod GetJrnID(Output pJrnID) As %Status
{
 set sc=1
 try {
   set file=##class(%File).GetFilename(##class(%SYS.Journal.System).GetCurrentFileName())
   set prefix=##class(%SYS.Journal.System).GetJournalFilePrefix() 
   set pJrnID=$tr($e(file,$l(prefix)+1,*),".")
 } catch ex {
   set sc=$$$ERROR($s(ex.%IsA("%Exception.SystemException"):5002,'ex.Code:5002,1:ex.Code),$lg(ex.Data)_" "_ex.Location_" "_ex.Name)
 }
 quit sc
}

0 0

score 1 · Answer 1 · 2019-08-14T05:32:15-04:00

Hi Alexey.

I'm not aware of such event handler.

However, I wonder why do you need it at all? If you provide use case, perhaps we can advice some other way to achieve it.

score 1 · Answer 2 · 2019-08-14T05:54:13-04:00

Hi Alexander,

Bypass is easy, and I've already suggested it in my initial post.
As to use cases... maybe I'll describe them later. The method of journal switch detection is not critical for them.

Thank you anyway.

score 0 · Answer 3 · 2019-09-04T15:18:24-04:00

Ignoring whether there is a %SYSTEMJournal:IsPrimary() or some such (I simply don't know). If primary and alternate are on their own separate disk devices (dev/sdj/pri_journals) you will see writes only on one of them. Not very bulletproof but depends on what you are looking for.

score 0 · Answer 4 · 2019-09-11T08:26:24-04:00

Simpler to watch the cconsole.log?

I can't find an example but; test if the primary and alternate are on different paths?

06/23/18-19:37:30:760 (19971) 0 CACHE JOURNALING SYSTEM MESSAGE
Journaling switched to: /trak/site/live/jrnpri/MIRROR-TCMIRROR-20180623.010

score 0 · Answer 5 · 2019-09-11T08:53:43-04:00

Hi Murray,

I was interested in journal file switching recognition rather than primary to alternative directory switch.

Thanks anyway.

score 0 · Answer 6 · 2019-09-11T08:57:44-04:00

ahh, OK, I don't know why I got primary/alternate in my head... so the example is the standard message;

Journaling switched to: ...

score 1 · Answer 7 · 2019-09-04T18:40:32-04:00

If you want to detect a journal change, it means, you want to do something, when such a change took place.

I think, you could do something like this:

a) Start a process at system start (for example, from %ZSTART), which will handle journal changes
b) In that process, get the current journal file
c) try to open this file
d) if (c) fails, wait a few seconds and then goto c)
e) if (c) succeds, handle journal file change, then goto b)

A simplified routine, but untested (as a suggestion):

journalchanges ; Handle journal file changes

#define SWITCHES $system.Util.GetSwitch()
#define SHUTDOWN ##Expression(2**16)
#define BITAND(%a,%b) $zboolean(%a,%b,1)
   
   set jouFile=$$getCurrentFile()  // get the current journal file
   if jouFile="" Quit              // should not happen, but who knows...
   
   while '$$$BITAND($$$SWITCHES, $$$SHUTDOWN) { // stop, if shutdown is in progress
      open jouFile:"WL":1 // try to get ownership
      continue:'$test     // locked (by the journaling system), we try once again
      close jouFile       // because we were interested in status only
   
      // Now you can use <jouFile> to handle the file change
      // By now, he journaling system uses a new journal file
      
      set jouFile=$$getCurrentFile() // get the new journal file
      if jouFile = "" quit           // just to be on the safe side
   }
   
   quit  // we are done
   
    
getCurrentFile()
{
   try { return ##class(%SYS.Journal.System).GetCurrentFile().Name } catch { return "" }
}

score 0 · Answer 8 · 2019-09-11T09:14:39-04:00

Hi Julius,

My solution is similar to yours while it is based on journal API only. If anybody is interested, I'll post it in a few days (being on vacations at the moment).

Thank you for reminding to check if the shutdown is in progress, while I am not sure how often my code should wake up to detect it for sure. Yours waking up each one second as I see.

score 0 · Answer 9 · 2019-09-16T08:24:53-04:00

Hi Julius,

Your solution is beautiful; alas, this code

open jouFile:"WL":1 // try to get ownership

instantly returns $Test=1 in Caché for Linux as its behavior differs from Windows version in this case according to documentation:
Caché Development References > Caché I/O Device Guide > Sequential File I/O: OPEN Mode Locking

I've checked it in Cache for UNIX (Red Hat Enterprise Linux for x86-64) 2017.2.2 before writing here.

score 0 · Answer 10 · 2019-09-17T03:49:24-04:00

When the file is opened as "W" (not "WL") in Caché for Linux, other process can immediately open it as "WL". For some reason journal files are opened as "W", so we have a "false positive" effect if attempt to check journal switching this way.

score 0 · Answer 11 · 2019-09-10T11:06:19-04:00

I'm not sure if this is the simplest or the quickest, but if your licensing includes the ability of creating an Ensemble Production (that's the 2012 term, anyway - we've not yet upgraded to HealthShare or Iris) you could create a pair of Services based on the EnsLib.File.PassthroughService - one for the primary and one for the secondary journaling directories. Each service will scan the directory (it's configurable how often with the Call Interval parameter) and you can fire a Business Process or Operation when the new file hits. You can even set up a File Specification to make sure that if someone manually puts a file in one of the directories that the process may not 'fire' if it's not supposed to.

You could even have different rules for each process - for example, if the Secondary Journaling directory gets a new file created in it, the Business Process could have a rule to fire off an email to the system administrator(s) warning that the drive housing the primary journaling directory may be full or compromised in some way.

Hope this helps!

score 0 · Answer 12 · 2019-09-11T09:17:06-04:00

Alexey Maslov · Sep 11, 2019

Roger,

Ensemble, IRIS, etc are not available in my case. Thanks anyway.

0 0