journal restore failed with ZJRNFILT

Answer by Eduard Lebedyuk 17 July 2019

Should be

  If glo["ABC",type="K" Set restmode=0           /*except if a kill on ^ABC*/

Instead of

  If glo["^ABC",type="K" Set restmode=0           /*except if a kill on ^ABC*/

0 0

Comment by lw wei 18 July 2019

I change to

ZJRNFILT(jid,dir,glo,type,restmode,addr,time) /*Filter*/
Set restmode=1 /*Return 1 for restore*/
If glo["ABC",type="K" Set restmode=0 /*except if a kill on ^ABC*/
Quit
;

it still not work, and has `kill command` in journal :

148304	2019-07-18 16:29:22	2915	SET	否	^XYZ(196)	/home/lw/mgr/user/
148348	2019-07-18 16:29:22	2915	SET	否	^XYZ(197)	/home/lw/mgr/user/
148392	2019-07-18 16:29:22	2915	SET	否	^XYZ(198)	/home/lw/mgr/user/
148436	2019-07-18 16:29:22	2915	SET	否	^XYZ(199)	/home/lw/mgr/user/
148480	2019-07-18 16:29:22	2915	SET	否	^XYZ(200)	/home/lw/mgr/user/
148520	2019-07-18 16:29:22	2915	KILL	否	^ABC(1)	/home/lw/mgr/user/
148560	2019-07-18 16:29:22	2915	KILL	否	^ABC(2)	/home/lw/mgr/user/
148600	2019-07-18 16:29:22	2915	KILL	否	^ABC(3)	/home/lw/mgr/user/
148640	2019-07-18 16:29:22	2915	KILL	否	^ABC(4)	/home/lw/mgr/user/
148680	2019-07-18 16:29:22	2915	KILL	否	^ABC(5)	/home/lw/mgr/user/
148720	2019-07-18 16:29:22	2915	KILL	否	^ABC(6)	/home/lw/mgr/user/

"["means contain, so there is no diffence between "ABC" and "^ABC" here??

Comment by lw wei 18 July 2019

Consider the following when using ^ZJRNFILT:

If the startup routine (^STU) calls ^JRNRESTO, it does not call the filter routine under any circumstances.

how can I know whether the startup routine calls ^JRNRESTO or not?

Comment by lw wei 18 July 2019

when I use "do ^JRNRESTO" it works?

This utility uses the contents of journal files
to bring globals up to date from a backup.

Restore the Journal? Yes => Yes
Use current journal filter (ZJRNFILT)? enter Yes or No, please
Use current journal filter (ZJRNFILT)? yes
Use journal marker filter (MARKER^ZJRNFILT)? enter Yes or No, please
Use journal marker filter (MARKER^ZJRNFILT)? no
Apply filter to every selected file? Yes => Yes
Process all journaled globals in all directories? yes
Are journal files created by this Cache instance and located in their original
paths? (Uses journal.log to locate journals)? yes
Specify range of files to process

Enter ? for a list of journal files to select the first and last files from
First file to process:

Entry must be from the list, please re-enter

First file to process:

Entry must be from the list, please re-enter

First file to process:

Entry must be from the list, please re-enter

First file to process: 1 /home/lw/mgr/journal/20190718.008
Final file to process: /home/lw/mgr/journal/20190719.009 =>
Prompt for name of the next file to process? No => No

The following actions will be performed if you answer YES below:

* Listing journal files in the order they will be processed
* Checking for any missing journal file on the list ("a broken chain")

The basic assumption is that the files to be processed are all
currently accessible. If that is not the case, e.g., if you plan to
load journal files from tapes on demand, you should answer NO below.
Check for missing journal files? Yes => Yes

Journal files in the order they will be processed:
1. /home/lw/mgr/journal/20190718.008
2. /home/lw/mgr/journal/20190718.009
3. /home/lw/mgr/journal/20190718.010
4. /home/lw/mgr/journal/20190719.001
5. /home/lw/mgr/journal/20190719.002
6. /home/lw/mgr/journal/20190719.003
7. /home/lw/mgr/journal/20190719.004
8. /home/lw/mgr/journal/20190719.005
9. /home/lw/mgr/journal/20190719.006
10. /home/lw/mgr/journal/20190719.007
11. /home/lw/mgr/journal/20190719.008
12. /home/lw/mgr/journal/20190719.009

While the actual journal restore will detect a journal integrity problem
when running into it, you have the option to check the integrity now
before performing the journal restore. The integrity checker works by
scanning journal files, which may take a while depending on file sizes.
Check journal integrity? No => No
The journal restore includes the current journal file.
You cannot do that unless you stop journaling or switch
journaling to another file.
Do you want to switch journaling? Yes => Yes
Journaling switched to /home/lw/mgr/journal/20190719.010

You may disable journaling of updates for faster restore for all
databases other than mirrored databases. You may not want to do this
if a database to restore is being shadowed as the shadow will not
receive the updates.
Do you want to disable journaling the updates? Yes => Yes
Updates will NOT be journaled
Before we job off restore daemons, you may tailor the behavior of a
restore daemon in certain events by choosing from the options below:

DEFAULT: Continue despite database-related problems (e.g., a target
database is not journaled, cannot be mounted, etc.), skipping affected
updates

ALTERNATE: Abort if an update would have to be skipped due to a
database-related problem (e.g., a target database is not journaled,
cannot be mounted, etc.)

DEFAULT: Abort if an update would have to be skipped due to a
journal-related problem (e.g., journal corruption, some cases of missing
journal files, etc.)

ALTERNATE: Continue despite journal-related problems (e.g., journal
corruption, some missing journal files, etc.), skipping affected updates

Would you like to change the default actions? No => No

Start the restore? Yes => Yes

***WARNING: Not restoring /home/lw/mgr/ because the database is not journaled

***WARNING: Not restoring /home/lw/mgr/cachelib/ because the database is not journaled

***WARNING: Not restoring /home/lw/mgr/cachetemp/ because the database is not journaled

***WARNING: Not restoring /home/lw/mgr/cache/ because the database is not journaled

***WARNING: Not restoring /home/lw/mgr/docbook/ because the database is not journaled

***WARNING: Not restoring /home/lw/mgr/samples/ because the database is not journaled

/home/lw/mgr/journal/20190718.008
100.00%
***Journal file finished at 11:47:30

/home/lw/mgr/journal/20190718.009
92.15% 94.17% 96.04% 98.06% 99.93%100.00%
***Journal file finished at 11:47:30

/home/lw/mgr/journal/20190718.010
62.62% 64.67% 66.53% 68.58% 90.91%100.00%
***Journal file finished at 11:47:30

/home/lw/mgr/journal/20190719.001
78.55% 84.13% 88.61% 99.42%100.00%
***Journal file finished at 11:47:30

/home/lw/mgr/journal/20190719.002
73.58% 79.37% 91.91%100.00%
***Journal file finished at 11:47:30

/home/lw/mgr/journal/20190719.003
100.00%
***Journal file finished at 11:47:30

/home/lw/mgr/journal/20190719.004
100.00%
***Journal file finished at 11:47:30

/home/lw/mgr/journal/20190719.005
100.00%
***Journal file finished at 11:47:30

/home/lw/mgr/journal/20190719.006
100.00%
***Journal file finished at 11:47:30

/home/lw/mgr/journal/20190719.007
100.00%
***Journal file finished at 11:47:30

/home/lw/mgr/journal/20190719.008
100.00%
***Journal file finished at 11:47:30

/home/lw/mgr/journal/20190719.009
100.00%
***Journal file finished at 11:47:30

[journal operation completed]
Do you want to rename your journal filter? yes
Journal filter ZJRNFILT renamed to XJRNFILT

Answer by Alexey Maslov 19 July 2019

Why do you think that ZJRNFILT doesn't work, maybe your journal files just don't contain any `set ^ABC(I)=I` command? After all, our guesses are easy to check. Just inject a couple of statements in your code:

ZJRNFILT(jid,dir,glo,type,restmode,addr,time) /*Filter*/
  Set restmode=1                              /*Return 1 for restore*/
  If glo["ABC",type="S",$i(^mtempfilt("S"))
  If glo["ABC",type="K",$i(^mtempfilt("K")) Set restmode=0 /*except if a kill on ^ABC*/
  Quit

and check ^mtempfilt value after the journal restoration:

  zw ^mtempfilt

0 0

Comment by lw wei 19 July 2019

Thank for your reply. When I used the method of "do ^JRNRESTO" , the filter worked fine and I got the data as I expected. So my problem is that the following code is not working properly, or I missed something.

s RestOref=##class(Journal.Restore).%New()
s RestOref.FirstFile="20190717.007"
s RestOref.LastFile="20190717.007"
s RestOref.JournalLog="journal.log"
s RestOref.Filter="^ZJRNFILT"
s Status=RestOref.Run()

Comment by Alexey Maslov 19 July 2019

I use the similar code for the same purpose and it works. Here is my (debug) version has been written awhile ago:

  ; In:
  ; pFirst - 1st journal file,
  ; pLast - last journal file,
  ; pSDB - source DB directory where journals were generated,
  ; pTDB - target DB directory where journals should be applied
  ;
  ; Sample:
  ; zn "test" set target=$zu(12,""), source=target ; start in target namespace assuming source == target
  ; k ^mtempfilt ; if you injected your ZJRNFILT
  ; d jrnrest^ztestFF("20190917.001","20190919.008",source,target)
  ; zwrite ^mtempfilt
  ; 
 
jrnrest(pFirst,pLast,pSDB,pTDB)

 new (pFirst,pLast,pSDB,pTDB) ;de!!!
 new $namespace s $namespace="%SYS"
 s pLast=$g(pLast,pFirst)
 s pSDB=$g(pSDB,$zu(12)_"user\")
 s pTDB=$g(pTDB,$zu(12)_"test\")
 s RestOref=##class(Journal.Restore).%New()
 s RestOref.FirstFile=pFirst
 s RestOref.LastFile=pLast
 s RestOref.RollBack=0 ;? default = 1
 w !,pFirst_" to "_pLast_"; "_pSDB_" => "_pTDB_" OK?" r ans#1 w ! quit:"nN"[ans
 s sc=RestOref.RedirectDatabase(pSDB,pTDB) if 'sc g jrnbad
 s sc=RestOref.SelectUpdates(pSDB) if 'sc g jrnbad ; all globals; need it to fire RedirectDatabase
 s RestOref.Filter="^ZJRNFILT1" ; means nothing as ^ZJRNFILT is always used
 s t0=$zh
 s CHATTY=0 ; has no effect
 s sc=RestOref.Run() if 'sc g jrnbad
 s t1=$zh
 w "sc="_sc_" dt="_$fn(t1-t0,"",3),!
jrnbad
 if $g(sc)'="",'sc d $system.Status.DisplayError(sc)
 quit

journal restore failed with ZJRNFILT

Answers