Checking your Backups and the number of journals to keep

Article

Mario Sanchez Macias · Nov 15, 2021 2m read

Open Exchange

Working in support, I usually get asked how many days I should keep journals. Should it be two days or after two backups? More? Less? Why two?

The correct answer (for most of the environments) is that you should keep the journals since the last validated Backup. I.e., until you don't check if a Backup is valid (restoring the file and checking with the Integrity utility), you can't be sure there is a good copy of your data and can't purge the journals safely.

For example, imagine you need to restore your system after a hardware failure that corrupted some databases. The first step is to go to the last backups and restore them. But, what happens if the Backup is corrupt or saved into a defective disk? You will need to look for a previous Backup until you find a correct and clean copy. Then, if you want to recover to the very last moment, you will need to apply the journals. If you only have one or two days, it won't be enough, and you may lose data.

The only way to make sure that your backups are valid is by checking them! And for checking, I mean restoring the databases and validating the data on them. The way to check the data inside a database is by using the Integrity utility.

When I explain this, most administrators consider it a complex task, which takes a lot of time and is hard to automate. So, I decided to build a super simple Backup checker/validator that helps you validate Backups easily. I am uploading the utility to the open exchange site.

The utility is a simple Backup checker/validator for Backups done with InterSystems Iris. It will restore your Backup file (.cbk) automatically and will run an integrity report afterward. All the "magic" is done in the restoreAll method of the Installer class. You can borrow the code and improve it to send you an email when it finishes with the results.

Once the Backup has been restored and the Integrity check run, the docker log (and messages.log) will contain the restore and integrity check results. The databases restored will appear in a Restore folder.

Robert Cemper · Nov 16, 2021

running the docker image uncovered the result (interactive:=> docker-compose up)
only at stopping of the container. So I changed installer.sh and run the method from iris session

This gave me the full dimension of this EXCELLENT TOOL! especially the integrity check!
I just can say WOW.
So delegating of routine jobs of a System Manager becomes safe and easy.

This is the output that kicked me:

-------------------------------------------------------------------------------
irisowner@7aa9b053e544:/opt/irisapp$ iris session iris -U "%SYS" 
Node: 7aa9b053e544, Instance: IRIS
%SYS>do ##class(BackupChecker.Installer).restoreAll("backup.cbk")
Starting to restore the Backup... It will take some time, be patient!
The following directories will be restored:
c:\intersystems\iris\mgr\appcode\ =>
                         /ISC/utiles/Restore/c:\intersystems\iris\mgr\appcode\/
c:\intersystems\iris\mgr\appdata\ =>
                         /ISC/utiles/Restore/c:\intersystems\iris\mgr\appdata\/
c:\intersystems\iris\mgr\cacheuser\ =>
                       /ISC/utiles/Restore/c:\intersystems\iris\mgr\cacheuser\/
c:\intersystems\iris\mgr\remote\ =>
                          /ISC/utiles/Restore/c:\intersystems\iris\mgr\remote\/
c:\intersystems\iris\mgr\user\ =>
                            /ISC/utiles/Restore/c:\intersystems\iris\mgr\user\/ Cannot mount database in /ISC/utiles/Restore/c:\intersystems\iris\mgr\cacheuser\/ because
its default collation is not available (see messages.log).
c:\intersystems\iris\mgr\cacheuser\ will not be restored. Cannot mount database in /ISC/utiles/Restore/c:\intersystems\iris\mgr\remote\/ because
its default collation is not available (see messages.log).
c:\intersystems\iris\mgr\remote\ will not be restored. ***Restoring c:\intersystems\iris\mgr\appcode\ to /ISC/utiles/Restore/c:\intersystems\iris\mgr\appcode\/ at 10:18:55
469 blocks restored in 0.0 seconds for this pass, 469 total restored. ***Restoring c:\intersystems\iris\mgr\appdata\ to /ISC/utiles/Restore/c:\intersystems\iris\mgr\appdata\/ at 10:18:55
413 blocks restored in 0.0 seconds for this pass, 413 total restored. Starting skip of c:\intersystems\iris\mgr\cacheuser\.      skipped 77 blocks in .007096 seconds. Starting skip of c:\intersystems\iris\mgr\remote\.      skipped 77 blocks in .006635 seconds. ***Restoring c:\intersystems\iris\mgr\user\ to /ISC/utiles/Restore/c:\intersystems\iris\mgr\user\/ at 10:18:55
656 blocks restored in 0.1 seconds for this pass, 656 total restored. ***Restoring c:\intersystems\iris\mgr\appcode\ to /ISC/utiles/Restore/c:\intersystems\iris\mgr\appcode\/ at 10:18:55
1 blocks restored in 0.0 seconds for this pass, 470 total restored. ***Restoring c:\intersystems\iris\mgr\appdata\ to /ISC/utiles/Restore/c:\intersystems\iris\mgr\appdata\/ at 10:18:55
1 blocks restored in 0.0 seconds for this pass, 414 total restored. Starting skip of c:\intersystems\iris\mgr\cacheuser\.      skipped 1 blocks in .000007 seconds. Starting skip of c:\intersystems\iris\mgr\remote\.      skipped 1 blocks in .000008 seconds. ***Restoring c:\intersystems\iris\mgr\user\ to /ISC/utiles/Restore/c:\intersystems\iris\mgr\user\/ at 10:18:55
1 blocks restored in 0.0 seconds for this pass, 657 total restored. ***Restoring c:\intersystems\iris\mgr\appcode\ to /ISC/utiles/Restore/c:\intersystems\iris\mgr\appcode\/ at 10:18:55
1 blocks restored in 0.0 seconds for this pass, 471 total restored. ***Restoring c:\intersystems\iris\mgr\appdata\ to /ISC/utiles/Restore/c:\intersystems\iris\mgr\appdata\/ at 10:18:55
1 blocks restored in 0.0 seconds for this pass, 415 total restored. Starting skip of c:\intersystems\iris\mgr\cacheuser\.      skipped 1 blocks in .000022 seconds. Starting skip of c:\intersystems\iris\mgr\remote\.      skipped 1 blocks in .000008 seconds. ***Restoring c:\intersystems\iris\mgr\user\ to /ISC/utiles/Restore/c:\intersystems\iris\mgr\user\/ at 10:18:55
1 blocks restored in 0.0 seconds for this pass, 658 total restored. Mounting /ISC/utiles/Restore/c:\intersystems\iris\mgr\appcode\/
    /ISC/utiles/Restore/c:\intersystems\iris\mgr\appcode\/  ... (Mounted) Mounting /ISC/utiles/Restore/c:\intersystems\iris\mgr\appdata\/
    /ISC/utiles/Restore/c:\intersystems\iris\mgr\appdata\/  ... (Mounted) Mounting /ISC/utiles/Restore/c:\intersystems\iris\mgr\user\/
    /ISC/utiles/Restore/c:\intersystems\iris\mgr\user\/  ... (Mounted) 
[Journal not applied to any directory] 
Restored completed!

Starting the Integrity report
Directory: /ISC/install/mgr/
No errors found
Summary of blocks in /ISC/install/mgr/         87 Pointer Level blocks         696kb (16% full)
     5,908 Data Level blocks             46MB (84% full)
     2,553 Big String blocks             19MB (86% full) # = 1,011
     8,563 Total blocks                  66MB (84% full)
     2,951 Free blocks                   23MB Elapsed time = 14.3 seconds. Directory: /ISC/install/mgr/irisaudit/
No errors found
Summary of blocks in /ISC/install/mgr/irisaudit/         34 Pointer Level blocks         272kb (0% full)
        54 Data Level blocks            432kb (33% full)
         4 Big String blocks             32kb (99% full) # = 1
       107 Total blocks                 856kb (20% full)
        21 Free blocks                  168kb Elapsed time = 6.1 seconds. Directory: /ISC/install/mgr/irislocaldata/
No errors found
Summary of blocks in /ISC/install/mgr/irislocaldata/         35 Pointer Level blocks         280kb (1% full)
       213 Data Level blocks           1704kb (58% full)
        25 Big String blocks            200kb (74% full) # = 13
       288 Total blocks                2304kb (49% full)
     1,130 Free blocks                 9040kb Elapsed time = 6.5 seconds. Directory: /ISC/install/mgr/iristemp/
No errors found
Summary of blocks in /ISC/install/mgr/iristemp/         46 Pointer Level blocks         368kb (0% full)
        81 Data Level blocks            648kb (33% full)
         0 Big String blocks
       142 Total blocks                1136kb (19% full)
     1,177 Free blocks                 9416kb Elapsed time = 8.5 seconds. Directory: /ISC/install/mgr/user/
No errors found
Summary of blocks in /ISC/install/mgr/user/         50 Pointer Level blocks         400kb (0% full)
        73 Data Level blocks            584kb (27% full)
         0 Big String blocks
       138 Total blocks                1104kb (14% full)
     1,270 Free blocks                10160kb Elapsed time = 8.9 seconds. Directory: /ISC/utiles/Restore/c:\intersystems\iris\mgr\appcode\/
No errors found
Summary of blocks in /ISC/utiles/Restore/c:\intersystems\iris\mgr\appcode\/         32 Pointer Level blocks         256kb (2% full)
       379 Data Level blocks           3032kb (71% full)
        43 Big String blocks            344kb (88% full) # = 12
       469 Total blocks                3752kb (66% full)
     1,067 Free blocks                 8536kb Elapsed time = 0.1 seconds. Directory: /ISC/utiles/Restore/c:\intersystems\iris\mgr\appdata\/
No errors found
Summary of blocks in /ISC/utiles/Restore/c:\intersystems\iris\mgr\appdata\/         57 Pointer Level blocks         456kb (0% full)
       241 Data Level blocks           1928kb (66% full)
       100 Big String blocks            800kb (92% full) # = 30
       413 Total blocks                3304kb (61% full)
       995 Free blocks                 7960kb Elapsed time = 0.1 seconds. Directory: /ISC/utiles/Restore/c:\intersystems\iris\mgr\user\/
No errors found
Summary of blocks in /ISC/utiles/Restore/c:\intersystems\iris\mgr\user\/         68 Pointer Level blocks         544kb (1% full)
       447 Data Level blocks           3576kb (70% full)
       126 Big String blocks           1008kb (90% full) # = 33
       656 Total blocks                5248kb (65% full)
       752 Free blocks                 6016kb Elapsed time = 0.2 seconds. Directory: /usr/irissys/mgr/enslib/
No errors found
Summary of blocks in /usr/irissys/mgr/enslib/        130 Pointer Level blocks        1040kb (33% full)
    14,684 Data Level blocks            114MB (97% full)
     4,407 Big String blocks             34MB (88% full) # = 1,606
    19,236 Total blocks                 150MB (94% full)
     1,500 Free blocks                   11MB Elapsed time = 12.6 seconds. Directory: /usr/irissys/mgr/irislib/
No errors found
Summary of blocks in /usr/irissys/mgr/irislib/        180 Pointer Level blocks        1440kb (60% full)
    39,001 Data Level blocks            304MB (98% full)
    11,177 Big String blocks             87MB (86% full) # = 4,234
    50,373 Total blocks                 393MB (95% full)
        59 Free blocks                  472kb Elapsed time = 7.9 seconds. Directory: /usr/irissys/mgr/zpm/
No errors found
Summary of blocks in /usr/irissys/mgr/zpm/         44 Pointer Level blocks         352kb (11% full)
     2,129 Data Level blocks             16MB (97% full)
       453 Big String blocks           3624kb (88% full) # = 175
     2,641 Total blocks                  20MB (94% full)
       431 Free blocks                 3448kb Elapsed time = 7.7 seconds. Run successfully and found NO errors! 
Databases are VALIDATED!
%SYS>
-------------------------------------------------------------------------------

0 0

Alexander Koblov · Nov 15, 2021

Mario, where is the link to the utility?

Mario Sanchez Macias · Nov 15, 2021

Added link ;-)

1 0

Ben Spead · Nov 15, 2021

Well done @Mario Sanchez Macias !! A great utility to keep everyone from having to re-invent the wheel themselves :) Thank you for contributing

It was an additional pleasure to see your verbose comments in dock-compose.yml.
not just the usual plain parameters with no explanation of what the intention is!

Mario Sanchez Macias · Nov 16, 2021

Thanks a lot!

That was my intention, trying to show something as simple as possible so anyone can look inside and adapt the code to what they need.

To make it simpler I could have used the API methods and calls provided by default, but I preferred to "hack" those methods to allow restore where I wanted without requesting any prompt and finally running the Integrity check with multiple process. This also allow me to control the return of the integrity report and execute an IF to check if there was an error or not.

I was logging to the messages.log (which is also displayed in the docker logs) just in case you prefer to run it not interactively or in a docker up session. You can of course leave the instance up and call the method.

I didn't want to spend much time in the way of calling the docker instance as I am sure that in a real environment most of the customers will have to write a script moving or renaming the .cbk files and executing the app.