cmPurgeBackup as useful add-on to IRIS/Caché Online Backup

Introduction

Despite the fact that InterSystems has long recommended using external backup tools, many users have opted to use the internal Online Backup facility, which is included in all distributions of InterSystems products (IRIS Data Platform, Caché, etc.). The reasons why are quite obvious:

• This solution works right out of the box, and it does not require you to use third-party tools.
• It is easy to configure in accordance with your specific local backup settings.
• It supports practically all network file system protocols (SMB/CIFS, NFS, iSCSI, etc.).
• It can be used on any mirror node without fear that you will be forced to pause the virtual machine.
• It supports the selective restoration of databases, and you can even rename them.

Does it sound like I am just promoting the benefits? Let's move on to a discussion of the disadvantages:

• The backup speed is slower than copying the database using third-party tools.
• There is no mechanism to ensure that there is always enough free storage space to complete the next backup.

The first drawback is somewhat exaggerated: the copy speed really depends on the storage that is used. However, when you copy a database at the OS file system level, the copy speed is similar to using the internal Online Backup tool.

The second drawback is more serious, and there have been repeated discussions in our community about how to overcome it. I would like to propose a solution that has been proven by many years of use on our customers' servers.

Installing and configuring the cm.Task.PurgeBackup

cmPurgeBackup is an utility that can be used to erase old Online Backup files. It runs as an IRIS/Caché Task Manager task. It is fully compatible with all predefined classes of backup tasks, such as FullDBList, IncrementalDBList, etc., and it does not require any changes to the code that is used in these tasks.

It is easy to install and configure the cmPurgeBackup task. First you import the cm.Task.Purge.inc and cm.Task.PurgeBackup.cls to the namespace of your choice (meaning you don't have to import it to %SYS), compile the class. Then go to the Task Manager menu and schedule a new task. Name it somehow (e.g., cmPurgeBackup). Now select the namespace where the task class was imported to, and select cm.Task.PurgeBackup class from the drop-down list of types.

You don't have to change the default values of the task properties, but some explanations are still in order.

The FreeSpace property has priority over FullBackupsToKeep, therefore in order to preserve the required amount of free disk space, Purge Backup can save fewer full Backup files than the number specified in the FullBackupsToKeep property. Nevertheless, the program will ensure that the results of the last full backup and all subsequent incremental backups are never erased. At the same time, the default value will be the minimum allowed FreeSpace value. The acceptability of the FreeSpace property is checked every time the task is run. If the value is found to be too small, it will be increased accordingly.

HistoryLifeSpan also affects the retention period for Full Backup (.log) log files. This decision is made on the basis of the fact that these files take up little space, but they contain valuable information about how the size of the full backup has grown over time, meaning about the databases themselves.

If the LogFile is configured without specifying the path, then <install_dir>/mgr will be used by default. Another option is to use the system log (messages.log for IRIS or cconsole.log for Caché). In this case, just leave the LogFile field empty. If there are any errors when accessing the specified log file, then the task will switch to using the system log.

You can flexibly determine how frequently the task is run, while it seems logical to specify that it be run every time that the Full Backup task is finished running.

How often do you want the Task Manager to execute this task? After another task is finished.

Run after which task? FullDBList

This allows you to clear the storage of old .cbk files immediately after the completion of a successful backup while at the same time ensuring that no files are deleted if the full backup is unsuccessful.

FAQ

• How does cmPurgeBackup know in which directories to look for (and delete) backup files? It knows:
  • from the properties of the backup tasks that are scheduled in Task Manager; the program takes in account the tasks with classes from %SYS.Task.Backup package;
  • from the link to the last full backup (^SYS("BACKUP","LASTFULL"));
  • from the backup history (^SYS("BUHISTORY"));
  • from the description of typical backup tasks (^SYS("BACKUPTASK")).
• How safe is the cmPurgeBackup task? Is there any danger that it can delete something that it shouldn't? The following precautions are taken:
  • Only certain predetermined file types are allowed to be deleted. These are .cbk (backup)  and .log (backup log) files.
  • The task keeps its own log file in either a file with the fixed name (purgeback.log) or in the system log file (messages.log for IRIS or cconsole.log for Caché). This prevents arbitrary log files from being overwritten.
• Which InterSystems products are supported by cmPurgeBackup? While cmPurgeBackup was initially developed in Caché 2008.1 early days, I can’t guarantee that it is compatible with any server product version published during the last decade. Current version of cmPurgeBackup was tested with the following InterSystems products:
  • InterSystems IRIS Version 2019.1, 2020.1,
  • Caché Version 2017.2,

therefore I don’t expect any problem with other contemporary versions.

• How can I get cmPurgeBackup? cmPurgeBackup is an Open Exchange application and can be downloaded by following the link below.