Looking for a good routine/class for purging Caché backups

A customer is using Caché online backups and needs to automatically purge the cbk files with a scheduled task.

This is a wheel has been reinvented uncountable times already and I know somebody out there has a well written, extremely robust version that has already stood the test of time.

Does anyone have a nice routine/class/task for purging old Caché backup files? 

  • 0
  • 1
  • 806
  • 5
  • 3

Answers

Funny you should ask this as I was just looking at how to do this today.  

Most operating systems offer a way to search for files given a certain filter, such as being older than a certain date, and then piping that list to another command, such as delete.

Here is a class method I wrote to do this on a Windows 2012 R2 server running Cache

ClassMethod PurgeFiles(Path As %String, OlderThan As %Integer)
{
    set Date=$zd($h-OlderThan)
    set cmd="forfiles /P "_Path_" /D -"_Date_" /C ""cmd /c del @path"""
    set sc=$zf(-1,cmd)
}

This method accepts a path and an integer indicating the number of days to keep files for.  It then uses constructs a command line which uses the "forfiles" command passing the path and a calculated date.  For each file it finds, it executes the command cmd /c del <path> which deletes the file.

There are probably more elegant ways to do this, cross platform compatible, but this is one solution that I had.

 

If you look at the %Library.File:FileSet() query and the %Library.File:Delete() method I think you could probably convert this to an OS-independent solution in about 10 min or less :)  Just sort the FileSet() query results by DateCreated and then stop processing the files once you get to the cutoff date for what you want to keep :) 

I would refrain from using $ZF(-1,cmd) because if the callout gets hung at the O/S for any reason, the job issuing the $ZF will be hung.

I’d suggest using InterProcess Communication to the O/S or the %Fileset query as shown below.

Here's an example of how one might use the FileSet query in the %File class and the Delete class method in %File to purge backup files in a given directory before a given date.

/// Purge backups older than <var>DaysToKeep</var>
/// <var>Path</var> points to the directory path containing the backups.
/// Only *.cbk files will be purged
ClassMethod PurgeBackups(Directory As %String, DaysToKeep As %Integer = 14) As %Integer
{
	// Calculate the oldest date to keep files on or after
	set BeforeThisDate = $zdt($h-DaysToKeep_",0",3)

	// Gather the list of files in the specified directory
	set rs=##class(%ResultSet).%New("%File:FileSet")
	do rs.Execute(Directory,"*.cbk","DateModified")

	// Step through the files in DateModified order
	while rs.Next() {
		set DateModified=rs.Get("DateModified")
		if BeforeThisDate]DateModified {
			// Delete the file
			set Name=rs.Get("Name")
			do ##class(%File).Delete(Name)
		}
		// Stop when we get to files with last modified dates on or after our delete date
		if DateModified]BeforeThisDate quit
	}
}

 

Bravo! Make sure to tag this as "Code Sample" so people can find it easily in the future

I apologize - I thought it was possible to add Tags to Answers, but I guess not.  The Code Sample tag would need to be added by Marc to the original question.

The article Recommendations on installing the InterSystems Caché DBMS for production environment  has some code that does this.

My comment  has a class that only deletes old backups as a scheduled task.

Kind regards,

Stephen

Class App.PurgeOldBackupFiles Extends %SYS.Task.Definition
{

Property BackupsToKeep As %Integer(MAXVAL = 30, MINVAL = 1) [ InitialExpression = 30, Required ];

Property BackupFolder As %String [ Required ];

Property BackupFileType As %String [ Required ];

Method OnTask() As %Status
{
//s BackupsToKeep = 2
//s Folder = "c:\backupfolder"
//s BackupFileType = "FullAllDatabases" // or "FullDBList"

SortOrder = "DateModified"

If ..BackupsToKeep<1 Quit $$$ERROR($$$GeneralError,"Invalid - Number of Backups to Keep must be greater than or equal to 1")
If ..BackupFolder="" Quit $$$ERROR($$$GeneralError,"Invalid - BackupFolder - not supplied")
if ..BackupFileType = "" Quit $$$ERROR($$$GeneralError,"Invalid - BackupFileType - not supplied")
if (..BackupFileType '= "FullAllDatabases")&&(..BackupFileType '= "FullDBList") Quit $$$ERROR($$$GeneralError,"Invalid - BackupFileType")

BackupCount=0
     k zPurgeOldBackupFiles(..BackupFileType)
     Set rs=##class(%ResultSet).%New("%Library.File:FileSet")
     w !,"backuplist",!

     s BackupFileWildcard = ..BackupFileType _ "*.cbk"
     set status=rs.Execute(..BackupFolder, BackupFileWildcard, SortOrder)
     WHILE rs.Next() {
          Set FullFileName=rs.Data("Name")
          Set FName=##class(%File).GetFilename(FullFileName)
          Set FDateTime=##class(%File).GetFileDateModified(FullFileName)
          w "File "_FName_" "_FDateTime,!
          Set FDate=$PIECE(FDateTime,",")
          Set CDate=$PIECE($H,",")
          s BackupCount=$I(BackupCount)
          s zPurgeOldBackupFiles(..BackupFileType, BackupCount)=FullFileName 
     }
     s zPurgeOldBackupFiles(..BackupFileType, "BackupCount")=BackupCount
     do rs.Close()
     if BackupCount > ..BackupsToKeep {
          for i=1:1:BackupCount-..BackupsToKeep {
               s FullFileName = zPurgeOldBackupFiles(..BackupFileType, i)
               d ##class(%File).Delete(FullFileName)
               w "File Purged "_FullFileName_" Purged",!
          }
     }
     q status
}

}