Experiences in synchronising non-CACHE.DAT data in a Mirrored environment

Primary tabs

Hi All

I'm looking for some field experiences, lessons learned, or actual deployed solutions to the problem of replicating non-CACHE.DAT data in a mirrored Cache environment.  

Environment:

  • Operating System is IBM AIX
  • Hardware is IBM Power w/ virtualised storage
  • Cache 2017.1
  • 2x Production mirror members
  • 1x DR mirror member

 

There are a number of different data types that we're looking at mirroring, and a number of identified solutions.  The two main types of data to be mirrored outside of Cache mirroring are:

  • CSP  files underneath database directories
  • 'File Drop' directories, monitored by daemons running in Cache.  These files are eventually consumed by these daemons and imported into Cache databases, but it is not instantaneous.

The CSP files very rarely change, and we'll likely be able to use rsync to copy the files over.  For the 'dropped' files we will require little, potentially zero, data loss from these files during a mirror failover.  

We have a number of answers in mind for this listed below, but I'm also hoping that there's others out there that have faced, and hopefully solved, similar problems.  Anecdotes, comments, solutions;  everything is welcome.

 

My current short list for mirroring non-cache.dat data is:

  • 'File Server' presenting the directories via CIFS/NFS/SMB to both host;  this file server can be highly available by other means
  • Shared Storage; since our storage is virtualised, we can move it between hosts on demand.  In the event of a failover, these commands can be issued to the storage controllers to re-mount the disks on the new production mirror server
  • Parallel shared disk technology, such as IBM GPFS clustered filesystem

 

Thanks for any input you may have!

  • 0
  • 0
  • 251
  • 0
  • 1

Answers

What about something like Syncthing?  I use it to keep company files mirrored & backed up.  It's the same technology as BitTorrent Sync.  I avoid BitTorrent sync intentionally to prevent my data from going through their servers.

I did once use Syncthing to perform a Windows file server migration from a Win 2K server to an Isilon.  Most/Many/All of the users used different combinations of mapped drives, network locations, and shortcuts.  I used Syncthing to mirror the data between the two locations while I was able to migrate the users over time and normalize the methods they used to access files.

Depending on the amount of data this may not be a viable solution for you.