How to do snapshot recovery on a secondary host if IRIS is installed on local OS directory and database and Journal and WIJ are placed on 3 different mounted directory coming from Storage
I have installed IRIS community edition on a local directory in Redhat Linux 9.4.
Created a 10G database on a separate mount point directory which comes out of a storage volume of 50G.
Also placed Journal and WIJ on 2 separate mount point directories which comes out of two different storage volume of 50G each.
I want to take a snapshot and recover the snapshot on a secondary host and check the consistency of the database.
As Storage snapshot will be taken, only Data, Journal and WIJ file placed on Storage volumes will be snapshotted.
Let me know what is the process of recovering the snapshot data in secondary.
As per my understanding, We only can mount the database, journal and WIJ on secondary. We do not need to start the instance in the secondary. So WIJ and Journal will not be of any use.
So I am confused if this is the recommended configuration? How customer build their environment ? and what would be the correct way to recover snapshot data ?
Thanks in advance for the assistance.
Comments
Recovering from a storage snapshot on a secondary host with the setup described involves several considerations to ensure database integrity and consistent functionality.
-
Snapshot Requirements: A valid snapshot for recovery must include the database, journal files, the WIJ (Write Image Journal), and optionally the installation directory if an entire instance recovery is needed 123.
-
Recovery Procedure:
- Mount the Snapshot: Mount the snapshot volumes (database, journal, and WIJ directories) onto the secondary host.
- Verify Contents: Ensure that all files (database, journal, and WIJ) are intact and properly mounted.
- Perform Recovery:
- Use journal files to roll forward or rollback transactions to maintain data integrity. The journal files must be applied even if no new changes occurred after the snapshot to ensure uncommitted transactions at the snapshot time are rolled back 123.
- Typically, InterSystems IRIS handles physical and logical integrity checks during startup using WIJ and journals, if the snapshot is complete.
-
Journals and WIJ:
-
Recommended Configuration:
- For optimal recoverability and performance, system components should be separated across distinct physical or logical storage devices:
- Avoid using symbolic links for database directories for reliability 4.
-
Testing and Best Practices:
If your configuration excludes certain components from the snapshot, it might compromise recovery or data consistency. Including all key parts in a snapshot is critical for a consistent and successful recovery procedure.
Sources:
I am trying to create an application consistent snapshot on Primary host and recover the same from a secondary host. But am receiving the following error always.
*** Recovery started at Mon Mar 2 06:39:35 2026
Current default directory: /IRIS/mgr
Log file directory: /IRIS/mgr/
WIJ file spec: /WIJ/IRIS.WIJ
Recovering local (/WIJ/IRIS.WIJ) image journal file...
Starting WIJ recovery for '/WIJ/IRIS.WIJ'.
0 blocks pending in this WIJ.
WIJ pass # is 0.
Starting fast WIJ compare
Finished comparing 1992 blocks in 0 seconds
**WIJ blocks NOT matching the database ones are saved in MISMATCH.WIJ
Thanks for sharing the detailed setup — it’s helpful to understand how you’ve separated IRIS binaries, database, journal, and WIJ across different mount points.
In a storage-level snapshot scenario, consistency is the key concern. Even if IRIS is installed on the local OS directory, you typically need to ensure the instance is placed into a proper backup state before taking the snapshot. Otherwise, restoring only the database files without coordinated journal/WIJ handling can lead to recovery complications.
On the secondary host, you generally do need an IRIS instance (same version and patch level) to mount the restored database and perform crash recovery. Journals are important if the snapshot was not taken in a fully quiesced state, since IRIS may need them to roll forward transactions and ensure consistency. WIJ also plays a role in recovery integrity depending on the shutdown state.
Many customers either:
-
Use external freeze/thaw mechanisms integrated with IRIS backup APIs, or
-
Put IRIS into online backup mode before snapshot, or
-
Use IRIS mirroring for cleaner secondary validation instead of raw storage snapshots.
Simply mounting database files without starting IRIS would not allow proper consistency validation, since recovery logic runs during instance startup.
It would also be good to confirm whether you’re planning a crash-consistent snapshot or an application-consistent one — that usually determines the exact recovery steps.
Hope this helps clarify the approach a bit.
Thank you, William, for your response. It has provided significant clarity.
I am planning to perform an application-consistent snapshot, and it would be helpful if you could guide me through the exact recovery steps. This will allow me to identify any gaps in my process, as I am consistently encountering the WIJ mismatch error.
Hi Malaya, thank you for your detailed explanation. I really appreciate the clarity you provided. I am also looking to perform an application-consistent snapshot and would love if you could share the exact recovery steps. Understanding them in detail will help me identify where my process might be going wrong, especially since I keep running into the WIJ mismatch error. Your guidance would be really valuable!
See:
- Backup Strategies (External Backup)DOC
- Script sample for running an external Caché / Ensemble backup on Windows (using ##Class(Backup.General).ExternalFreeze() / ExternalThaw())
- Does ExternalFreeze command update the data files before suspending write daemon?
- How to find the journal file that is being switched after Freeze?
Thank you, Vitaliy, for sharing the useful links. They have been very helpful.
I would also like to confirm whether there are any cache-related settings in Red Hat 9.4 that could be contributing to the increased frequency of WIJ errors in my environment.