I would like to understand the mechanism of "How data is written to the IRIS Database and in what sequence data is written to Journal, WIJ and IRIS database"
I would like to understand the mechanism of "How data is written to the IRIS Database and in what sequence data is written to Journal, WIJ and IRIS database"
Comments
WIJ (write image journal) is used for data before it is saved to the database. This is used to keep the data consistency (block level in database) and avoid database degradation. If IRIS (or the OS) crash, WIJ will have "dirty blocks" (not written to databases) that will be weritten during the next IRIS startup.
Journal files (or mirror journal, in case you have mirroring) are used after data saved to the database. This will help for transactions rollback and also to ensure a good restore point up to the "current time" along with the latest backup restore.
The sequence is:
1. Read a block was changed from the database
2. Write the modified block to WIJ
3. Write the modified block to the database
4. Write the global SETS and KILLS (and also transaction start/stop) to the journal file
I'd suggest to start with the documentation chapter Resiliency and Data Integrity
Then you can come back if you still have questions.
To add to what @Yaron Munz posted, it is important to note that journaling is an independent data flow that doesn't depend on the write daemon. How deeply you need to go into understanding the data flow depends on why you are asking. If you are trying to resolve a problem, reach out to the WRC.
Thanks Eric for the response. Actually I need a simple understanding. As much as it sounds stupid, I want to understand the workflow of Journal, WIJ and Database when we have a transaction.
My understanding is :
Transaction ----> Write deamon writes to WIJ ------> Write Deamon writes to Database--------> Journals are updated.
My Questions are:
1. What is the content of WIJ, these contains just pointers of data or actual data?
2. If Journals are written after data is committed to database then in case of application consistent snapshot creation, during iris freeze, database writes are suspended, so how the journal files are still getting generated. (I understand Write daemon does not play a role in Journal)
3. Suppose I have a time gap between IRIS freeze and snapshot generation and in between couple of journal files are generated. If I have snapshotted WIJ, Journal and database and on Secondary host I want to recover the latest data by applying from the updated journal, can we do that.
4. I am trying to snapshot WIJ, JNL and data and trying to start the instance in the secondary host by mapping the snapshots. But I am getting WIJ mismatch error following which Journal Recovery error. Why is that??
Thank you for providing your feedback
Hi Malaya,
First, by "Freeze", I think you mean executing ##class(Backup.General).ExternalFreeze(). That method pauses the write daemon at the end of a cycle. This does not stop the journal daemon. Journals are not written at the end of the write daemon cycle, they are written continually as you make updates.
The usage of the WIJ is described here: https://docs.intersystems.com/iris20253/csp/docbook/DocBook.UI.Page.cls?KEY=GCDI_wij.
Data blocks that are going to be written to databases are first written to the WIJ. If there is a crash while the databases are being updated, some blocks may have made it to disk, and some not. IRIS detects this and applies the blocks from the WIJ at the next startup.
If you are getting warnings about MISMATCH.WIJ, contact the WRC for help. There may be a problem with how you are backing up, or there may be another problem with your storage.
One other thing to keep in mind about journal files:
When you restore from a backup, you usually want to bring the system up to the current point in time. For that, you need journal files from the time of the backup up to the current time. That means that some of the journal files you need aren't going to be in your backup, because they didn't exist when you took your snapshot. You would need to account for this in your backup planning.