To complete my expertise in data storage, I joined the editor Oracle, before joining InterSystems in 2001.
My skills in data processing have therefore been extended, from multi-model storage, interoperability (and more particularly on health exchange standards), to real-time analysis and artificial intelligence (AI / ML), so as to cover a broad spectrum of the data lifecycle in all industries.
Pour compléter mon expertise en stockage de données, j'ai ensuite intégré l'éditeur Oracle, avant de rejoindre InterSystems France dès 2001.
Mes compétences en traitement de la donnée se sont dès lors étendues, depuis le stockage multi-modèles, l'interopérabilité (et plus particulièrement sur les standards d'échanges en santé), jusqu'à l'analyse en temps-réel et à l'intelligence artificielle (IA/ML), de manière à couvrir un large spectre du cycle de vie de la donnée.
The failover works fine as well if instead of a shutdown of the Linux server, I execute a "iris force iris".
Hi @Kamal Suri
below my results in a mirror env.
After stopping the Primary Linux server, the IRIS instance on the Backup Linux server becomes Primary and forces the previous Primary IRIS to shutdown when it comes back (as expected).
When you restart the IRIS instance in the previous Primary Linux, it becomes Backup.
Below the messages.log from the previous Backup who became Primary :
07/09/24-11:47:15:808 (5427) 0 [Database.MountedRO] Mounted database /usr/iris/mgr/user/ (SFN 5) read-only. Mirrored DB needs activation.
07/09/24-11:47:29:355 (5312) 0 [Database.MirrorActivated] Activated database /usr/iris/mgr/user/ (SFN 5) in mirror 'MIRROR' for use on this mirror member; mounted read-only.
07/09/24-11:47:34:176 (5552) 0 [Utility.Event] Mirror 'MIRROR' catchup started for 1 database.
07/09/24-11:47:34:883 (5552) 1 [Utility.Event] :mirror:MIRROR:USER (sfn 5) remains inactive because it is not active on the primary
07/09/24-11:51:14:822 (5465) 1 [Generic.Event] MirrorClient: Primary AckDaemon failed to answer status request.
07/09/24-11:51:14:823 (5465) 0 [Generic.Event] MirrorClient: Backup waiting for old Dejournal Reader (pid: 5466, job #33) to exit
07/09/24-11:51:15:698 (5465) 0 [Generic.Event] MirrorClient: Set status for MIRROR to Transition
07/09/24-11:51:18:705 (5449) 0 [Utility.Event] Backup taking over for primary in arbiter controlled mode
07/09/24-11:51:18:752 (5649) 0 [Utility.Event] Applying journal data for mirror "MIRROR" starting at 382140 in file #5(/usr/iris/mgr/journal/MIRROR-MIRROR-20240709.005)
07/09/24-11:51:18:784 (5449) 0 [Utility.Event] Manager initialized for MIRROR
07/09/24-11:51:18:812 (5449) 0 [Generic.Event] INTERSYSTEMS IRIS JOURNALING SYSTEM MESSAGE
Journaling switched to: /usr/iris/mgr/journal/MIRROR-MIRROR-20240709.006
07/09/24-11:51:18:822 (5449) 0 [Utility.Event] Scanning /usr/iris/mgr/journal/MIRROR-MIRROR-20240709.005
07/09/24-11:51:18:853 (5449) 0 [Utility.Event] No open transactions to roll back
07/09/24-11:51:18:855 (5449) 0 [Generic.Event] MirrorServer: New primary activating databases which are current as of 382140 (0x0005d4bc) in mirror journal file #5
07/09/24-11:51:18:855 (5449) 0 [Generic.Event] Changed database /usr/iris/mgr/user/ (SFN 5) to read-write due to becoming primary.
07/09/24-11:51:18:855 (5449) 0 [Generic.Event] MirrorServer: Primary startup reset to Agent Controlled failover
07/09/24-11:51:19:012 (5449) 0 [Utility.Event] Initializing Interoperability during mirror initialization
07/09/24-11:51:19:013 (5449) 2 [Utility.Event] Becoming primary mirror server
07/09/24-11:51:28:743 (5192) 0 [Utility.Event] [SYSTEM MONITOR] Mirror status changed. Member type = Failover, Status = Primary
07/09/24-11:51:29:683 (5652) 0 [Utility.Event] Journal File Compression: Compressed /usr/iris/mgr/journal/20240709.009 from 327680 bytes to 102400 bytes, compression ratio = 3.2
07/09/24-11:51:29:705 (5652) 0 [Utility.Event] Journal File Compression: Compressed /usr/iris/mgr/journal/MIRROR-MIRROR-20240709.005 from 393216 bytes to 106496 bytes, compression ratio = 3.7
07/09/24-11:51:54:196 (5449) 0 [Utility.Event] Forcing down old primary ("UBUNTU/IRIS") which is currently in a trouble state
07/09/24-11:53:53:566 (5687) 0 [Generic.Event] MirrorServer: Starting connection for: UBUNTU/IRIS (failover member) (ver 4) at offset 0x00031e68 (204392) in file #6
07/09/24-11:53:54:669 (5687) 0 [Generic.Event] MirrorServer: Client up to date, linking with journal daemon @ 0x00031e68
07/09/24-11:53:54:858 (5687) 0 [Generic.Event] MirrorServer: Switched from Agent Controlled to Arbiter Controlled failover
and the messages.log from the previous Primary who is now Backup :
07/09/24-11:53:50:157 (6577) 0 [Utility.Event] [SYSTEM MONITOR] Mirror state: Member type = Failover, Status = Transition
07/09/24-11:53:50:236 (6543) 0 [Generic.Event] Auditing to /usr/iris/mgr/irisaudit/
07/09/24-11:53:50:274 (6543) 0 [Utility.Event] Enabling logons
07/09/24-11:53:50:282 (6543) 0 [Utility.Event] Initializing Interoperability during system startup
07/09/24-11:53:50:877 (6695) 0 [Utility.Event] DejournalProcessor: Applying journal data for mirror "MIRROR" starting at 382140 in file #5(/usr/iris/mgr/journal/MIRROR-MIRROR-20240709.005)
07/09/24-11:53:50:877 (6557) 0 [Utility.Event] Retrieving journal file #6 for mirror MIRROR from 192.168.102.131|2188
07/09/24-11:53:51:905 (6695) 0 [Utility.Event] DejournalProcessor: Applying journal data for mirror "MIRROR" starting at 0 in file #6(/usr/iris/mgr/journal/MIRROR-MIRROR-20240709.006)
07/09/24-11:53:51:919 (6550) 0 [Utility.Event] LMF Info: Connected to license server 127.0.0.1,4002
07/09/24-11:53:53:071 (6697) 0 [Utility.Event] Applying journal data for mirror "MIRROR" starting at 204392 in file #6(/usr/iris/mgr/journal/MIRROR-MIRROR-20240709.006)
07/09/24-11:53:53:200 (6557) 0 [Utility.Event] Retrieving journal file #6 for mirror MIRROR from 192.168.102.131|2188...(repeated 1 times)
07/09/24-11:53:53:200 (6557) 0 [Utility.Event] GetECPSessionData: Got ECP data from agent len=1
07/09/24-11:53:53:200 (6557) 0 [Utility.Event] Manager initialized for MIRROR
07/09/24-11:53:53:302 (6557) 0 [Utility.Event] Found UBUNTU-2/IRIS as primary, becoming backup
07/09/24-11:53:53:303 (6557) 0 [Utility.Event] (MIRROR) Becoming a backup mirror member
07/09/24-11:53:53:567 (6699) 0 [Utility.Event] (MIRROR) Starting Mirror.Client("MIRROR-MIRROR-20240709.006",204392,6)
07/09/24-11:53:53:569 (6699) 0 [Generic.Event] MirrorClient: Connected to primary: UBUNTU-2/IRIS (ver 4)
07/09/24-11:53:53:582 (6701) 0 [Utility.Event] Arbiter connection established (192.168.102.1:2188)
07/09/24-11:53:54:861 (6699) 0 [Generic.Event] MirrorClient: Switched from Agent Controlled to Arbiter Controlled failover on request from primary
07/09/24-11:53:54:916 (6699) 0 [Generic.Event] MirrorClient: The backup node has become active
07/09/24-11:54:01:989 (6718) 0 [Utility.Event] Journal File Compression: Compressed /usr/iris/mgr/journal/MIRROR-MIRROR-20240709.005 from 393216 bytes to 106496 bytes, compression ratio = 3.7
07/09/24-11:54:20:174 (6577) 0 [Utility.Event] [SYSTEM MONITOR] Mirror status changed. Member type = Failover, Status = Backup
![InterSystems Developer Community Expert](https://images.credly.com/images/0451c765-1220-41c0-b22f-7c777dbe17dc/image.png)
![InterSystems Developer Community Moderator](https://images.credly.com/images/288a3661-72e9-4275-a943-063ae9e7910f/image.png)
![Open Sesame!](https://community.intersystems.com/sites/default/files/badges/303371180-6de11138-f4e7-4113-b5cc-836269f07693.png)
![1,000 Points](https://community.intersystems.com/sites/default/files/badges/290121674-53216c4b-fe0c-44e7-9d75-2f24390135ed.png)
![Challenge Starter](https://community.intersystems.com/sites/default/files/badges/295959213-b83dd91a-663f-4f07-8e0e-e3287b7c2309.png)
![Popular writer](https://community.intersystems.com/sites/default/files/badges/poular_writer.png)
![Notable writer](https://community.intersystems.com/sites/default/files/badges/303371140-d391d770-6fa4-4540-a8d5-926f8dfe9f61.png)
![Blogger Badge](https://community.intersystems.com/sites/default/files/badges/blogger_badge_0.png)
![Reporter Badge](https://community.intersystems.com/sites/default/files/badges/reporter_badge_0.png)
And to answer your question, when you simply restart the Primary Linux server, the Backup member is automatically switching to become primary.
Below the messages.log from the Backup member while the Primary is restarting.
07/09/24-12:05:49:781 (6502) 0 [Utility.Event] DejournalProcessor: Applying journal data for mirror "MIRROR" starting at 0 in file #7(/usr/iris/mgr/journal/MIRROR-MIRROR-20240709.007) 07/09/24-12:05:50:953 (6508) 0 [Utility.Event] Applying journal data for mirror "MIRROR" starting at 200088 in file #7(/usr/iris/mgr/journal/MIRROR-MIRROR-20240709.007) 07/09/24-12:05:51:076 (6362) 0 [Utility.Event] Retrieving journal file #7 for mirror MIRROR from 192.168.102.130|2188...(repeated 1 times) 07/09/24-12:05:51:076 (6362) 0 [Utility.Event] GetECPSessionData: Got ECP data from agent len=1 07/09/24-12:05:51:076 (6362) 0 [Utility.Event] Manager initialized for MIRROR 07/09/24-12:05:51:185 (6362) 0 [Utility.Event] Found UBUNTU/IRIS as primary, becoming backup 07/09/24-12:05:51:186 (6362) 0 [Utility.Event] (MIRROR) Becoming a backup mirror member 07/09/24-12:05:51:445 (6510) 0 [Utility.Event] (MIRROR) Starting Mirror.Client("MIRROR-MIRROR-20240709.007",200088,7) 07/09/24-12:05:51:448 (6510) 0 [Generic.Event] MirrorClient: Connected to primary: UBUNTU/IRIS (ver 4) 07/09/24-12:05:51:458 (6512) 0 [Utility.Event] Arbiter connection established (192.168.102.1:2188) 07/09/24-12:05:52:737 (6510) 0 [Generic.Event] MirrorClient: Switched from Agent Controlled to Arbiter Controlled failover on request from primary 07/09/24-12:05:56:740 (6510) 0 [Generic.Event] MirrorClient: The backup node has become active from a status query 07/09/24-12:05:59:766 (6517) 0 [Utility.Event] Journal File Compression: Compressed /usr/iris/mgr/journal/MIRROR-MIRROR-20240709.006 from 262144 bytes to 81920 bytes, compression ratio = 3.2 07/09/24-12:06:17:920 (6377) 0 [Utility.Event] [SYSTEM MONITOR] Mirror status changed. Member type = Failover, Status = Backup 07/09/24-14:31:54:643 (6512) 2 [Utility.Event] Arbiter connection lost 07/09/24-14:31:55:304 (6362) 0 [Utility.Event] Skipping connection to arbiter while still in Arbiter Controlled failover mode. 07/09/24-14:31:55:419 (6510) 0 [Generic.Event] MirrorClient: Switched from Arbiter Controlled to Agent Controlled failover on request from primary 07/09/24-14:32:25:437 (6584) 0 [Utility.Event] Arbiter connection established (192.168.102.1:2188) 07/09/24-14:32:28:235 (6510) 0 [Generic.Event] MirrorClient: Switched from Agent Controlled to Arbiter Controlled failover on request from primary 07/09/24-14:33:11:438 (6510) 1 [Generic.Event] MirrorClient: Primary AckDaemon failed to answer status request. 07/09/24-14:33:11:438 (6510) 0 [Generic.Event] MirrorClient: Backup waiting for old Dejournal Reader (pid: 6511, job #28) to exit 07/09/24-14:33:12:250 (6510) 0 [Generic.Event] MirrorClient: Set status for MIRROR to Transition 07/09/24-14:33:12:254 (6362) 0 [Utility.Event] Backup taking over for primary in arbiter controlled mode 07/09/24-14:33:12:304 (6589) 0 [Utility.Event] Applying journal data for mirror "MIRROR" starting at 356088 in file #7(/usr/iris/mgr/journal/MIRROR-MIRROR-20240709.007) 07/09/24-14:33:12:330 (6362) 0 [Utility.Event] Manager initialized for MIRROR 07/09/24-14:33:12:362 (6362) 0 [Generic.Event] INTERSYSTEMS IRIS JOURNALING SYSTEM MESSAGE Journaling switched to: /usr/iris/mgr/journal/MIRROR-MIRROR-20240709.008 07/09/24-14:33:12:368 (6362) 0 [Utility.Event] Scanning /usr/iris/mgr/journal/MIRROR-MIRROR-20240709.007 07/09/24-14:33:12:388 (6362) 0 [Utility.Event] No open transactions to roll back 07/09/24-14:33:12:391 (6362) 0 [Generic.Event] MirrorServer: New primary activating databases which are current as of 356088 (0x00056ef8) in mirror journal file #7 07/09/24-14:33:12:391 (6362) 0 [Generic.Event] Changed database /usr/iris/mgr/user/ (SFN 4) to read-write due to becoming primary. 07/09/24-14:33:12:391 (6362) 0 [Generic.Event] MirrorServer: Primary startup reset to Agent Controlled failover 07/09/24-14:33:12:406 (6362) 0 [Utility.Event] Initializing Interoperability during mirror initialization 07/09/24-14:33:12:410 (6362) 2 [Utility.Event] Becoming primary mirror server 07/09/24-14:33:13:141 (6590) 0 [Utility.Event] Journal File Compression: Compressed /usr/iris/mgr/journal/20240709.010 from 196608 bytes to 73728 bytes, compression ratio = 2.7 07/09/24-14:33:13:166 (6590) 0 [Utility.Event] Journal File Compression: Compressed /usr/iris/mgr/journal/MIRROR-MIRROR-20240709.007 from 393216 bytes to 77824 bytes, compression ratio = 5.1 07/09/24-14:33:26:097 (6377) 0 [Utility.Event] [SYSTEM MONITOR] Mirror status changed. Member type = Failover, Status = Primary 07/09/24-14:36:51:766 (6651) 0 [Generic.Event] MirrorServer: Starting connection for: UBUNTU/IRIS (failover member) (ver 4) at offset 0x00030d98 (200088) in file #8 07/09/24-14:36:52:875 (6651) 0 [Generic.Event] MirrorServer: Client up to date, linking with journal daemon @ 0x000310a4 07/09/24-14:36:52:878 (6651) 0 [Generic.Event] MirrorServer: Switched from Agent Controlled to Arbiter Controlled failover
And from the side of the former primary who's now restarted as backup :
07/09/24-14:36:45:825 (1689) 0 [Utility.Event] Mirror manager for MIRROR starting 07/09/24-14:36:45:842 (1697) 0 [Utility.Event] Starting TASKMGR 07/09/24-14:36:45:860 (1704) 0 [Utility.Event] [SYSTEM MONITOR] System Monitor started in %SYS 07/09/24-14:36:45:909 (1690) 0 [Utility.Event] Shard license: 1 07/09/24-14:36:45:910 (1711) 0 [Utility.Event] WorkQueue: Starting work queue daemon parent=1690 07/09/24-14:36:46:058 (1704) 0 [Utility.Event] [SYSTEM MONITOR] Mirror state: Member type = Failover, Status = Transition 07/09/24-14:36:46:319 (1568) 0 [Generic.Event] Auditing to /usr/iris/mgr/irisaudit/ 07/09/24-14:36:46:421 (1568) 0 [Utility.Event] Enabling logons 07/09/24-14:36:46:429 (1568) 0 [Utility.Event] Initializing Interoperability during system startup 07/09/24-14:36:47:653 (1682) 0 [Utility.Event] LMF Info: Connected to license server 127.0.0.1,4002 07/09/24-14:36:47:930 (1830) 0 [Utility.Event] DejournalProcessor: Applying journal data for mirror "MIRROR" starting at 356088 in file #7(/usr/iris/mgr/journal/MIRROR-MIRROR-20240709.007) 07/09/24-14:36:47:930 (1689) 0 [Utility.Event] Retrieving journal file #8 for mirror MIRROR from 192.168.102.131|2188 07/09/24-14:36:48:968 (1830) 0 [Utility.Event] DejournalProcessor: Applying journal data for mirror "MIRROR" starting at 0 in file #8(/usr/iris/mgr/journal/MIRROR-MIRROR-20240709.008) 07/09/24-14:36:50:285 (1833) 0 [Utility.Event] Applying journal data for mirror "MIRROR" starting at 200088 in file #8(/usr/iris/mgr/journal/MIRROR-MIRROR-20240709.008) 07/09/24-14:36:50:523 (1689) 0 [Utility.Event] Retrieving journal file #8 for mirror MIRROR from 192.168.102.131|2188...(repeated 1 times) 07/09/24-14:36:50:523 (1689) 0 [Utility.Event] GetECPSessionData: Got ECP data from agent len=1 07/09/24-14:36:50:523 (1689) 0 [Utility.Event] Manager initialized for MIRROR 07/09/24-14:36:50:712 (1689) 0 [Utility.Event] Found UBUNTU-2/IRIS as primary, becoming backup 07/09/24-14:36:50:713 (1689) 0 [Utility.Event] (MIRROR) Becoming a backup mirror member 07/09/24-14:36:51:239 (1835) 0 [Utility.Event] (MIRROR) Starting Mirror.Client("MIRROR-MIRROR-20240709.008",200088,8) 07/09/24-14:36:51:242 (1835) 0 [Generic.Event] MirrorClient: Connected to primary: UBUNTU-2/IRIS (ver 4) 07/09/24-14:36:51:266 (1837) 0 [Utility.Event] Arbiter connection established (192.168.102.1:2188) 07/09/24-14:36:52:353 (1835) 0 [Generic.Event] MirrorClient: Switched from Agent Controlled to Arbiter Controlled failover on request from primary 07/09/24-14:36:56:358 (1835) 0 [Generic.Event] MirrorClient: The backup node has become active from a status query 07/09/24-14:36:57:774 (1850) 0 [Utility.Event] Journal File Compression: Compressed /usr/iris/mgr/journal/MIRROR-MIRROR-20240709.007 from 393216 bytes to 77824 bytes, compression ratio = 5.1 07/09/24-14:37:16:627 (1704) 0 [Utility.Event] [SYSTEM MONITOR] Mirror status changed. Member type = Failover, Status = Backup