Windows 7 performs shutdown too fast for Cache to close and so it gets forced down

 Windows 7 and some other Microsoft Operating Systems can shutdown too fast for large applications, such as a Cache instance with a large amount of data and changes, to close gracefully. This results in the instance being forced down by the OS and so causing problems on the next start up.

 

To solve this we can change the OS shutdown timeout values to give Cache more time to close gracefully. We do this by editing some settings in the Registry.

 

1.Click Start (on Win 7) and type “regedit” and make sure you run registry editor as administrator.

 

2.Go to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control and double click WaitToKillServiceTimeout entry on the right side and enter its value as 12000 (for 12 seconds ) or higher to increase the time the shutdown will wait for services to stop. This will give the Cache Service more time to close gracefully.

 

3.Go to HKEY_CURRENT_USER\Control Panel\Desktop , on the right pane create new string value (right click on "Desktop" in the navigation panel on the left and select "New" and then "String Value") named WaitToKillAppTimeout and give its value as 12000 (double click on your newly created WaitToKillAppTimeout and then enter a value like 12000 in the Value box). If that string already exists then double click it and enter its value as 12000 or whatever you think it will take to close any Cache/Ensemble instances.

 

 

That should give Cache or Ensemble enough time to close before the Shutdown process kills it!

 

Steve

  • + 5
  • 0
  • 1149
  • 9

Comments

I have this same problem on Windows 10, and it can and does mess up Cache.

Caché should always be shut down separately before the OS is shut down, and that would completely avoid this issue, allowing Caché to go through its normal shutdown procedure.

Stephen, thank you for the info.

May I ask you to clarify a little: should the console log messages like this one

08/29/16-09:41:03:376 (4864) 1 Operating System shutdown!  Cache performing fast shutdown.

be considered as sympthoms of this issue?

Alexey

That cconsole.log message is a good example of the OS shutting down Cache and not necessarily gracefully, but it is NOT a definite indication of the OS killing processes that are not closing quick enough for the OS shutdown. I think if I were to see this I would then check that everything closed nicely, because it is an indication that the OS triggered the shutdown and if I don't have a long enough Timeout setting then there is a good chance that Cache processes could be killed.

In other words, not  a symptom of the  OS shutting down too quickly, but an indication that the OS is the cause of the shutdown and so we should check that the Cache shutdown was a good one.

I hope that answers your question.

 

Steve

Steve, 

> if I were to see this I would then check that everything closed nicely

How to do this check? Every Caché startup after its fast shutdown is corresponded with the following message in console log:

09/16/16-13:48:38:305 (2132) 2 Previous system shutdown was abnormal, system forced down or crashed

while there are no logged signs of "normal" forcing down (system tables dump, etc). Maybe this rule has exceptions, but I've never seen them.

==
Thank you,
Alex

 

There is another parameter we should consider here :

ShutdownTimeout

 Number of seconds Caché should wait for shutdown to complete normally before timing out and forcing a shutdown. The default is 300 seconds (5 minutes). The range is 0 to a maximum of 231–1 seconds.
 
That means if there is a user process (or a background job !) that is not stopping immediately, Caché will by default wait 5 minutes before starting the system shutdown.
The time in 'WaitToKillServiceTimeout' shold be at least 'ShutdownTimeout', plus the max. time to really shut down Caché.
 
Right?

As to documentation for Caché v. 2015.1, ShutdownTimeout parameter ranges from 120 to a maximum of 100,000 seconds with the default of 300 seconds. In my case its value is 120 seconds, but in the worst cases I've managed to find in my log shutdown performed faster, approx. 40 seconds, e.g.: 

05/18/16-18:51:45:817 (3728) 1 Operating System shutdown!  Cache performing fast shutdown.
05/18/16-18:52:27:302 (3728) 1 Forced 11 user processes.  They may not have completed.
05/18/16-18:52:27:302 (3728) 0 Fast shutdown complete
05/18/16-18:52:27:474 (3728) 0 CONTROL exited due to force
05/18/16-18:52:27:630 (3656) 0 JRNDMN exited due to force
05/18/16-18:52:27:614 (3560) 0 GARCOL exited due to force
05/18/16-18:52:27:802 (1064) 0 EXPDMN exited due to force
05/18/16-18:52:27:786 (3760) 0 No blocks pending in WIJ file
05/18/16-18:52:27:880 (3760) 0 WRTDMN exited due to force

while one can see word "force" in the log... It seems that OS shutdown is a special case of forcing Caché down, without waiting ShutdownTimeout seconds. I plan to adjust the registry value as suggested in this article and check what will happen on the next OS shutdown (when I decide to do it).

The quick check for a good clean shutdown is to search for LCK files. Every Cache database will have an LCK file associated with it when it is mounted and these are deleted when the instance shuts down, so if there are lck files still present then this is a good indication that the shutdown was incomplete. The one exception to this is the lck file associated with the Journal files. This is not deleted at shutdown because it may be used to help/diagnose in the recovery process at next start up - you will see the file holds a path and server name.

Beyond looking for lck files and checking the cconsole.log the only other source of information (about the shutdown) is the Windows Event log.

Checking .LCK files is useless in most cases as Caché service auto-starts with OS startup. Of course, switching auto-start off is not a problem for development/testing environment.

Frank touched another interesting question: how long WaitToKillServiceTimeout should be? If we set it to ShutdownTimeout + Typical_Real_Shutdown_Time, and Caché hangs during OS shutdown, I bet that typical Windows admin won't wait 5 minutes and finish with hardware reset...  Choosing between bad and worse, I'd set
WaitToKillServiceTimeout = Typical_Real_Shutdown_Time
letting OS to force Caché down in rare cases when it hangs.