Hi Julius,
My solution is similar to yours while it is based on journal API only. If anybody is interested, I'll post it in a few days (being on vacations at the moment).
Thank you for reminding to check if the shutdown is in progress, while I am not sure how often my code should wake up to detect it for sure. Yours waking up each one second as I see.
Hi Murray,
I was interested in journal file switching recognition rather than primary to alternative directory switch.
Thanks anyway.
Actual rules used for names and surnames transliteration are more complex as they can be phonetically dependent. E.g. "Егор" -> "Egor", but "Иеремия" -> "Iyeremiya".
Eduard,
You have just forgotten about "Щ" in your awesome one-liner, while the $replacing of "Ш" is excessive. So, it should look like that:
$tr($zcvt($replace($replace($tr(russian, "абвгдезийклмнопрстуфхыэАБВГДЕЗИЙКЛМНОПРСТУФХЫЭЖЦЧШЮЯжцчшюяьЬъЪ", "abvgdeziyklmnoprstufhyeABVGDEZIYKLMNOPRSTUFHYE婨味䍨卨奵奡穨瑳捨獨祵祡"),"щ","shch"),"Щ","Shch"),"O","UnicodeBig"),$c(0))
Another part of my solution was published as a comment here.
This trick with answers (fin.txt) and messages (fout.txt) files was possible with background job only as only this kind of Caché processes allows the separation principal-input device from principal-output one.
Controlling the execution state would not be so easy in this case as fout.txt is kept opened during the execution of JRNREST. I partially bypass this limitation looping my dejournaling activity for each journal file I need to restore and feeding JRNREST with only one file on each iteration (pLast = pFirst); so it's possible to report the restore state of each file that has been restored with very modest delay as the restoration of one journal file is usually a quick process.
HTH and happy coding to you!
I'm confused a bit because you:
...don't want it to run in the background
and
...don't want to use interactive mode
at the same time. So, what do you really want? Maybe your question worth a separate article.
The approach discussed can be useful for background jobs as UI is not available.
Running standard JRNREST utility would be easiest way otherwise.
Robert,
As to (simulated) named arguments approach:
(+): Good readability.
(-): We should be pretty sure that values would not contain separators, neither "=" nor ":", although screening of them is not a very hard problem ("\=", "\:", "\\").
(-): Handling binary data (e.g., $list or $bit) seems to be more serious problem. In this case it seems that one argument structured as $lb($lb("name1",value1),...$lb("nameN",valueN)) is better option.
I use option 1 (macros) with MAC routines and usually option 2 (parameters) with classes. I'd always use macros if it was possible to define them class wide without placing inside INC routines.
David,
you can try
set rc=$zf(-1,"export MYVAR="_someValue)
while it would hardly work as Caché process environment will be destroyed after the process halt. Honestly, I've never tried it.
I usually use signal files to pass values back from Caché, as it was shown in the 1st sample here: How to return the status code of Cache process to OS shell script?
See Timothy Leavitt's answer to the same post for another option which is more convenient when you are to return numeric values.
Hi Alexander,
Bypass is easy, and I've already suggested it in my initial post.
As to use cases... maybe I'll describe them later. The method of journal switch detection is not critical for them.
Thank you anyway.
No, it works and all installations but yes, it needs 8 Bit characters
It needs Unicode characters as well, otherwise it fails. Just compare 8 bit instance run:
USER>w "IsUnicode = ",$zbitget($zversion(0),1) IsUnicode = 0 USER>s x="f i=1:1:100 w:i#3=0 ""Fizz"" w:i#5=0 ""Buzz"" w:'$x i w ! " w $l(x) 54 USER>s xx=$zwpack(x) S xx=$ZWPACK(x) ^ <WIDE CHAR>
with Unicode one:
USER>w "IsUnicode = ",$zbitget($zversion(0),1) IsUnicode = 1 USER>s x="f i=1:1:100 w:i#3=0 ""Fizz"" w:i#5=0 ""Buzz"" w:'$x i w ! " w $l(x) 54 USER>s xx=$zwpack(x) USER>x $zwunpack(xx) 1 2 Fizz ...
Hi James,
Agree with you, parsing terminal output is not the smartest solution. I always try to avoid it using intermediate files. E.g. (from real life):
rm -f $mydir/db_temp
NspToCheck=$nspace
csession $instance -U%SYS << EOF > /dev/null
set fn="$mydir/db_temp"
o fn:("WNS"):1 u fn
try {
zn "$NspToCheck"
w \$s(\$zu(5)=\$zcvt("$NspToCheck","U"):\$zutil(12,""),1:"$NspToCheck")
} catch {
w \$p(\$ze,">")_">"
}
c fn
h
EOF
DbQms=$(cat "$mydir/db_temp")
Here the default DB of the namespace $NspToCheck (or $ZError code) is written to $mydir/db_temp file, than it goes to $DbQms shell variable and processed as needed.
Initial answer was amended.
I'm confused a bit. Do you keep asking on this issue? If not, please describe your situation in details.
Sorry, didn't mentioned your *new* quesion about environment variables.
After cheching my großbuch I found that I was wrong with starting version: $zf(-100) is available since 2017.2.1 (Build 801_3), while your build hasn't got this "_3" suffix.
Zdeněk,
I'm just curious: are you intentionally testing $zf() stuff on command (`pwd`) that doesn't exist in your particular OS?
Hi Murray,
Thank you for responding.
You mentioned earlier that switching to Direct I/O would slow down Caché On-line Backup; it sounded strange for me: if database I/O and journal I/O can use different strategies, why On-line Backup can't always use buffered I/O regardless of database I/O strategy?
Your sample works for me:
USER>s rc = $zf(-100,"/SHELL","pwd") /cachesys/mgr/user USER>w $zv Cache for UNIX (Ubuntu Server LTS for x86-64) 2017.2.2 (Build 865U) Mon Jun 25 2018 10:48:26 EDT
What Caché version are you using? $zf(-100) was added in 2017.2.1 (Build 801_3). For older versions use $zf(-1 or -2), while upgrade to latest Caché (or even IRIS?) release would probably be the better choice.
The simplest way to interact from within bash with Caché looks like this:
#!/bin/bash ... csession TEST -U%SYS << EOF set \$namespace="%SYS" write ##class(Security.System).AutheEnabledGetStored("SYSTEM") halt EOF ... next line of your bash script ...
Output of Caché `write` and `zwrite` commands will go to STDOUT. As usual, you can redirect it wherever you want, e.g.
csession TEST -U%SYS << EOF >> /home/james/mysession.log
As parsing csession log can be a nasty task, I usually try to avoid it by construct:
#!/bin/bash ... tf=/home/file.tmp ... csession TEST -U%SYS << EOF > /dev/null open $tf:("NWS"):1 use $tf write ##class(Security.System).AutheEnabledGetStored("SYSTEM") close $tf halt EOF ... result=$(cat "$tf")
Murray,
Do you recommend this setting (wduseasyncio=1) for production as well? If it slows down all disk i/o, what will happen with journaling?
I use the similar code for the same purpose and it works. Here is my (debug) version has been written awhile ago:
; In:
; pFirst - 1st journal file,
; pLast - last journal file,
; pSDB - source DB directory where journals were generated,
; pTDB - target DB directory where journals should be applied
;
; Sample:
; zn "test" set target=$zu(12,""), source=target ; start in target namespace assuming source == target
; k ^mtempfilt ; if you injected your ZJRNFILT
; d jrnrest^ztestFF("20190917.001","20190919.008",source,target)
; zwrite ^mtempfilt
;
jrnrest(pFirst,pLast,pSDB,pTDB)
new (pFirst,pLast,pSDB,pTDB) ;de!!!
new $namespace s $namespace="%SYS"
s pLast=$g(pLast,pFirst)
s pSDB=$g(pSDB,$zu(12)_"user\")
s pTDB=$g(pTDB,$zu(12)_"test\")
s RestOref=##class(Journal.Restore).%New()
s RestOref.FirstFile=pFirst
s RestOref.LastFile=pLast
s RestOref.RollBack=0 ;? default = 1
w !,pFirst_" to "_pLast_"; "_pSDB_" => "_pTDB_" OK?" r ans#1 w ! quit:"nN"[ans
s sc=RestOref.RedirectDatabase(pSDB,pTDB) if 'sc g jrnbad
s sc=RestOref.SelectUpdates(pSDB) if 'sc g jrnbad ; all globals; need it to fire RedirectDatabase
s RestOref.Filter="^ZJRNFILT1" ; means nothing as ^ZJRNFILT is always used
s t0=$zh
s CHATTY=0 ; has no effect
s sc=RestOref.Run() if 'sc g jrnbad
s t1=$zh
w "sc="_sc_" dt="_$fn(t1-t0,"",3),!
jrnbad
if $g(sc)'="",'sc d $system.Status.DisplayError(sc)
quit
Why do you think that ZJRNFILT doesn't work, maybe your journal files just don't contain any `set ^ABC(I)=I` command? After all, our guesses are easy to check. Just inject a couple of statements in your code:
ZJRNFILT(jid,dir,glo,type,restmode,addr,time) /*Filter*/
Set restmode=1 /*Return 1 for restore*/
If glo["ABC",type="S",$i(^mtempfilt("S"))
If glo["ABC",type="K",$i(^mtempfilt("K")) Set restmode=0 /*except if a kill on ^ABC*/
Quitand check ^mtempfilt value after the journal restoration:
zw ^mtempfilt
But it can temporarily switch off journalling at all, or just suspend the transaction
Switching journaling off [for the current process] is unreliable as it's ignored in MIRRORred environment. Not sure about suspending transactions.
Some details are in “Suspending All Current Transactions” section of “Transaction Processing” chapter of documentation.
Looking through the docs, I'm getting curious if there is a discrepancy between its different parts:
1) Suspending All Current Transactions
You can use the TransactionsSuspended() method of the %SYSTEM.Process class to suspend all current transactions system-wide...
2) IRIS 2019.2 Class Reference. %SYSTEM.Process
The TransactionsSuspended(switch) class method controls a switch that will allow a process to temporarily suspend transactions.
I hope that the only second statement is true, isn't it?
System-wide setting of an environment variable is rather good solution sometimes, especially when Caché/IRIS is a single service of the given server. If was discussed a while ago along with some TZ issue with Caché, which was actual for those days versions: https://community.intersystems.com/post/linux-tz-environment-variable-not-being-set-and-impact-cach%C3%A9
Subsrcript values are inserted into a varaible / global in a way that they are automatically sorted, first in canonical form and second (as I understand) in byte order, so stringy non canonical numbers will still be sorted, but they will apear after canonical numbers and before alpha characters.
The latter is not always true. A quick sample is:
s MsgDTH="0.001005933", d=MsgDTH+10, b="1E1", c="1A1"
s a(b)=b, a(c)=c, a(MsgDTH)=MsgDTH, a(d)=d zwrite a
a(10.001005933)=10.001005933 // canonical number
a("0.001005933")="0.001005933" // non-canonical number
a("1A1")="1A1" // alpha-numeric string
a("1E1")="1E1" // non-canonical number
As to sharding, we would hardly use it as we don't use persistent objects at all. As to other IRIS advanced system dependent features, we can, but we don't want to port them to Cache. E.g. we can write some code to improve the life in clouds, but we don't need it in Cache because of lack of plans of new Cache deployments in clouds.
Our nearest plan is to continue development in Cache having some kind of code convertor to IRIS. It would be nice if ISC would provide such utility by itself, at least as a starting point for further customisation.
Taking this approach we would not loose any functionality. We confess that we are restricting ourselves to the features available in both platforms this way, but it does not seem to be a huge price for the benefits of single code base.
Katherine,
thank you for thorough explanation of possible connection problems. A question I have: you mentioned an internal method
Specifically, Caché SSH client sometimes closes the session during remote command long (and silent) execution implemented using Execute() method. It happens in local network, so we don't expect communication problems. We noticed that this most likely happens under high load of SSH client Caché instance, when several SSH sessions with different SSH servers are concurrently in use.