$zf call out to Window DIR command not working when running concurrent process

Question

Question

David Sterngast · Apr 4, 2021

I have an application that is called by terminal in local cache environment. The application basically has two processes that runs and does the below call to retrieve files and sub-directories in separate folders. When the main process starts, it jobs off and calls the second process that gathers all files in a directory by using a window dir and file command that pipes the results to a file. The main process continues on and does the same thing but from a different folder. When checking out what was retrieved by the secondprocess, there were directories and files missing. When I run this as just one process it works properly. I am running this on a 2017 license cache instance. I have also tried this on IRIS instance and got the same results. Could you please advise me what is causing the window directory to not work properly when running the processes at the same time?

Main process one

S FILE=SDIR_"DIRLIST.TXT"

I $$FSIZECK(FILE) S FSIZE=1 Q ""

S X=$ZF(-1,"DEL """_FILE_"""")

S X=$ZF(-1,"DIR """_SDIR_""" >> """_FILE_"""")

Second process one

S FILE=SDIR_"DIRLIST.TXT"

I $$FSIZECK(FILE) S FSIZE=1 Q ""

S X=$ZF(-1,"DEL """_FILE_"""")

S X=$ZF(-1,"DIR """_SDIR_""" >> """_FILE_"""")

Product version: IRIS 2020.4

$ZV: cache 2017 and IRIS

Discussion (6)0

Log in or sign up to continue

Vitaliy Serdtsev · Apr 12, 2021

Important:
Jobbed Process Permissions are Platform-dependent
Running Programs or System Commands with $ZF(-100)

So that we speak the same language, I made a simple example using Using the Work Queue Manager

Class dc.test [ Abstract ]
{

ClassMethod MyJob(SDIR As %String)
{
  s FILE=##class(%File).NormalizeFilename("DIRLIST.TXT",SDIR)

  q:##class(%File).GetFileSize(FILE)>0

  d ##class(%File).Delete(FILE)

  s X=$ZF(-1,$$$FormatText("DIR %1 >> %2",$$$quote(##class(%File).NormalizeDirectory(SDIR)),$$$quote(FILE)))
}

/// d ##class(dc.test).test()
ClassMethod test()
{
  s N=4
  s queue=$system.WorkMgr.Initialize(,.sc,N)
  f i=1:1:N d queue.Queue("##class(dc.test).MyJob","C:\Temp\test "_i)
  d queue.WaitForComplete()
}

}

I copied different files to the following directories:

C:\Temp\test 1
C:\Temp\test 2
C:\Temp\test 3
C:\Temp\test 4

After calling d ##class(dc.test).test() , everything worked out as expected: DIRLIST.TXT were created in the corresponding directories each with its own content.

0 0

score 1 · Answer 1 · 2021-04-04T12:19:00-04:00

Let's analyze this: [leaving aside that $Zf(-1 .. is Deprecated)

You run the same sequence in 2 Caché/IRIS processes in parallel.
There is no synchronization between them
Both run in the same namespace ==> therefore the same default directory where SDIR_"DIRLIST.TXT" is located ???? " .. but from a different folder... " is SDIR different ????? if not :
Both processes spawn 2 times a sub-process for each $ZF(-1)
let's name the sub processes fg1, fg2 and bg1,bg2
The sub_process from foreground is not synchronized to the sub-process of background and your construct has no control on the sequence they are running.
If the run sequence fg, fg1,fg2,bg,bg1,bg2 there should be no problem. But that's rater unlikely
it looks like fg,bg,fg1,fg2,bg1 (partially deleting results of fg2), bg2

To verify what's really happening run fg,fg1,fg1,bg (JOB ), bg1,bg2
So they both can't influence each other.

score 3 · Answer 2 · 2021-04-04T13:30:06-04:00

Instead of the double $ZF() I'd suggest using the query in Library class %File.

 
set rs=##class(%ResultSet).%New("%File:FileSet")
do rs.Execute(SDIR,"*",1)
while rs.Next() { write !,rs.GetData(1) }

score 0 · Answer 3 · 2021-04-06T15:02:39-04:00

Thanks for your response.

There is a main process that jobs off another process in parallel.

There is no communication/ synchronization between the processes.

Both run in the same namespace and the same main folder.

The process are different because there sub-folders are different SDIR_"DIRLIST.TXT" is located.

SDIR is different

The main processes spawn $zf(-1) and does $ZF(-1)

When running in parallel.

The first process completes without any issue.

The child process the dir.txt file created from the dir is missing files and sub-folders.

score 0 · Answer 4 · 2021-04-07T03:10:37-04:00

Robert Cemper · Apr 7, 2021

Sorry! That doesn't indicate any reason. $zf(-1) is just a spawn.

0 0

score 0 · Answer 5 · 2021-04-08T13:47:15-04:00

To answer your question that they can't influence each other, that is correct. I can run each process independently from each other they work. I can also run it as one process and it works properly. When I run it in parallel it fails. Might be due to that it is a tied terminal process, the privileges are different when spawning a job or window process when I use $ZF. The zf was used to call window command functions.