Question
David Sterngast · Apr 4, 2021

$zf call out to Window DIR command not working when running concurrent process

 

I have an application that is called by terminal in local cache environment. The application basically has two processes that runs and does the below call to retrieve files and sub-directories in separate folders.    When the main process starts, it jobs off and calls the second process that gathers all files in a directory by using a window dir and file command that pipes the results to a file.  The main process continues on and does the same thing but from a different folder.  When checking out what was retrieved by the secondprocess, there were directories and files missing.   When I run this as just one process it works properly.    I am running this on a 2017 license cache instance.   I have also tried this on IRIS instance and got the same results.  Could you please advise me what is causing the window directory to not work properly when running the processes at the same time?

 

Main process one

S FILE=SDIR_"DIRLIST.TXT"

I $$FSIZECK(FILE) S FSIZE=1 Q ""

S X=$ZF(-1,"DEL """_FILE_"""")

S X=$ZF(-1,"DIR """_SDIR_""" >> """_FILE_"""")

 

 

Second process one

S FILE=SDIR_"DIRLIST.TXT"

I $$FSIZECK(FILE) S FSIZE=1 Q ""

S X=$ZF(-1,"DEL """_FILE_"""")

S X=$ZF(-1,"DIR """_SDIR_""" >> """_FILE_"""")

 

 

Product version: IRIS 2020.4
$ZV: cache 2017 and IRIS
00
1 0 6 117
Log in or sign up to continue

Replies

Let's analyze this:    [leaving aside that $Zf(-1 .. is Deprecated)

  1. You run the same sequence in 2 Caché/IRIS  processes in parallel.
  2. There is no synchronization between them
  3. Both run in the same namespace ==> therefore the same default directory where SDIR_"DIRLIST.TXT" is located ???? " .. but from a different folder... "    is SDIR different ????? if not :
  4. Both processes spawn 2 times a sub-process for each $ZF(-1)
  5. let's name the sub processes fg1, fg2 and bg1,bg2
  6. The sub_process from foreground is not synchronized to the sub-process of background and your construct has no control on the sequence they are running.
  7. If the run sequence  fg, fg1,fg2,bg,bg1,bg2  there should be no problem. But that's rater unlikely
  8. it looks like fg,bg,fg1,fg2,bg1 (partially deleting results of fg2), bg2  

To verify what's really happening run fg,fg1,fg1,bg (JOB ), bg1,bg2   
So they both can't influence each other.

Instead of the double $ZF()  I'd suggest using the query in Library class %File.

 
set rs=##class(%ResultSet).%New("%File:FileSet")
do rs.Execute(SDIR,"*",1)
while rs.Next() { write !,rs.GetData(1) }

Thanks for your response.  

There is a main process that jobs off another process in parallel.

There is no communication/ synchronization between the processes.

Both run in the same namespace and the same main folder.

The process are different because there sub-folders are different SDIR_"DIRLIST.TXT" is located.

SDIR is different

The main processes spawn $zf(-1) and does  $ZF(-1)

When running in parallel.

The first process completes without any  issue.

The child process  the dir.txt file created from the dir is missing files and sub-folders.

Sorry!  That doesn't indicate any reason. $zf(-1) is just a spawn.

To answer your question that they can't influence each other, that is correct.  I can run each process independently from each other they work.  I can also run it as one process and it works properly.   When I run it in parallel it fails.   Might be due to that it is a tied terminal process, the privileges are different when spawning a job or window process when I use $ZF.      The zf was used to call window command functions.  

Important:
Jobbed Process Permissions are Platform-dependent
Running Programs or System Commands with $ZF(-100)

So that we speak the same language, I made a simple example using Using the Work Queue Manager

Class dc.test Abstract ]
{

ClassMethod MyJob(SDIR As %String)
{
  FILE=##class(%File).NormalizeFilename("DIRLIST.TXT",SDIR)

  q:##class(%File).GetFileSize(FILE)>0

  ##class(%File).Delete(FILE)

  X=$ZF(-1,$$$FormatText("DIR %1 >> %2",$$$quote(##class(%File).NormalizeDirectory(SDIR)),$$$quote(FILE)))
}

/// d ##class(dc.test).test()
ClassMethod test()
{
  N=4
  queue=$system.WorkMgr.Initialize(,.sc,N)
  i=1:1:queue.Queue("##class(dc.test).MyJob","C:\Temp\test "_i)
  queue.WaitForComplete()
}

}

I copied different files to the following directories:

C:\Temp\test 1
C:\Temp\test 2
C:\Temp\test 3
C:\Temp\test 4

After calling ##class(dc.test).test() , everything worked out as expected: DIRLIST.TXT were created in the corresponding directories each with its own content.