the functions $order() and $query() are your best friends ;-))

Write, for example, a method like this:

ClassMethod(ref)
{
  if $data(@ref)#10 write ref,"=",@ref,! // do something with this node
  set i=""
  for  set i=$order(@ref@(i)) quit:i=""  do ..Show($name(@ref@(i)))
}

So, for testdata like

set tmp(1,2,3)="Joe"

set tmp(1,2,3,4,1)="Paul"

set tmp(1,2,3,4,2)="Paula"

set tmp(1,2,3,5,6)="Tom"

start some testing


do ##class(yourclass).Show($na(tmp))  // shows all entries

do ##class(yourclass).Show($na(tmp(1,2)))  // also shows all entries

do ##class(yourclass).Show($na(tmp(1,2,3)))  // shows Joe

do ##class(yourclass).Show($na(tmp(1,2,3,4))  // shows Paul and Paula

do ##class(yourclass).Show($na(1,2,3,5)))  // shows Tom

do ##class(yourclass).Show($na(tmp(1,2,3,5,6)))  // shows Tom

do ##class(yourclass).Show($na(tmp(2,3)))  // shows nothing

Now, I hope, you can figure out how to get out your desired data

 set x="1.2.3", y=$name(tmp)
 for i=1:1:$length(x,".") { set y=$name(@y@($piece(x,".",i))) }
 set @y=""
 kill x,y,i

You can put al the above in one line, if you are in terminal. Or put it in a method

ClassMethod setTemp(x) [ PublicList = tmp]
{
 set y=$name(tmp)
 for i=1:1:$length(x,".") { set y=$name(@y@($piece(x,".",i))) }
 set @y=""
} 

Or you take the shorthand

set x="1.2.3", @("tmp("_$tr(x,".",",")_")=""""")

But this works only as long as x contains number_period_numeber_period_...No leading zeros, no alphabetics...

A warning about long time open transactions will be placed in cconsole.log and alerts.log (both files in $system.Util.InstallDirectory()_"mgr" directory) - at least  this is the case what I see on a customers system. The entries look like this:

04/05/19-02:31:34:470 (21012) 1 [SYSTEM MONITOR] TransOpenSecs Warning: One or more transactions open longer than 10 minutes. Process id  18624 22208 (only top 5 shown)
08/29/19-14:31:03:802 (18576) 1 [SYSTEM MONITOR] TransOpenSecs Warning: One or more transactions open longer than 10 minutes. Process id  4128 (only top 5 shown)
09/04/19-17:16:19:090 (21344) 1 [SYSTEM MONITOR] TransOpenSecs Warning: One or more transactions open longer than 10 minutes. Process id  25872 (only top 5 shown)

So it should be as easy as monitoring those two logfiles. But how it looks like on an ECP server, don't ask me.

You could try this way (I assume, your image is in *.jpg, *.png, *.gif, *.ico or *.bmp  format and you use a Windows OS):

- save the stream to a temp file
- do $zf(-1,"pathToIrfanView.exe pathToInpFile  /resize=(100,100)  /aspectratio  /convert=pathToOutFile")
- read the output into a stream and proceed as needed

Don't forget, if one of the above path contains a space char, you have to put in quotes the whole path.

/resize=(100,100)  /aspectration  means, the picture will be resized to a max width/height of 100 by maintaning the aspect ratio. Example: input=2000x3000 out =67x100,  input=3000x2000  out=100x67

For Unix exists similar tools to use with $zf(-1,...), for example "identify" to get the dimensions of the picture  and "convert" or "imagemagick" for resizing.

If your routine works in a terminal session (by issuing a DO ^Routinname command) but does not work, when the same routine is started via the Job commannd (JOB ^Routinname) then you lack something in background, what you have in the foreground.

Put an errortrap in your routine and simple report a possible error (and if you wich, additionally the execution progress of your routine).


rvExt2012 ; Test
    set $zt="bgErr"
    // ...
    // ...
    do signal("checkpoint 1") // this is optional
    // ...
    // ...    do signal("checkpoint 2") // this is optional
    // ...
    // ...
    do signal("Done")
    quit

bgErr do signal("Error: "_$ze)
    quit

signal(msg) do $system.Event.Signal($zp, $username_", "_$roles_", "_msg)
    quit

  

Now, in the terminal session enter following  line

 
job ^rvExt2012 do { set $lb(s,m)=$system.Event.WaitMsg("",1) write m,! } while s

The intention behind my post was, to give you one idea (of many other possibilities), how to convert XML to CSV. A empty (chars) element is just one of some other unhandled cases (missing tags, other tags inside of COLx tag, etc.).

If you need some speed and your XML-File obeys following constraints:

- the file is a correct XML-File
- contains only the 'Data', 'Details' and 'ColX' tags
- no selfclosing tags, like <sometag/>

then you could try the QAD-way (quick-and-dirty) of conversion.

Again, below an example routine (without excessive testing).
All ISC people and ordinary programer, please look the other way ;-))

Convert() Public
{
   set len=32000 // chunk size, a safe value is:
                 // 32767 - longestColumnValue - tagSizes
   set fi="c:\temp\example-t.xml" // inp file (xml)
   set fo="c:\temp\example-t.csv" // output file (csv)

   open fi:"ru":1 open:$t fo:"nw":1
   if '$t close fi quit

   set xml=$$inp(fi,len) // first xml-chunk

   set i=0, p=0
   while 1 {
      set i=$locate(xml,"\<\/?\w+\>",i,j,v) // next (opening or closing) tag

      if i {                                            // let see, what we got 
         if v="<Details>" set row="", p=-1 // start a new row
elseif v="</Details>" out(fo,row) p=0 // complete, write out
elseif p,v["<Col" p=j, o=$zstrip(v,"*AP") // new column, keep start
elseif p,v["</Col" {$li(row,o)=$$val($e(xml,p,i-1)) // get value
         }                                   // everything else is don't care
         set i=j

else {
         set tmp=$$inp(fi,len) // next xml-chunk
         if tmp="" quit                                   // done

         // remove processed data, add new one
         if p>0 set xml=$e(xml,p,*)_tmp,p=1,i=0 else xml=$e(xml,i,*)_tmp,i=0 }
      }
   }
   close fi
   close fo
}

val(val)
{
   quit $zstrip(val,"<>w") // add handling of charcter entities like &lt; etc.
}

out(fo,row)
{
   use fo
   write $listtostring(row,";",1),! // delimiter!
}

inp(fn,len)
{
   use fn
   try read xml#len catch xml="" // in case, $zeof-mode is off
   quit xml
}

The above converter reads a simple test XML-file with two million 'ColX' items in 5 seconds and creates a CSV file with 100000 rows and 20 columns (in each row).

Instead of file you can also use an stream.

If you "donate" your XML-File a version info and an extra root-node:

<?xml version="1.0" encoding="UTF-8" ?>
<Data>
  <Details>
   ...
  </Details>
</Data>

and use, for example, the %XML.Textreader class (see belov).

Then with few lines of code the job is done:

XML ; XML to CSV
#define ROOT "Data"
#define DEL    ";"        ##; your CSV-delimiter, $c(9) or ";" or ...
#;
   set inpFile="c:\temp\example.xml"
   set outFile="c:\temp\example.csv"
   if ##class(%XML.TextReader).ParseFile(inpFile, .rdr) {
      if rdr.Read(), rdr.NodeType="element", rdr.Name=$$$ROOT {
         open outFile:"nw":1
         if $t {
            use outFile
            while rdr.Read() {
               if rdr.NodeType="element",rdr.Name="Details" {
                  set line=""
               } elseif rdr.NodeType="chars" {
                 set line=line_$lb(rdr.Value)
               } elseif rdr.NodeType="endelement",rdr.Name="Details" {
                 w $lts(line,$$$DEL),!
               } elseif rdr.NodeType="endelement",rdr.Name=$$$ROOT {
                  close outFile
                  quit
               }
            }
         } else { w "file open problem",! }
      } else { w "XML root-element problem",! }
   } else { w "XML structure problem",! }

Do you really think it makes a difference if my routine contains "set xx=xx+1" instead of "set xx=1+xx"?

If yes, try the following:

Times2 ; execution time measurement

  s num=0,t=$zh f i=1:1:1E6 { s num=num+1 } w $j($zh-t,8,6),!
  s num=0,t=$zh f i=1:1:1E6 { s num=num+1 } w $j($zh-t,8,6),!
  q

my output values are

USER>d ^Times2
0.047048
0.038218

USER>d ^Times2
0.034727
0.035160

USER>d ^Times2
0.044252
0.036175

USER>d ^Times2
0.045639
0.035366

Both loops are exactly the same! And now, please explain why the times are partly more than 20% different?

With time measurements keep in mind:

- usually, you are not alone on a Cache server
  There are many other processes, some of them belongs to Cache other to the OS
  
- the time resolution (whatever you use: $now(), $zh) is also limited

- it depends also on the time, how long your mesurement runs (you are not alone!)
 

This is my short testroutine:

Times(iter=1E3,count=4) ; show times

    w ?3,"count   num+1   1+num   =$i()    $i()",!
    w ?15,"times in microseconds",!
    w $tr($j("",40)," ",-1),!
    
    f i=1:1:count d time(iter) s iter=iter*10
    q
    
time(iter)
{
    s f=1E6/iter // factor for "one operation in microseconds"
    
    w $j(iter,8)
    s num=0,t=$zh f i=1:1:iter { s num=num+1 } d t($zh-t*f)
    s num=0,t=$zh f i=1:1:iter { s num=1+num } d t($zh-t*f)
    
    s num=0,t=$zh f i=1:1:iter { s num=$i(num) } d t($zh-t*f)
    s num=0,t=$zh f i=1:1:iter { i $i(num) } d t($zh-t*f)
    w !
}

t(t)
{
    w $j(t,8,3)
}


and this is the output


USER>d ^Times(1,8)
   count   num+1   1+num   =$i()    $i()
               times in microseconds
----------------------------------------
       1   2.000   1.000   2.000   1.000
      10   0.100   0.100   0.100   0.200
     100   0.030   0.030   0.080   0.080
    1000   0.044   0.042   0.088   0.090
   10000   0.028   0.028   0.075   0.077
  100000   0.027   0.027   0.064   0.050
 1000000   0.018   0.014   0.031   0.032
10000000   0.011   0.011   0.031   0.032

USER>d ^Times(1,8)
   count   num+1   1+num   =$i()    $i()
               times in microseconds
----------------------------------------
       1   4.000   0.000   2.000   1.000
      10   0.100   0.100   0.100   0.100
     100   0.040   0.030   0.080   0.580
    1000   0.044   0.041   0.088   0.088
   10000   0.028   0.028   0.075   0.077
  100000   0.027   0.027   0.073   0.076
 1000000   0.027   0.021   0.032   0.032
10000000   0.011   0.011   0.031   0.032

USER>d ^Times(1,8)
   count   num+1   1+num   =$i()    $i()
               times in microseconds
----------------------------------------
       1   3.000   1.000   2.000   1.000
      10   0.100   0.000   0.100   0.100
     100   0.040   0.030   0.080   0.590
    1000   0.045   0.041   0.088   0.090
   10000   0.028   0.028   0.075   0.077
  100000   0.027   0.027   0.073   0.075
 1000000   0.015   0.012   0.031   0.032
10000000   0.011   0.011   0.031   0.032

USER>

USER>

USER>d ^Times(1,8)
   count   num+1   1+num   =$i()    $i()
               times in microseconds
----------------------------------------
       1   3.000   0.000   3.000   1.000
      10   0.100   0.000   0.100   0.100
     100   0.030   0.030   0.080   0.630
    1000   0.046   0.042   0.088   0.090
   10000   0.028   0.028   0.075   0.077
  100000   0.027   0.027   0.073   0.075
 1000000   0.014   0.012   0.032   0.032
10000000   0.011   0.011   0.031   0.032

USER>

I consider time measurements only as a rough approximations