New post
AllAccepted answers
Julius Kavay · Mar 23, 2020

$listbuild() and all other functions, like $length(), $char() etc.  are  (I call them) basic  functions of COS and have nothing to do with objects. The language of Cache (and of course IRIS )  handles data as  raw (basic) data and not as objects, like some other (but not all) languages.

For example, in JavaScript you can do

 var  text="Some text";
 alert("'" + text + "' has "+text.length + " characters");
 alert("'Some text' has " + "Some text".length + " characters");

because both, a (text)variable and a (text)string have an (object)property name 'length'.

Beside the raw data type COS also has object data type and most of the standard COS functions will accept an object property as argument, for example:

Class Test.Class Extends %RegisteredObject
{
Property Data as %String;
}

set text="Some text"
set obj = ##class(Test.Class).%New()
set obj.Data = "Some text";
write $length(text),!
write $length(obj.Data),!
Julius Kavay · Mar 17, 2020 
$lb(data1, data2, data3, ... dataN) is built as a string of item1 item2 item3 ... itemN

itemN:= <len> <type> <dataN>

assuming, you are on a little-endian CPU and

l1 = $length(data)
l2 = l1 + 1 // for type byte
l3 = l2 + 1 // for the length itself

then the length is as follows

if l1<=253 then len: <l3>
elseif l1<=65534 len:= <0> <l2-lowbyte> <l2-highbyte>
else len:= <0> <0> <0> <l2-lowbyte-loworderword> ... <l2-highbyte-highorderword>

And don't forget,
$lb(123) is not the same as $lb(1230/10), hence we have a $ls() function!

Julius Kavay · Mar 14, 2020

the functions $order() and $query() are your best friends ;-))

Write, for example, a method like this:

ClassMethod(ref)
{
  if $data(@ref)#10 write ref,"=",@ref,! // do something with this node
  set i=""
  for  set i=$order(@ref@(i)) quit:i=""  do ..Show($name(@ref@(i)))
}

So, for testdata like

set tmp(1,2,3)="Joe"

set tmp(1,2,3,4,1)="Paul"

set tmp(1,2,3,4,2)="Paula"

set tmp(1,2,3,5,6)="Tom"

start some testing


do ##class(yourclass).Show($na(tmp))  // shows all entries

do ##class(yourclass).Show($na(tmp(1,2)))  // also shows all entries

do ##class(yourclass).Show($na(tmp(1,2,3)))  // shows Joe

do ##class(yourclass).Show($na(tmp(1,2,3,4))  // shows Paul and Paula

do ##class(yourclass).Show($na(1,2,3,5)))  // shows Tom

do ##class(yourclass).Show($na(tmp(1,2,3,5,6)))  // shows Tom

do ##class(yourclass).Show($na(tmp(2,3)))  // shows nothing

Now, I hope, you can figure out how to get out your desired data

Julius Kavay · Mar 14, 2020 
 set x="1.2.3", y=$name(tmp)
 for i=1:1:$length(x,".") { set y=$name(@y@($piece(x,".",i))) }
 set @y=""
 kill x,y,i

You can put al the above in one line, if you are in terminal. Or put it in a method

ClassMethod setTemp(x) [ PublicList = tmp]
{
 set y=$name(tmp)
 for i=1:1:$length(x,".") { set y=$name(@y@($piece(x,".",i))) }
 set @y=""
}

Or you take the shorthand

set x="1.2.3", @("tmp("_$tr(x,".",",")_")=""""")

But this works only as long as x contains number_period_numeber_period_...No leading zeros, no alphabetics...

Julius Kavay · Jan 12, 2020

A warning about long time open transactions will be placed in cconsole.log and alerts.log (both files in $system.Util.InstallDirectory()_"mgr" directory) - at least  this is the case what I see on a customers system. The entries look like this:

04/05/19-02:31:34:470 (21012) 1 [SYSTEM MONITOR] TransOpenSecs Warning: One or more transactions open longer than 10 minutes. Process id  18624 22208 (only top 5 shown)
08/29/19-14:31:03:802 (18576) 1 [SYSTEM MONITOR] TransOpenSecs Warning: One or more transactions open longer than 10 minutes. Process id  4128 (only top 5 shown)
09/04/19-17:16:19:090 (21344) 1 [SYSTEM MONITOR] TransOpenSecs Warning: One or more transactions open longer than 10 minutes. Process id  25872 (only top 5 shown)

So it should be as easy as monitoring those two logfiles. But how it looks like on an ECP server, don't ask me.

Julius Kavay · Dec 21, 2019

You could try this way (I assume, your image is in *.jpg, *.png, *.gif, *.ico or *.bmp  format and you use a Windows OS):

- save the stream to a temp file
- do $zf(-1,"pathToIrfanView.exe pathToInpFile  /resize=(100,100)  /aspectratio  /convert=pathToOutFile")
- read the output into a stream and proceed as needed

Don't forget, if one of the above path contains a space char, you have to put in quotes the whole path.

/resize=(100,100)  /aspectration  means, the picture will be resized to a max width/height of 100 by maintaning the aspect ratio. Example: input=2000x3000 out =67x100,  input=3000x2000  out=100x67

For Unix exists similar tools to use with $zf(-1,...), for example "identify" to get the dimensions of the picture  and "convert" or "imagemagick" for resizing.

Julius Kavay · Sep 22, 2019

If your routine works in a terminal session (by issuing a DO ^Routinname command) but does not work, when the same routine is started via the Job commannd (JOB ^Routinname) then you lack something in background, what you have in the foreground.

Put an errortrap in your routine and simple report a possible error (and if you wich, additionally the execution progress of your routine).


rvExt2012 ; Test
    set $zt="bgErr"
    // ...
    // ...
    do signal("checkpoint 1") // this is optional
    // ...
    // ...    do signal("checkpoint 2") // this is optional
    // ...
    // ...
    do signal("Done")
    quit

bgErr do signal("Error: "_$ze)
    quit

signal(msg) do $system.Event.Signal($zp, $username_", "_$roles_", "_msg)
    quit

  

Now, in the terminal session enter following  line

 
job ^rvExt2012 do { set $lb(s,m)=$system.Event.WaitMsg("",1) write m,! } while s
Julius Kavay · Sep 10, 2019

The intention behind my post was, to give you one idea (of many other possibilities), how to convert XML to CSV. A empty (chars) element is just one of some other unhandled cases (missing tags, other tags inside of COLx tag, etc.).

If you need some speed and your XML-File obeys following constraints:

- the file is a correct XML-File
- contains only the 'Data', 'Details' and 'ColX' tags
- no selfclosing tags, like <sometag/>

then you could try the QAD-way (quick-and-dirty) of conversion.

Again, below an example routine (without excessive testing).
All ISC people and ordinary programer, please look the other way ;-))

Convert() Public
{
   set len=32000 // chunk size, a safe value is:
                 // 32767 - longestColumnValue - tagSizes
   set fi="c:\temp\example-t.xml" // inp file (xml)
   set fo="c:\temp\example-t.csv" // output file (csv)

   open fi:"ru":1 open:$t fo:"nw":1
   if '$t close fi quit

   set xml=$$inp(fi,len) // first xml-chunk

   set i=0, p=0
   while 1 {
      set i=$locate(xml,"\<\/?\w+\>",i,j,v) // next (opening or closing) tag

      if i {                                            // let see, what we got 
         if v="<Details>" { set row="", p=-1 // start a new row
         } elseif v="</Details>" { d out(fo,row) s p=0 // complete, write out
         } elseif p,v["<Col" { s p=j, o=$zstrip(v,"*AP") // new column, keep start
         } elseif p,v["</Col" {s $li(row,o)=$$val($e(xml,p,i-1)) // get value
         }                                   // everything else is don't care
         set i=j

      } else {
         set tmp=$$inp(fi,len) // next xml-chunk
         if tmp="" quit                                   // done

         // remove processed data, add new one
         if p>0 { set xml=$e(xml,p,*)_tmp,p=1,i=0 } else { s xml=$e(xml,i,*)_tmp,i=0 }
      }
   }
   close fi
   close fo
}

val(val)
{
   quit $zstrip(val,"<>w") // add handling of charcter entities like &lt; etc.
}

out(fo,row)
{
   use fo
   write $listtostring(row,";",1),! // delimiter!
}

inp(fn,len)
{
   use fn
   try { read xml#len } catch { s xml="" } // in case, $zeof-mode is off
   quit xml
}

The above converter reads a simple test XML-file with two million 'ColX' items in 5 seconds and creates a CSV file with 100000 rows and 20 columns (in each row).

Instead of file you can also use an stream.

Julius Kavay · Sep 9, 2019

If you "donate" your XML-File a version info and an extra root-node:

<?xml version="1.0" encoding="UTF-8" ?>
<Data>
  <Details>
   ...
  </Details>
</Data>

and use, for example, the %XML.Textreader class (see belov).

Then with few lines of code the job is done:

XML ; XML to CSV
#define ROOT "Data"
#define DEL    ";"        ##; your CSV-delimiter, $c(9) or ";" or ...
#;
   set inpFile="c:\temp\example.xml"
   set outFile="c:\temp\example.csv"
   if ##class(%XML.TextReader).ParseFile(inpFile, .rdr) {
      if rdr.Read(), rdr.NodeType="element", rdr.Name=$$$ROOT {
         open outFile:"nw":1
         if $t {
            use outFile
            while rdr.Read() {
               if rdr.NodeType="element",rdr.Name="Details" {
                  set line=""
               } elseif rdr.NodeType="chars" {
                 set line=line_$lb(rdr.Value)
               } elseif rdr.NodeType="endelement",rdr.Name="Details" {
                 w $lts(line,$$$DEL),!
               } elseif rdr.NodeType="endelement",rdr.Name=$$$ROOT {
                  close outFile
                  quit
               }
            }
         } else { w "file open problem",! }
      } else { w "XML root-element problem",! }
   } else { w "XML structure problem",! }