Robert Cemper · Sep 11, 2020 go to post

Confirm. No Caché in containers.
I built one myself a year or more ago. Not real rocket since but quite some effort. 
(late thanks to @Dmitry Maslennikov and  @Luca Ravazzolo )
Now I'd position it as less rocket than before. But not really worth the effort.
I finally grabbed a VM instead for my "exercises". 
Today I would just use WSL2 instead with a default Linux distribution.

Robert Cemper · Sep 11, 2020 go to post

laughI enjoyed it!  reminds me of competitions on how to get some results by a minimum of characters.  eg. list a global.

With expanding / compressing code I Studio [ <ctrl>E, <ctrl><shift>E ]  this is all history

Robert Cemper · Sep 6, 2020 go to post

just reading it:
ERROR <Ens>ErrParsingExpression: Error parsing expression
'GenerateNACK("For NewBorn Msg, PID21 Mother Idendentifier should have MRN number",%ErrorStatus)':
0--------10--------20--------30--------40--------50--------60--------70--------80--^ 83 !!!

ERROR <Ens>ErrInvalidName: Invalid name at offset 83   >  

either variable %ErrorStatus is missing or not an object of type %Status

Robert Cemper · Sep 5, 2020 go to post

simply NO! no OS would allow this.
1 port = 1 listener
but you may build a listener dispatching to various services according to some content / signal you receive.
or using a kind of load balancer to distribute incoming messages

Robert Cemper · Sep 3, 2020 go to post

The Native API connects to IRIS by starting a background JOB
In namespace %SYS you can have access to every fresh started  job.
The routine name is %ZSTART.mac.
You have to create it if not existing yet or to adapt it.
My example is just reduced to your case.
Writing a log in global ^%CHECK is just for testing.
I used Native API and also Studio for testing.

%ZSTART ; User startup routine.
#; cannot be invoked directly
    quitSYSTEM ;
#; InterSystems IRIS starting
    quit
JOB ;
#; JOB'd process begins
    new ipaddr,halt
    set ipaddr=$system.Process.ClientIPAddress()
    set halt=(ipaddr="172.17.0.2")
    set ^%CHECK($I(^%CHECK))=$lb(halt,$job,$ZDT($zts,3,,3),ipaddr,$znspace,$io)
#; add here all checking for allowed IP Address
#; halt if access is not allowed
    if halt halt
    quit
CALLIN ;
#; connect over CALLIN
    quit
LOGIN ;
    quit

This is a test result with Studio and 2 Native API. One  blocked and the other allowed.

^%CHECK=5
^%CHECK(1)=$lb(0,"15044","2020-09-03 16:22:54.667","192.168.0.9","USER","|TRM|:|15044")
^%CHECK(2)=$lb(0,"17100","2020-09-03 16:23:41.247","192.168.0.9","%SYS","|TCP|51773|17100")
^%CHECK(3)=$lb(0,"9880","2020-09-03 16:24:20.612","192.168.0.9","%SYS","|TCP|51773|9880")
^%CHECK(4)=$lb(1,"19544","2020-09-03 16:24:41.925","172.17.0.2","%SYS","|TCP|51773|19544")
^%CHECK(5)=$lb(0,"3268","2020-09-03 16:25:27.691","172.17.0.3","%SYS","|TCP|51773|3268") 
Robert Cemper · Sep 3, 2020 go to post

let's have  your  binary  input in variable data 

JAVA                                  ObjectScript
byte[] data = new byte[1024];                                            ;
Bytebuf buf  = new ByteBuf(data)      set buf=data              ; make your work copy  
byte type = buf.read()                set type=$extract(buf)    ; if type = character 
                                      set type=$ascii(buf)      ; if type = binary
int len = buf.ReadInt()               set len=$zlascii($extract(buf,2,5))           ; if order low->high
                                      set len=$zlascii($reverse($extract(buf,2,5))) ; li order high->low
 
byte[] bodyBuff = new byte[len]       
buf.Read(bodyBuff)                    
String str = new String(strBody)      set str=$extract(buf,6,*) ; take rest of buf


from the description there are 2 open points:

  • is type a binary value or a character 
     
  • for length it could be
  • -         low byte first   so 515  => x/02 03 00 00  ;  default
  • -  or high byte first  so  515 => x/00 00 02 03   ;  need to reverse order

therefore I have added both variants

Robert Cemper · Sep 2, 2020 go to post

OK. there is a bunch of system utilities that are protected from studio.  
Typically the real powerful and sometimes dangerous old stuff.

Robert Cemper · Sep 2, 2020 go to post

That's interesting: 
How did you look for GBLOCKCOPY.int  ?  there never was a  .mac!
And it is even on the news IRIS.

Instance: IRIS
USER>zn "%SYS"
%SYS>zl GBLOCKCOPY
%SYS>p
GBLOCKCOPY ; Fast global copy from database to namespace
         ; Copyright (c) 2020 by InterSystems Corporation.
         ; Cambridge, Massachusetts, U.S.A.  All rights reserved.
         ; Confidential property of InterSystems Corporation.
         n
         i $p($g(^|"^^"_$zu(12)|%SYS("GBLOCKCOPY")),"~",1)'="1.0" k ^|"^^"_$zu(12)|%SYS("GBLOCKCOPY")
         i '$d(^|"^^"_$zu(12)|%SYS("GBLOCKCOPY")) s ^|"^^"_$zu(12)|%SYS("GBLOCKCOPY")="1.0~"
         s POP=0
         w !,"This routine will do a fast global copy from a database to another database or"
         w !,"to a namespace. If a namespace is the destination, the global will follow any"
         w !,"mappings set up for the namespace."
         ;
MENU     ;

If this is missing you may also miss other important pieces:
Check your installation or reinstall it

Robert Cemper · Sep 2, 2020 go to post

This is a late reply and the content my explain why.
There is a set of available classes and methods to fulfill your requirements.
Though you have to be careful as there are side conditions to be considered that require special attention.

You have to wrap these class methods in some routine or class and then plug it into the standard task manager.

From your question I understand 4 different tasks:

  • Compact globals in a database   (=force global to make better use of data blocks)
    ##Class(SYS.Database).Compact(..............)
  • Return unused space for a database (=truncate DB)
    ##class(SYS.Database).ReturnUnusedSpace(........)
  • Defragment a database (=make Blocks contiguous) 
    ##class(SYS.Database).Defragment(........)##class(SYS.Database.BackgroundDefragment).Start(........)  ;;[not so useful for task manager]
  • Compact free space in a database (=really returns disk space)
    ##class(SYS.Database).FileCompact.(........)##class(SYS.Database.BackgroundFileCompact).Start(........)  ;;[not so useful for task manager]

The official documetation also has some examples.
My personal favorite is routine DATABASE.int in namespace %SYS
It covers all these functions for interactive use and illustrates also all possible requirements and limits in detail.
The labels you should check are:
COMPACT()
DBTruncat()
DEFRAG()
FileCompact()

This might also explain why there is not just a method out of the box.

Robert Cemper · Sep 2, 2020 go to post

Dear @Sylvain Guilbaud 
As this is the oldest unanswered question I found I'll give it a try:
One of the most attractive features of DeepSee is its close link to the Caché (now IRIS) database using 
the DSTIME mechanics for updates.
With an external linked DB you don't have this "built-in" support.
Other DB's have similar features but including them requires some "art work".
And that doesn't seem to happen so often.

On the other hand the question was never expired nor closed.
It might happen that new results (almost 5 years later) show up now as it moves to the front.

J'espère que tu vas bien et en bonne santé!  yes
Salutations de Vienne,

Robert (toujours actif) wink

Robert Cemper · Sep 2, 2020 go to post

@Oliver Wilms 
#1) make  sure your ECP Server is willing to accept ECP AplicationServers: 

#2) on your Application Server define your Data Server

%SYS>set Name="MainServer"
%SYS>set Properties("Address")="127.0.0.1" 
%SYS>set Properties("Port")=1972
%SYS>s Status=##Class(Config.ECPServers).Create(Name,.Properties)
%SYS>zw Status
Status=1

see docs: class Config.ECPServers

#3) connect to your Data Server

%SYS>set ecpstate=##class(SYS.ECP).GetServerConnState(Name)
%SYS>zw ecpstate
ecpstate=1      ;; Not Connected
%SYS>set tSC=##class(SYS.ECP).ServerAction(Name,3)   ;; 3 -  Change to Normal
 %SYS>zw tSC
tSC=1
%SYS>set ecpstate3=##CLASS(SYS.ECP).GetServerConnState(Name)
%SYS>ZW ecpstate3
ecpstate3=5   ;; 5 - Normal

see docs: class SYS.ECP

#4) now in SMP:

Robert Cemper · Sep 1, 2020 go to post

So ECP might be even more interesting as you can copy directly to the target DB.
Instead of copy local, copy DB or file, load global.

At that moment what is the file system of your installation? 
The 4GB limit did ring a bell in my head. 

Robert Cemper · Sep 1, 2020 go to post

4GB is a significant amount.
A different approach could be to have just an additional DB  (not journaled)
where instead of a loop you use MERGE to copy the global
MERGE ^|"newdb"| myGLOBAL = ^myGLOBAL
once your collection is done you dismount the deb and just copy it  wherever you need it. 
"newdb" is the extended Global reference. Either just the DB or a helping namespace.

So you can check exactly what data you move.

Of course, this works also across ECP if it is available.

Robert Cemper · Aug 31, 2020 go to post

make sure your / is really just a / and not surrounded by some nonprinting stuff
do a check e.g on https://www.xmlvalidation.com/ 

(ending at line 1 character 183)  could also mean you are missing
<?xml version="1.0" encoding="UTF-8"?>  as first line.

next possible trap: your xml is not encode in UTF-8  (but in some other [windows?]
code after modification with Notepad.exe or similar.

Robert Cemper · Aug 31, 2020 go to post

very quick:  $DATA() returns:

  • 0 no node, no data
  • 1 node exist and has data
  • 10 node exist has descendants but no data
  • 11  node exist has descendants and has data

# is the modulo operator and #10 means you just get the rightmost part of the $Data() result : 1 or 0 (true/false)
So $Data(^EDLIST("DIAG",code),value)#10 means: If the node has data than the content is in  value.
otherwise value is useless, skip it

Robert Cemper · Aug 31, 2020 go to post

@Evgeny Shvarov 
I appreciate that process with  HUMAN verification. 
And there was never any issue with the other  35 applications.
As a massive user, I state:  II is just OK as it is !  Pls. don't change!
 

Robert Cemper · Aug 30, 2020 go to post

OK.
There was no quick shot from the hip out of the community to help me.
So I went to dig into the subject myself and found an acceptable solution.
The result is now available on Open Exchangeand it is also LINUX / UNIX aware and not just for windows

Robert Cemper · Aug 29, 2020 go to post

A test with 102,794 rows confirmed the performance difference.
#1)  on index xitm
Rows selected: 61320 Global Ref: 488,861 Commands: 5.244,585 Cached Query: %sqlcq.CACHE.cls16

#2) on index ycol
Rows selected: 61320 Global-Ref: 133,812 Commands: 3.909,205 Cached Query: %sqlcq.CACHE.cls13 

Reducing the query output to SELECT COUNT(ID) makes the difference even more dramatic
#1) Performance: 0.457 sec  Global-Ref: 574,193 Commands: 2.138,695
#2) Performance: 0.082 sec  Global-Ref: 205,973 Commands:   724,288

Robert Cemper · Aug 27, 2020 go to post

This might make your task easier

^EDLIST("DIAG", "X123")=" internal health"
^EDLIST("DIAG", "X234")=" External health"
. . . 

So you will need this Function / Method to Update your segment:
The class is just a container for the method

Class ED.Update [abstract] {
/// assumption you have the full Segment already in a string
ClassMethod AddDescription(ByRef segment as %String) as %Boolean
{  set code=$piece(segment,"^")
   if $data(^EDLIST("DIAG",code),value)#10 set $piece(segment,"^",2)=value
        quit $test }
}

now all you have to do

/// ...  get the segment from DG1
    if  ##class(ED.Update).AddDescription(.segment)  {
          /// .....Update the segment in DG1
   }

If the code is not defined you just skip the update

Robert Cemper · Aug 27, 2020 go to post

"plan to load into a global"

in which way?  straight global, class, .. ??? pls. explain

Robert Cemper · Aug 26, 2020 go to post

Try:

Class test.Person Extends (%Persistent, %XML.Adaptor) { 
Property Id As %Integer [ Calculated, SqlComputeCode = { Set {*}={%%ID}}, SqlComputed ];
Property Name; 
}

if you don't want to touch the Class you may inherit it from some common Super-Class. 

Robert Cemper · Aug 24, 2020 go to post

With pure routines, this is an almost impossible attempt.
Since DO doesn't  have a differentiator between calling a function or procedure, internal or public or just looping  [ do {}  while ]

If you do it inside a class you have all this information on functions and procedures in %Dictionary.*
So you can define exactly what you search. And using the code generator you may even do it at compile time.