Could you please share those Terraform configs, or at leats key part of them (concerning Cache.cpf modifications and alike).

This sounds very interesting.

I could not give any data proven onclusion without looking into sar or mgstat data, but from your words it sounds like the bottleneck here is ObjectScript VM or engine interprocessor locks implementation. This is hard to believe taking into accont that we are talking about "io bound" experiment, but if you will show us sar metrics...

Few easy questions first:

- how much memory did you allocate for your global buffers?

- Did you see ^mgstat statistics at the moment your code was busy walking over huge globals?

- and did you play with global prefetching in this case?

P.S.

Let put aside write amplification problem, disable global modifications and attack read performance first.

Given the returned from Quote^%qcr expression you could use XECUTE to reevaluate the string, i.e.:

DEVLATEST:22:51:39:USER>set q= $$Quote^%qcr(lb)
 
DEVLATEST:22:51:54:USER>x "s u = "_q
 
DEVLATEST:22:52:30:USER>zw u
u=$lb(1,2,3,",",5)

ZWRITE command is implemented in the ObjectScript, and if you are happy with the way it's quoting $LB then you could reuse it's core functionality, i.e.

DEVLATEST:22:47:39:USER>set lb = $listbuild(1,2,3,",",5)
 
DEVLATEST:22:47:41:USER>write $$Quote^%qcr(lb)
$lb(1,2,3,",",5)

Yes, we need a way to change your own vote after you've accidentally pushed the wrong star (and the chance to push wrong start is dramatically increased if you have fat fingers and touchscreen. Done it many times :( ).

Let see to the keypad again, it's getting obvious instantly that keys are (mostly) located by groups of 3 symbols, and if there would not be those "s" (corresponding to "7777") and "z" (which produces "9999") then implementation will be simple formula with division by 3 and corresponding modulo. Also space is exception and is not a part of sequential numerics.

So given this assumprion let us create the 1st approximation (no compression, not name reduction, everything ie readable and commented):

0(s) public {
    set S=""
    for %=1:1:$length(s) {
        set P=""
        set = $e(s,%)
        set = $a(c) - $a("a")
        if c=" " 
            set P="0"
        elseif c="z" {
            set P="9999"
        elseif c="s" {
            set P="7777"
        elseif i<($a("s") - $a("a")) // ($a("s") - $a("a")) = 18
            set n=i\3+1,m=i#3+1
            set before2 = $a("1") ; 49
            set $p(P,$c(+ before2),m+1)=""
        else {
            set n=i-1\3+1,m=i-1#3+1
            set before2 = $a("1") ; 49
            set $p(P,$c(+ before2),m+1)=""
        }
        set:$extract(S,*)=$extract(P,1) S=S_" "
        set S=S_P
    }
    quit S
}

[Don't botther to count symbols - we will compress the code a bit]

This strange `set $piece(string,symbol,offset+1) = ""` is actually filling of a string with the given symbol.

Let us review those several ifs, we see, actually, 2 groups of them:

  • 3 ifs for exceptions fom formulae;
  • 2 ifs for disjointed rows of groupd by 3 keys. The formulae is atually the same, but witth some offset.

So let's get rid of ifs via $select and extra offset itroduced.

#; get rid of ifs, replace with $selects
1(s) public {
    set S=""
    for %=1:1:$length(s) {
        set P=""
        set = $e(s,%)
        set = $a(c) - $a("a") ; $a("a")=97
        set = '< 18 ; ($a("s") - $a("a")) = 18
        set $p(P, $c(- o\3+1 + $a("1")), - o#3+1+1)=""
        set P=$select(c=" ": "0", 
                      c="z": "9999",
                      c="s": "7777",
                      1:P)
        set:$extract(S,*)=$extract(P,1) S=S_" "
        set S=S_P
    }
    quit S
}

[I believe this step is still obvious]

[[I dislike the way I had to put expressions without pairs but we need as short as possible, sothis is inevitable evil.]]

Now this is time to get shorter, but stiill readable version:

#; name reduction
2(s) public {
    S=""
    %=1:1:$l(s) {
        P=""
        = $e(s,%)
        = $a(c) - $a("a") ; $a("a")=97
        = '< 18
        $p(P, $c(- o\3+1 + $a("1")), - o#3+1+1)=""
        P=$s(c=" ": "0", c="z": "9999", c="s": "7777", 1:P)
        s:$e(S,*)=$e(P,1) S=S_" "
        S=S_P
    }
    S
}

That was simple- select code in Studio, then press Ctrl+Shift+E. 

And the latest step, is to convert this barely readable code to 1 line mess hard-code stuff:

#; linearization
3(s) public {
 S="" %=1:1:$l(s) {P="",c=$e(s,%),i=$a(c)-97,o=i'<18,$p(P,$c(i-o\3+50),i-o#3+2)="",P=$s(c=" ":"0",c="z":"9999",c="s":"7777",1:P) s:$e(S,*)=$e(P,1) S=S_" " S=S_PS
}

That's (if you count the 1st indent symbol) 173 characters length.

P.S.

Eduard Lebedyuk has improved this result a little bit:

  • he has replaced quoted literals with numerics (because they are both represend canonical numerics
    from ObjectScript point of view);
  • and replaced (where possiblle) comparisons of characters with comparisons of their derived ordinals (minus 97, or "a")
#; +Eduard modifications
4(s) public {
 S="" %=1:1:$l(s){P="",c=$e(s,%),i=$a(c)-97,o=i>17,$p(P,$c(i-o\3+50),i-o#3+2)=P,P=$s(i<0:0,i=25:9999,i=18:7777,1:P),S=S_$s($e(S,*)=$e(P):" "_P,1:P)S
}

We are 158 now!

157 is impressive: my first result was 174, which Eduard has improved to 161. I didn't thought it might be improven even further :)

Yes, this is very long story. :)

But in any case, there is [better spelled] system method, which returns endianness flag from kernel code:

DEVLATEST:16:58:22:SAMPLES>w ##class(%SYSTEM.Version).IsBigEndian()
0
DEVLATEST:18:09:56:SAMPLES>w $system.Version.IsBigEndian()
0