Alexey Maslov · Apr 13, 2017 go to post

My point was if the compiler will go as far as protecting the developer from this type of quit misshap

It seems that this kind of checkup would be difficult to implement because of variety of methods how the code could be branched depending on its call type. E.g. 

 if $quit {
     ...
     quit rc
 } else {
     ...
     quit
 }

or 

 quit:$quit rc
 ...
 ...
 quit

Of course, both code fragments demonstrate not very good coding style, but they are semantically correct. If one gets many false positives from a (hypothetical) code checker, he would likely drop it.

Alexey Maslov · Apr 13, 2017 go to post

... studio will complain when you try and quit a for loop with a value

QUIT from inside a loop is considered quitting a loop rather then a function, so it should always be without a value.

...but it wont complain when you try and quit with nothing on the method

According to language definition, each function can be called as a function (set x=$$function(...)) or as a routine (do function(...)). The call type can be recognized inside the function using a construct:

   quit:$quit ReturnValue quit  ; $quit=1 if called as a $$function()

Methods are compiled to functions and behave the same way.

Alexey Maslov · Apr 6, 2017 go to post

Thank you, Ray.

Most production sites wouldn't plan their backups this way because it means that the only operation you can do on the backup image is restore the whole thing and start Caché.

Another reason of doing so can be the number of articles, docs, learning materials which taught us _always_ perform ExternalFreeze accompanied with ExternalThaw on every external snapshot making.

Alexey Maslov · Apr 5, 2017 go to post

 support staff can't access the System Management Portal

Your support staff may want to keep one open session with SMP for troubleshooting. 

Alexey Maslov · Apr 5, 2017 go to post

crash-consistent snapshot of the entire system, including all CACHE.DAT files, the manager directory, journals, and the WIJ.

Ray, may I ask you to clarify this a bit?

Is any snapshot of the entire system can be considered crash-consistent?

Alexey Maslov · Apr 4, 2017 go to post

if you need to start an ECP data server in a mode that prevents application servers from connecting to it

For such cases we implemented a facility which stops all users' sessions and disables new logons ECP-wide (on application level). Our aim was to prevent extra Caché restarts on data servers by any price, as each restart means loosing the global cache and slowing down of many app functions for ~1 hour or even longer.

Without this facility maybe I'd choose to temporarily stop %Service_ECP.

Alexey Maslov · Apr 3, 2017 go to post

RTFM, should I say to myself. <Database> tag is required, according to documentation:

  Required; within <Configuration>. Defines one or more databases used in the namespace. 

Taking in account that there was no need in any database [re]configuration, the problematic fragment was scaled down to: 

<!-- Configure the database that should exist upto this step: we don't need it, but the tag is required -->
 <Database Name="${MainDB}"
 Create="no"
/>

And it was enough to make it working. Despite of nasty message logged:

2017-04-03 12:32:18 1 CreateDatabase: Creating database USER1 in D:\InterSystems\Cache10\mgr\ with resource 

nothing bad happened neither with CACHESYS, nor with USER1 databases.

Alexey Maslov · Apr 2, 2017 go to post

only need to delete all created files

Good point. Besides, if there was <Configuration> section involved, you need to revert all configuration changes it had done. And it's not as easy as rename previous cache.cpfXXXXX back to cache.cpf, because %Installer commits every single change, so you need to remember which cache.cpfXXXXX was active before you started your experiments.

It seems it's usually easier to remove all changes made by %Installer manually, using SMP and some File Manager tool, or if there are too many of them, just run the setup being tested against another Cache instance which I can easily remove and reinstall from scratch.

Alexey Maslov · Apr 1, 2017 go to post

Thanks for taking part, Eduard. According to this record:

2017-03-31 19:24:51 0 : USER1 is ready to be configured using USER1 and %DB_%DEFAULT

MainDB was neither undefined nor empty.

My post was mainly about %Installer's usage; doesn't it use transactions internally by itself? I didn't notice any transaction tags in it's language definition.

As to the log level, my other scripts (not this one) are intended to be used during Cache installation. I don't know how to adjust the log level in this case, therefore I used to deliberately set it equal to "0" to be sure that my debug messages will be logged; having finished debugging, I just change all 

Level="0"

to

Level="3"
Alexey Maslov · Apr 1, 2017 go to post

Thanks for taking part, Eduard. According to this record:

2017-03-31 19:24:51 0 : USER1 is ready to be configured using USER1 and %DB_%DEFAULT

MainDB was neither undefined nor empty.

My post was mainly about %Installer's usage; doesn't it use transactions internally by itself? I didn't notice any transaction tags in it's language definition.

As to the log level, my other scripts (not this one) are intended to be used during Cache installation. I don't know how to adjust the log level in this case, therefore I used to deliberately set it equal to "0" to be sure that my debug messages will be logged; having been finished with debugging, I just change all 

Level="0"

to

Level="3"
Alexey Maslov · Mar 31, 2017 go to post

I've got into another issue trying to write a code which should configure some global mapping to already existing namespace. Here is its skeleton: 

 Include %occInclude

/// Create simple global mapping (main DB and cachetemp only)
Class %z.Mapping Extends %Library.RegisteredObject
{

/// Application Definition
XData qMS [ XMLNamespace = INSTALLER ]
{
<Manifest>
<If Condition='$L("${NAMESPACE}")=0'>
<!-- Report an error if the namespace wasn't specified -->
<Error Status="$$$NamespaceDoesNotExist">
<Arg Value="${NAMESPACE}"/>
</Error>
</If>

<If Condition='$zcvt("${NAMESPACE}","U")="%SYS"'>
<!-- Report an error if the namespace %SYS -->
<Error Status="$$$GeneralError">
<Arg Value="${NAMESPACE} should not be %SYS"/>
</Error>
</If>

<Invoke Class="%z.Mapping" Method="GetMainDB" CheckStatus="0" Return="MainDB" >
<Arg Value="${NAMESPACE}"/>
</Invoke>

<If Condition='$L("${MainDB}")=0'>
<!-- Report an error if the globals database was not detected -->
<Error Status="$$$GeneralError">
<Arg Value="${NAMESPACE} has no Globals database" />
</Error>
</If>

<Var Name="DBRESOURCE" Value="%DB_%DEFAULT"/>

<Namespace Name="${NAMESPACE}" Create="no" >

<Log Level="0" Text="${NAMESPACE} is ready to be configured using ${MainDB} and ${DBRESOURCE}" />
 
 <Configuration>

<!-- ??? Configure the database that should exist upto this step -->
  <Database Name="${MainDB}"
 Dir="${MainDB}"
 Create="no"
 Resource="${DBRESOURCE}"
 PublicPermissions=""
  />
<!-- ??? Configure the database that should exist upto this step -->

  <GlobalMapping Global="Q(&quot;0!&quot;):(END)"  From="CACHETEMP"/>
  <GlobalMapping Global="Q(&quot;0mseSTATUS&quot;):(&quot;0mseSTATUS~&quot;)"  From="${MainDB}"/>
  <GlobalMapping Global="Qa"     From="CACHETEMP"/>
  <GlobalMapping Global="Qi"     From="CACHETEMP"/>
  <GlobalMapping Global="Qi(&quot;MO&quot;)"     From="${MainDB}"/>
 
  </Configuration>

</Namespace>

</Manifest>
}

/// This is a method generator whose code is generated by XGL.
ClassMethod setup(ByRef pVars, pLogLevel As %Integer, pInstaller As %Installer.Installer, pLogger As %Installer.AbstractLogger) As %Status [ CodeMode = objectgenerator, Internal ]
{
#; Let our XGL document generate code for this method. 
Quit ##class(%Installer.Manifest).%Generate(%compiledclass, %code, "qMS")
}

ClassMethod GetMainDB(pNamespace) As %String
{
new $namespace $namespace="%SYS"
sc=##class(Config.Namespaces).Get(pNamespace,.p)
if 'sc $system.OBJ.DisplayError(sc) set ret="" gmdbq
ret=p("Globals")
gmdbq
quit ret
}

/// Invoke the installer passing in some variables
/// zn "USER1" s sc=$system.OBJ.Load("mapping.xml","ck") d:'sc $system.Status.DisplayError(sc)
/// s sc=##class(%z.Mapping).RunInstallWithLog($zu(12)_"mapping.log",2) d:'sc $system.Status.DisplayError(sc)
/// 
ClassMethod RunInstallWithLog(pLogfile As %String, pLogLevel = 1) As %Status
{
#dim tVars
#dim tStatus As %Status

Set tVars("NAMESPACE") = $zu(5)

// Construct a file logger
#dim tLogger As %Installer.FileLogger = ##class(%Installer.FileLogger).%New(1,pLogfile)

// Invoke the installer
Set tStatus = ..setup(.tVars,pLogLevel,,tLogger)
Do:$$$ISERR(tStatus) $system.OBJ.DisplayError(tStatus)
Quit tStatus
}

}

I've started it at the same Cache instance where I wrote it with following commands:

zn "USER1" s sc=##class(%z.Mapping).RunInstallWithLog($zu(12)_"mapping.log",2) d:'sc $system.Status.DisplayError(sc)

And it works. But if I'm excluding an [unneeded] code fragment that resides between two lines:

I'm getting several errors, please see below. Why this fragment is still needed nevertheless I marked my namespace with Create="no" attribute?

​2017-03-31 19:24:51 0 %z.Mapping: Installation starting at 2017-03-31 19:24:51, LogLevel=2
2017-03-31 19:24:51 0 : USER1 is ready to be configured using USER1 and %DB_%DEFAULT
2017-03-31 19:24:51 1 CreateNamespace: Creating namespace USER1 using /
2017-03-31 19:24:51 2 CreateNamespace: Modifying namespace USER1
2017-03-31 19:24:51 0 %z.Mapping: ERROR #5002: Cache error: <SUBSCRIPT>%LoadData+6^Config.Databases.1 ^SYS("CONFIG","CACHE","Databases","")
ERROR #5659: Property 'Config.Namespaces::Globals(3@Config.Namespaces,ID=CACHE||Namespaces||USER1)' required
ERROR #7202: Datatype value '' length less than MINLEN allowed of 1
ERROR #5659: Property 'Config.Namespaces::Routines(3@Config.Namespaces,ID=CACHE||Namespaces||USER1)' required
ERROR #7202: Datatype value '' length less than MINLEN allowed of 1
2017-03-31 19:24:51 0 %z.Mapping: ERROR #ConfigFailed: Unknown status code: <Ins>ConfigFailed (,,,,,,,)
  > ERROR #5002: Cache error: <SUBSCRIPT>%LoadData+6^Config.Databases.1 ^SYS("CONFIG","CACHE","Databases","")
  > ERROR #5659: Property 'Config.Namespaces::Globals(3@Config.Namespaces,ID=CACHE||Namespaces||USER1)' required
  > ERROR #7202: Datatype value '' length less than MINLEN allowed of 1
  > ERROR #5659: Property 'Config.Namespaces::Routines(3@Config.Namespaces,ID=CACHE||Namespaces||USER1)' required
  > ERROR #7202: Datatype value '' length less than MINLEN allowed of 1
2017-03-31 19:24:51 0 %z.Mapping: Installation failed at 2017-03-31 19:24:51
2017-03-31 19:24:51 0 %Installer: Elapsed time .178622s
Alexey Maslov · Mar 28, 2017 go to post

Timothy,

Both options succeded. Thank you so much! 

Just FYI, your 1st option

<Var Name="AddClassesErrors" Value="&quot;,5202,5373,&quot;" />

generated a code:

 Do tInstaller.SetVariable("AddClassesErrors",""",5202,5373,""")

and the 2nd one

<Var Name="AddClassesErrors" Value="#{&quot;5202,5373&quot;}" />

generated:

 Do tInstaller.SetVariable("AddClassesErrors",tInstaller.Evaluate("#{""5202,5373""}"))
Alexey Maslov · Mar 28, 2017 go to post

Timothy,

Both options succeded. Thank you so much! 

Just FYI, your 1st option

<Var Name="AddClassesErrors" Value="&quot;,5202,5373,&quot;" />

generated a code:

 Do tInstaller.SetVariable("AddClassesErrors",""",5202,5373,""")

and the 2nd one

<Var Name="AddClassesErrors" Value="#{&quot;5202,5373&quot;}" />

generated:

 Do tInstaller.SetVariable("AddClassesErrors",tInstaller.Evaluate("#{""5202,5373""}"))
Alexey Maslov · Mar 28, 2017 go to post

Hi Timothy and Sergey,

Timothy, putting the literal codes list inside a variable doesn't help much: I'm getting the same error. 

Thank you anyway.

P.S. The reason of the issue is a wrong code for variable evaluation, e.g. 

 <Var Name="AddClassesErrors" Value="5202,5373" />

is transformed to

 Do tInstaller.SetVariable("AddClassesErrors","5202","5373")
Alexey Maslov · Mar 22, 2017 go to post

Luca,

It was clear that we can use VMs from the very beginning of the project, and maybe we'll take this approach at last. I've just looked at the Docker's side willing to search more light/reliable alternative.

Thank you again.

Alexey Maslov · Mar 22, 2017 go to post

Thanks, Dmitry and Luca.

Meanwhile I read some docs and interviewed colleagues who had more experience with Docker than me (while w/o Caché inside). What I've got out of it: Docker doesn't fit well to this very case of mine, which is mostly associated with deployment of already developed app rather than with new development. The reasons are:

- Only one containerized app at the client's server (our case) doesn't mean so many benefits as if it would be several ones;

- Windows issues which Luca already mentioned;

- I completely agree that "data should reside on a host volume..." with only one remark: most likely that all this stuff will be maintained at the client's site by not very well skilled IT personal. It seems that in case of Doker/host volume/etc its configuring will be more complex than rolling up Cache for Windows installation with all possible settings prepared by %Installer based class.

Alexey Maslov · Mar 15, 2017 go to post

AFAIK, docker encourages to implement microservices. What about classic Cache based app deployment, can docker be useful in this case? By classic I mean the system accessible via several services (ActiveX, TCP sockets, SOAP and REST), multi-tasking, etc with initial 1-2 GB database inside. At the moment I'm to choose a solution to roll up several dozen of rather small systems of such kind. Should I look at docker technology in this very case?

I am bound to ver. 2015.1.4. Host OS most likely will be Windows 2008/2012, while I'd prefer to deploy our system on Linux, that's why I am searching a light solution how to virtualize or containerize it. Thank you!

Alexey Maslov · Mar 6, 2017 go to post

Several years ago our company (SP.ARM) had developed Krasnoyarsk Kray Regional HIS, which was finally deployed using haproxy to balance CacheActiveX traffic load among 7 ECP application servers. This kind of traffic is based on the same protocol as ODBC, so the behavior should be quite the same (long sessions via 1972/tcp). To achieve high availability of the proxy server, it was deployed in two similar copies connected with ucarp.

Our initial solution for load balancing and high availability of proxy was based on LVS+keepalived, but the engineers of maintainer company (Rostelecom) prefered to use haproxy+ucarp as they felt themselves more comfortable with them; we didn't mind.

Alexey Maslov · Mar 5, 2017 go to post

Hi Michael,
Using Studio's keyboard heavily, I can add to your list:
Shift-F2 - jump to previous bookmark
F6, Shift-F6 - switch to next and previous opened routine/class window
Ctrl-F7 - compile current routine/class
F7 - [re]compile the whole project
Ctrl-F9, F9, Shift-F9 - deal with break points (similar logic as with bookmarks)
Ctrl-E, Ctrl-Shift-E - switch to full commands/functions names, switch back to shortcuts
Ctrl-Shift-O - open project
Ctrl-O - open routine/class/etc.
P.S. Have not tried Atelier yet.

Alexey Maslov · Feb 22, 2017 go to post

Mark, may I ask your for some clarification? You wrote:

As a result THP are not used for shared memory, and are only used for each individual process. 

What's a problem here? Shared memory can use "normal" huge pages, meanwhile individual processes - THP. The memory layout on our developers' serber shows that it's possible.

# uname -a

Linux ubtst 4.4.0-59-generic #80-Ubuntu SMP Fri Jan 6 17:47:47 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

# cat /sys/kernel/mm/transparent_hugepage/enabled
[always] madvise never

# tail -11 /proc/meminfo
AnonHugePages:    131072 kB
CmaTotal:              0 kB
CmaFree:               0 kB
HugePages_Total:    1890
HugePages_Free:     1546
HugePages_Rsvd:      898
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      243808 kB
DirectMap2M:    19580928 kB
DirectMap1G:    49283072 kB

# ccontrol list

Configuration 'CACHE1'
        directory: /opt/cache1
        versionid: 2015.1.4.803.0.16768
        ...
# cat /opt/cache1/mgr/cconsole.log | grep Allocated
...
01/27/17-16:41:57:276 (1425) 0 Allocated 1242MB shared memory using Huge Pages: 1024MB global buffers, 64MB routine buffers

# grep -e AnonHugePages  /proc/*/smaps | awk  '{ if($2>4) print $0} ' |  awk -F "/"  '{print $0; system("ps -fp " $3)} '
...
/proc/165553/smaps:AnonHugePages:      2048 kB
UID         PID   PPID  C STIME TTY          TIME CMD
cacheusr 165553   1524  0 фев07 ?     00:00:00 cache -s/opt/cache1/mgr -cj -p18 SuperServer^%SYS.SERVER
...

Alexey Maslov · Feb 14, 2017 go to post

Just to complete an exercise: 

--- inst-guid.sh ---
#!/bin/bash
csession CACHE <<EOF | grep -E [0-9A-F\-]{36}
write ##class(%SYS.System).InstanceGUID()
halt
EOF
--------------------

$ ./inst-guid.sh
8BCD407E-EE5E-11E4-B9C2-2EAEB3D6024F
$
Alexey Maslov · Feb 14, 2017 go to post

You can get an output from a Caché routine provided it do some output to its principal device (= stdout), e.g. (approach #1):

$ csession CACHE "##class(%SYSTEM.License).CKEY()"

Cache Key display:
Based on the active key file '/vol/cachesys/mgr/cache.key'

     LicenseCapacity =   Cache 2017.1 Enterpriser - Concurrent Users:1000000, Anything You Need
     CustomerName =      ZAO XX.XXX
     OrderNumber =       9876543210
     ExpirationDate =    10/15/2114
     AuthorizationKey =  3141592653592718281828662607004081
     MachineID =

     currently available =    9997
     minimum   available =      97
     maximum   available = 1000000
$

as there is no way to pass an arbitrary text from COS routine to OS directly. To bypass it, just incorporate into your shell script a calling sequence like this (approach #2):

#!/bin/bash
csession CACHE <<EOF >output.txt 
write ##class(%SYS.System).InstanceGUID()
halt
EOF

After calling it, you will get output.txt like this:

Node: tdb01.xxxxx.com, Instance: CACHE

USER>
8BCD407E-EE5E-11E4-B9C2-2EAEB3D6024F
USER>

(Missing 'halt' command causes an error). All you have to do is to parse an output. To avoid this, you may want to write a small wrapper in COS that will provide an output as you need it, than you'll be able to follow an approach #1.

HTH.

Alexey Maslov · Feb 3, 2017 go to post

Thank you, Timur and other CPM contributors, you are doing the great job.

Freedom and independence are 2 greatest values which you can not buy for money. And you should keep it as long as possible.

This great slogan should be set in stone and cast in steel! Meanwhile, your (potential) target audience could be wider prefer you building FOSS product keeping proprietary COS/Caché features usage minimized and localized as much as possible.

While we at SP.ARM would hardly drop our home-brewed PM as it is pretty well integrated with our flagship product (qMS), personally I appreciate emerging of CPM and wonder why it is not CMPM?

Alexey Maslov · Jan 26, 2017 go to post

Class methods calls "decorating" with macros seems to be possible using #def1arg macros.

But how to automate the substutitution of all flavors of an instance method calls to macros in a real world project? The method calls may use different <oref> variables in different parts of a project, or even use ".." or $this:
 set res=oref.Method1(x,y,.z)
 do Ref.Method1(x,y)
 if ..Method1(x,,.z) { ... }
 set a=$this.Method1(x,y)

So we need to define a macro with at least two arguments: 1) for oref, 2) for all other method's arguments. Having no variable macro arguments list, how to achieve it?

Note: we discussed direct calls only; taking in account $method() and indirection (@, Xecute), the situation would be even worse.

Alexey Maslov · Jan 25, 2017 go to post

Dynamic dispatch is a hit on speed anyway.

Sounds like yet another reason not to use it in production code.

Macros can help in many cases but using $method() and other flavors of indirect method calls.

Alexey Maslov · Jan 25, 2017 go to post

Dynamic dispatch is a hit on speed anyway.

Sounds like yet another reason not to use it in production code.

Macros can help in many cases but using $method() and other flavors of indirect method calls.