New post
AllAccepted answers
Alexey Maslov · Jan 25, 2017

Always call "do a.AmethodName()"

I don't think it's a good idea as each extra nested call inevitably introduces extra performance drawback.

Anyway, imagine an existing large app with 100x classes and 1000x of methods calls, some of which can be done indirectly. In most common case an ability to do such transformation (all a.m() to a.Am()) sounds unrealistic, while your approach may fit small projects with simple calling conventions, especially when they are being started from scratch.

Alexey Maslov · Jan 25, 2017

1) The following:

USER>do a.Awtf()
 
    do $method(, "OnBefore" _ Method, Args...)
    ^
<METHOD DOES NOT EXIST>%DispatchMethod+2^Utils.Annotations1.1 *OnBeforenBeforenBeforenBeforenBeforenBeforenBeforenBeforenBeforenBeforenBeforenBefo,Utils.Annotations1
USER 58d1>

is not a great problem as can be easily corrected by adding a check whether a method named $e(method,2,*) really exists in the class.

2) The whole approach seems to be too complicated as you omitted an important step:
all calls of all app's methods should be switchable from "do a.methodName" to "do a.AmethodName" back and  forth. How to implement this?

Alexey Maslov · Jan 23, 2017

My answer would be simple: monitor your database growth and do the same on global level. It can help to estimate the future growth and with some luck even to point out inefficient solutions that may waste the space. 

E.g. we have a task that calculates all globals sizes (usually once a night) and store this data in a global that can be used as a source for visualization and allerting by means of some external toolset. At the moment integration with Zabbix is implemented.

Alexey Maslov · Jan 23, 2017 
 set mbSize=$$AllocatedSize^%GSIZE($Name(@Gref))/1048576 // Global @Gref size in MegaBytes

Pros:
- uses proven calculation algorithm, the same as interactive %GSIZE utility uses (we met errors in %GlobalEdit::GetGlobalSizeBySubscript() calculation, while it was in "rather old" v.2015.1.2, not sure about 2017.1);
- supports the whole global (Gref="^A") as well as its parts (Gref=$name(^A("subs1") or Gref=$name(^A("subs1","subs2")) or ...) as arguments.
Con(s):
- allows to perform only quick global size estimation, while in most cases it's quite enough;
- presize (slow) version can also be derived from %GSIZE source, but it's more tricky exercise...

Alexey Maslov · Jan 19, 2017

Murray, did you mean that LVM snapshot volume is a sequential file in contrast to VMWare snapshot delta disk which is random access one? So, with the similar DB write load the snapshot size would be greater in case of LVM, won't it?

I tried LVM snapshot and backup with Caché freezing once upon a time. AFAIR, the space should be preliminary reserved inside LV for snapshot volume, and if it was too low the snapshot could fail. 

Alexey Maslov · Jan 17, 2017

Hi Timur, thank you for the thorough writing! 

BTW, our company uses home-brewed package manager (while we don't call it that way) for our app updates distribution. Typical package for Caché comprises classes and routines in XML format, globals in FF (our proprietary) format, and a manifest file which contains the update description as well as its before- and after installation actions.

Alexey Maslov · Jan 17, 2017

You may also have a look at The Universal MUMPS(M) commander (Alt-NC) (http://minimdb.com/tools/altnc412.html).
It is similar to other commanders well known outside of Caché/M communities, such as Midnight Commander (mc, Linux), File and Archive Manager (far.exe, MS Windows), and of course their predecessor famous Norton Commander (nc, MS DOS) in many ways. Full screen editor which is included into commander is very simple and easy to learn. It is not so sophisticated as great programmers' tools of the past (such as DEC's EDT), but a bit smarter than notepad (MS Windows) as it is full functional without a mouse.

Pros: it runs from a command line, it does't force any extra tcp port to be opened, it supports as much terminal types as OS does, it has a non-% version so you don't need CACHESYS:RW privilege to load the code.

Cons: the screen design is pretty ancient, but should we expect too much from CHUI application which roots come from early 1990x?

Alexey Maslov · Jan 13, 2017

Hi Murray,
thank you for continuing your series.

Don't you think that VM image backup (despite of its importance) has a drawback as it may contain pretty huge amount of data that is unnecessary for simple database restoration? E.g., VM image may contain hundreds of gigabytes of journals useless for the database state in backup file. IMHO, in this case a kind of selective backup can be attractive. Not aware of Veeam, but sure that Acronis can do it on file system level. I wonder if selective external backup (e.g., in the case of Veeam) can be integrated with Cache DB freeze/thaw features with the same ease as a full one?

Alexey Maslov · Dec 29, 2016

c. Map the relevant globals to a non-journaled database

Sometimes it's being done just to conceal some typical app level drawbacks (such as missing the cases when  temporary globals can be used, (excessive) looped rewrites of persistent data, etc) although it may lead to more serious administrative level drawbacks.

Normally all app's globals (whether they are journaled or not) should by coordinated with each other; if a global should not (or may not), it is a good candidate to be mapped to CACHETEMP DB. Any fail-over scenario you may imagine includes a step of journal restoration. Believe me, it can be a great problem for admin to decide what to do with each non-journaled global after the server fault: which one can be fully (or partially) KILLed, and which one need to be ^REPAIRed. Mostly the right decision is impossible without the developer's intervention. So, the simple dev level solution can introduce much more serious admin level complications.

IMHO, a global should be considered of one of two types basing on its content type:
1. "normal" persistent data: such global should be placed in journaled DB without any exceptions;
2. temporary data which is need only during process run: map it to CACHETEMP or make it private ^||global.

Sometimes I was told by developers that a third type exists which comprises some pre-generated data that is stored to improve performance of some heavy jobs (e.g. reporting). The pre-generation process can be time (and resource) consuming, so it looks like that the best place for this 3d type globals is a non-journaled DB. After several database reparations I'd answer "No!". Depending on pre-generation process details, each of these globals can (and should) be put in one of two categories taking in account that reboots of modern servers are relatively rare events:
- if it's not too hard to regenerate the data: just automate it in your code and map the global to CACHETEMP;
- if not, consider your global as operational one and place it into journaled DB; to reduce excessive journaling during its generation, just choose between approaches "e" or "d" of Tani's article.

Alexey Maslov · Dec 8, 2016

Hi Murray,
Speaking of ECP, we usually imagine distributed data processing on several app servers. But what about distributed databases? E.g., can the solution to split the databases among several data servers just to distribute heavy journal and/or write daemon load be smart in some cases?
I see some drawbacks of this solution:
#1 AFAIK, there is no distributed transactions support in Caché.
#2 To couple it with mirroring, one should deploy N mirrors, where N is the number of (primary) data servers; having no option of "coherent mirroring", their N backup members can have different latency against their primaries, so (baring in mind #1) switching mirror members can have worse consequences as in traditional case of only one (primary) data server.

Have I missed something? Maybe you've seen some field cases when distributing databases looked smart, haven't you?

Thank you,

Alexey Maslov · Dec 6, 2016

Presumably, it's a security issue. Check effective UID and GID of your Caché processes. To do it, you may check parameters.isc file from Caché install directory for lines like these: 

security_settings.cache_user: cacheusr
security_settings.cache_group: cacheusr

Unlikely user cacheusr has access rights to other user's home directory.

csession processes are the exception from others as they inherit calling user's UID.

IMHO, it's better to use some neutral folder for file exchange, e.g. "/tmp/myexchange", as in this case it's much easier to establish appropriate assess rights for each side involved in exchange.

P.S. UNIX® Users, Groups and Permissions stuff is well-documented, see:  http://docs.intersystems.com/latest/csp/docbook/DocBook.UI.Page.cls?KEY=...

Alexey Maslov · Nov 27, 2016

Eugene mentions OMI and ECP as traditional interconnection kinds of transport. None of them is quite secure, one may check docs for proof (as to ECP: http://docs.intersystems.com/latest/csp/docbook/DocBook.UI.Page.cls?KEY=... ).

ECP is defined as a basic service in contrast to resource based ones. Basic services can't efficiently use resource/roles/users security model by design. It's understood as their layer is usually too low to apply it.

Eugene's services can be dealt with the same precautions as other basic services, i.e. inside secured perimeter only. Port 500x can (and should) be closed on external firewall.