This is a very interesting discussion. I do get the point that while our code looks like looping through a set of fields, behind the scenes the entire physical data block containing each item is pulled into memory. And if our items are inside parent objects in a p / c relationship, then the children are located in the same block and this is quite inefficient if and when we are just interested in the parents. Definitely something I hadn't properyly considered before.

But as Peter points out, mostly we are interested in looking at / processing both together, that's why they are modelled as P / C in the first place. And the marketing spiel of Intersystems for the last few decades was that the "clustering effect" achieved by hiearchical global tree structures was what made Globals so efficient, as data that is commonly used together is stored physically together.

So my personal take-away from this is not that p / c relationships are obsolete but that I need to consider more carefully the use cases for data access, in particular how likely it is that the top level information only will be required, before committing to it.

Hi Alexey

 Strings longer than a half of a block are not cached on application servers. In reality, this threshold is a bit lower – around 3900 bytes for 8 KB blocks. This decision was made by the developers to keep cache clean of BLOBs and CLOBs: such data is usually written once and is rarely read afterwards.

I believe this ECP caching limitation has been addressed in v2016.1. From the release notes:

Enhanced caching on ECP application servers.  Application servers now cache big string data values.  ECP also automatically prefetches some database blocks and caches them on application servers in advance.

Have you got an example of how to use the Pause and Resume methods?

All the examples I've seen so far use the Queue and WaitForComplete methods, i.e. the process that has access to the workmgr object is waiting for everything to be completed and is therefore not accepting any other signals.

Please find below this test code of mine, using QueueCallBack and Wait methods that allows stopping the work, even if it's not very elegant. But it looks as if the Wait method is waiting for %exit =1 only. How would I suspend / resume?

Class DEV.WK.Tst1 Extends %RegisteredObject
{

ClassMethod Run() As %Status
{
^wk.Tst1
sc=..ManageQ()
sc
}

ClassMethod ManageQ() As %Status
{
sc=$$$OK
queue=$SYSTEM.WorkMgr.Initialize("/multicompile=1",.sc) if ('sc) sc=$$$ERROR($$$GeneralError,"Error initialising work queues: "_$system.Status.GetErrorText(sc)) sc }
for i=1:1:100 {
startchunk=i*1000,endchunk=((i+1)*1000)-1
sc=queue.QueueCallback("##class("_$classname()_").Work","##class("_$classname()_").QCallBack",startchunk,endchunk) if 'sc sc=$$$ERROR($$$GeneralError,"Error queuing up work: "_$system.Status.GetErrorText(sc)) return sc }
}
sc=queue.Wait(,.AtEnd) ^wk.Tst1("AtEnd")=AtEnd
if sc&&'AtEnd sc=$$$ERROR($$$GeneralError,"Processing halted") }
queue.Clear()
sc
}

ClassMethod Work(StartId As %Integer, StopId As %Integer) As %Status
{
^wk.Tst1("work",$j,StartId)=""
60
$$$OK
}

ClassMethod QCallBack(StartId As %Integer, StopId As %Integer) As %Status
{
if $d(^wk.Tst1("work",%job)) ^wk.Tst1("work",%job,StartId)=1 }
if $g(^wk.Tst1("halt"))=1 { %exit=1 }
$$$OK
}
 

ClassMethod Halt() As %Status
{
^wk.Tst1("halt")=1
$$$OK
}

}

Can I confirm that it is possible to set up a mirror with an async reporting mirror member only, i.e. I'm not interested in mirroring as a fail-over mechanism (in this specific context). I just want a reporting server with production data that serves ad-hoc queries instead of these queries being run directly against my operational data processing engine.

Further, can you please tell me why I would chose an async mirror over shadow journalling? 

Hi Kyle

When / why would I chose an iFind index over a full text index

INDEX MissionKW ON Mission(KEYS) [ TYPE=BITMAP ];

that I can search via SQL as such

SELECT * FROM Sample.Company WHERE Mission %CONTAINS ('agile')

 

That SQL code seems a lot more intuitive then the one one required by iFind where I have to know the index name.

This looks very promising so far, thank you!

Just in case you are in the market for ideas on potential new features:

- It would be nice to have the option to only render persistent classes (i.e. to make it look a bit more like an E-R diagram).

- in UML associations are depicted as annotations on the arrows, rather than foreign key properties of the pointing class. Again, would be nice to toggle this via an option (i.e. to be able to make it look a bit less like an E-R diagram)

- the ability to control what to show for an individual class, e.g. I have a mega class (in terms of number of properties) that is fairly core to the system and therefore many classes point to it. I would ideally like the ability to just show the class name itself and not its members as that would unnecessarily dominate some diagrams.