New post
AllAccepted answers
Laura Cavanaugh · May 12, 2020

I'll debug it today, and check the $storage.  I know I have references hanging around even after I kill objects, because the original Parent object and the cloned object still point to their children -- that is, the code is somewhat disjointed where I remove a child, but the "Youngest" property still points to the removed child, until the next loop, when I point "Youngest" to a new child. At that point, though, I was hoping the object would be removed from memory.

Laura Cavanaugh · May 11, 2020

Here is some pseudo code, redacted a little:

So, A Parent object has a one->many relationship with Child, as defined in the class definition.

Class Parent {
Relationship Children As Child [ Cardinality = many, Inverse = Parent ];

Property MostRecentChild as Child;
}

Class Child {
Relationship Parent As Parent [ Cardinality = one, Inverse = Children ];

// a few other objects, both as parents, and object properties
}

Set ParentID="",Child=""
for {

    // order is by parent, then by child DESC (youngest first, or really, when they were added to our database)

    set Parent  ID=$o(^||TEMP($j,"Parent",ParentID)) q:ParentID=""

    // clones everything; including the Instance's relationship, and the Instance's Objects
    set Parent Obj=##class(Parent).%OpenId(ParentID)
    set clone=ParentObj.%ConstructClone(1)
    for {

      //looping on count as well - just took that part out

     {
       set ChildID=""
       set ChildID=$o(^||TEMP($j,"Parent",ParentID,"Children",count,ChildID)) quit:ChildID=""
       Set ChildObj=##class(Child).%OpenId(ChildID)
       set key=clone.Children.FindOref(ChildObj)  // SEEMS TO WORK! Should it?
       set youngest=clone.Children.GetAt(key)
       set clone.MostRecentChild=youngest
       Set ok=..EvaluateClone(clone) // do lots of stuff here; evaluate as if the clone has only these children
       // here, I'm removing the children from the relationship one at a time
       do clone.Children.RemoveAt(key) // update clone

       //update clone's properties

       //e.g. set clone.TotalAgeOfChildren = clone.TotalAgeOfChildren-youngest.Age

       kill youngest,ChildObj
    }

  }
    kill ParentObj,clone
}

Basically, I have a temp global of the parents, and their children, in reverse "added to database" order. 

A. I clone the Parent

B. set the clone's MostRecentChild to the "youngest" object (which is redundant the first time around), evaluate it with all the clone's current children

C.  then remove the youngest object from the clone.  

D. I get the next Child object, which according to the temp global, is the next youngest in our database.  Loop from B.

I'm evaluating the Parent with fewer and fewer children.  

The objects aren't really this simple; the Child is also the child of a different parent, which is also cloned in the Deep clone, which is a child of a different parent, also cloned. Do I need to kill more stuff?

Assume a Parent has on average 5 children, but could have 1 to 40.  After about 25k children, I get a STORE error.

Thanks,

Laura

Laura Cavanaugh · Nov 7, 2019

Eduard,

Just a note; you are absolutely correct about the $increment value of the ID not getting rolled back, and a simple test of classes saved inside a transaction show that the ID is incremented and not rolled back on error.

All that aside, we're going to stick with the %OnAfterSave, and the slower method of getting the PropertyB ID from a ClassA object. 

Thanks for great info.

Laura Cavanaugh · Nov 7, 2019

Hi Eduard,

We're not using relationships partly because of the time it takes to load up a property that is a relationship, and also this might just be how it was designed in the initial phase of the project, and now we live with it.

(We do use relationships elsewhere).

Good to know about the $increment not getting rolled  back, and I agree that using the ClassA.PropertyB.%Id() isn't a good practice; I will pass that on. I also am going to go make some simple classes and have some fun with them, to see what happens to the Ids.

Thanks for the input!

Laura

Laura Cavanaugh · Oct 17, 2019

And, yes, I just found this: https://cedocs.intersystems.com/latest/csp/docbook/DocBook.UI.Page.cls?KEY=GORIENT_class_method_variables_passing

So in conclusion, the ByRef is basically an "annotation", so that the programmer knows that an object will be changed and returned.  I use it all the time to change objects, on purpose, especially log files, and arrays that I need defined in a single source.

Laura Cavanaugh · Oct 17, 2019

I use ByRef a lot, but mostly to indicate that an array or object will be changed in the method. 

In fact, until now, I thought it was necessary to use ByRef in order to "continue the changes to the object back to the calling code".  However, after playing around with some code, including your example, I see that it's not necessary at all, which makes me think that it's useful as an "annotation", so that the programmer knows that the object WILL BE changed. 

Now, I feel that the real "pass by reference" is the dot in front of the variable, and the ByRef is merely an indicator that this variable could be changed when returned.

Is this true?  I'm not really sure what "passing a reference by reference" would produce. I came upon this article because I want several methods in a row to update a log file, which I sometimes add to the method signature as ByRef, and sometimes I don't, and I wanted to know why it still worked (i.e. my log file is created, and is updated in the calling method).

Here is your original example, but without the ByRef:

ClassMethod Test(UseByRef As %Boolean = {$$$YES})
{
    Set Obj = {}
    Set Obj.Property = 1
    Write "At the begining ",Obj.Property,!
    Do ..RefMethod(Obj)
    Write "After Value pass ",Obj.Property,!
    Do ..RefMethod(.Obj)  // pass by reference because I want the object to change
    Write "After ByRef pass ",Obj.Property,!
}

/// notice there's no ByRef in the method signature

ClassMethod RefMethod(Obj)
{
    Set Obj = {}
    Set Obj.Property = 500
}

USER>Do ##class(Utils.ObjByRef).Test()


At the begining 1
After Value pass 1
After ByRef pass 500

Laura Cavanaugh · Sep 5, 2019

Michael,

Thanks for this tip.  I'm using this utility to programatically export part of a very large global to a file, for archiving data.  I've decided that using the %Global.Export() method is the best way to go, since I have to do this in code without user input from a terminal. I considered ^%GOGEN and $system.OBJ.ExportToStream as well (comments on this?)

Previously I was merging the global part to a temp global in another namespace (the namespace would then be deleted), and then deleting the subscript from the main global.  Obviously this could temporarily greatly increase the amount of space this global uses; hence the switch to something better.

The global I'm archiving is very large, so I'd like to ensure that this utility doesn't copy the global to a temp global, a PPG, or any kind of format that would take up disk space (aside form the space the file is using).  

Do you know if this utility works directly from the global to the file, or does it use any temp globals along the way?

Thanks,

Laura

Laura Cavanaugh · Feb 8, 2019

Thank you all, for the comments.  It sounds like there's not too much of a problem keeping the INT code.  The space needed would indeed be smal compared to the space for the datasets.  It doesn't increase privacy or security.  Rather, it sounds more like the INT code is provided for our benefit, for debugging purposes, and is normally not saved on live systems almost out of tradition (and because who is debugging on live, eh? Well, I am, at times).

Thanks,

Laura

Laura Cavanaugh · Oct 9, 2018

I think the frozen plans from our fairly recent upgrade was indeed the problem, especially taking into account what Wolf said about adding new properties to a table.  I had done that recently, which makes the "SELECT-list and INTO-list mismatch" error more plausible. We upgraded to 2017.2.2.

I coudln't really test unfreezing the plan (which was in a Frozen/Upgrade status) because I had already changed the code to use a new object, %SQL.Statement.  And, this also makes sense, since using a new object probably created a new plan ... ?

We are not that sophisticated -- we don't have a DBA -- I'd prefer to let InterSystems code optimize the queries. We never optimize and freeze plans.  We just add Indexes to make queries faster.  I'll probably leave the queries frozen until we come across this problem again.  Unless I hear that it's OK to simply unfreeze them all.  What's the downside to that?

Thanks for the help,

Laura

Laura Cavanaugh · Oct 3, 2018

Update: using the %SQL.Statement class did help, and I had to change only a few lines of code (the Execute, Next, and Data methods).

Still curious as to why this was happening on our DEV server but not other servers, where the other servers still have the original code using %ResultSet class.  And why now -- we've been using this code for years, and upgraded to 2017 in July.

Thanks,

Laura

Laura Cavanaugh · Jul 12, 2018

I have been killing index globals and rebuilding indices (a lot), because I was also changing how I added the Tag.  

 /// Add Tag,Tag.Tag as Key, to claim array of Tags
Method AddTag(tag As Data.Tag) As %Status
{
set ok=$$$OK
try {
d ..TagsArray.SetAt(tag,tag.Tag)
do ..Tags.Insert(tag)
}
catch (ex) {
set ok=ex.AsStatus() 
}
quit ok
}

I had some version of code yesterday where my last query worked, and used the index.  I imagine that we'll be looking for tags based on the "Tag" property (i.e. 'LLC' or 'TWO') rather than the ID, but we could just use the ID.

Laura Cavanaugh · May 24, 2018

I can do that.  For future queries, I'm getting these 4 errors in the Audit Database for each error:

Description: Attempt to access a protected resource
Reoutine: EMSUpdatePassword+2^%SYS.SECURITY |"^^c:\intersystems\ensembledev\mgr\"|
Namespace: {namespace}
Event Data: <PROTECT>EMSUpdatePassword+2^%SYS.SECURITY *c:\intersystems\ensembledev\mgr\

Description: Attempt to access a protected database
Routine: Store+5^%ETN |"^^c:\intersystems\ensembledev\mgr\cachelib\"|
Namespace: {namespace}
Event Data: <PROTECT>Store+5^%ETN

Description: Attempt to access a protected database
Routine: Value^%STACK |"^^c:\intersystems\ensembledev\mgr\cachelib\"|
Namespace: {namespace}
Event Data: <PROTECT>Value^%STACK

Description: Attempt to access a protected resource
Routine: ^%SYS.SECURITY |"^^c:\intersystems\ensembledev\mgr\"|
Namespace: {namespace}
Event Data: <PROTECT>^%SYS.SECURITY *$TEXT(EMSUpdatePassword+2^%SYS.SECURITY)

I'll ask IS.

Thanks,

Laura