New post
Question
· Mar 15, 2016

Managing source code across Caché and Ensemble version upgrades

#Caché #Ensemble

One aspect of source code management is how to deal with the almost inevitable need to upgrade your Caché or Ensemble platform to a newer InterSystems release.

It's long been my experience that InterSystems does a very good job of maintaining backward compatibility. Code that works correctly on, say, 2012.1 is very likely to work correctly on, say, 2015.2 without any modification.

I'd guess that few sites are in a position to take a "big bang" approach and upgrade all of their instances (e.g. dev, test, live, DR) at the same time. So how do we deal with having different instances running different versions?

If we move code between instances by exporting and importing XML files, we can take advantage of this undertaking from the InterSystems documentation:

Unless specified in the upgrade notes for a specific release, exported XML files can be imported into later releases.

The reverse is NOT true. Later versions could be using new features not present in earlier versions, and could have a different class dictionary version number which alters how these classes are stored internally that cannot be undone when importing to a previous version.

In other words, if our code originates in instance A, then moves to B, then finally to C, we should upgrade C first, followed by B, and finally A.

If our code flow is more complex, our upgrade strategy needs to be more sophisticated. Where code can move freely between A, B and C perhaps we need to add a fourth instance D and only allow code to flow into D, never out. Then we can upgrade D to a newer InterSystems release and test our application there. Once verified on D, we might upgrade A, B and C all at the same time. Or spread those upgrades across a relatively short period, disallowing "reverse flow" of our code during the interim period.

Do you have any strategies or experiences to share on this topic?

Discussion (10)1
Log in or sign up to continue
Bill McCormick · Mar 15, 2016

Hi John..

You asked an interesting question here and then you laid out a structure that while perfectly logical broke my heart at the same time. Doing development on the oldest version is a pattern I have seen many times in the wild. But it leads people to a dead end. You can't always be 3-5 years behind the technology curve in your application. That way leads to product failure. You have to have a development process that lets you keep up with new features and key industry trends. If that requirement leads to pain in QD and means you are having to push your base to upgrade that is the process of modernization and relevancy. Fortunately most issues with pushing classes back show themselves at load / compile time as opposed to run time, SQL excepted. I spoke to some people in Ben Spead's team on this today and they try to do the opposite. They try to keep development on the current release and to verify with QD that the code loads, compiles and runs properly on older versions thru automation testing and such.

2 0
John Murray · Mar 16, 2016

I take your point about keeping up with the technology curve, but it's no good the next release of your app using all the whizzy new features of the latest Caché version if you're not able to deploy it to your customers until they can be coaxed to upgrade. Yes, you can use the new features as a carrot to entice them to upgrade, but meanwhile you probably have support commitments to them that mean occasionally patching the app release they're currently using.

0 0
Timothy Leavitt · Mar 15, 2016

Here are two perspectives, from different development workflows:

My team (working on a large Caché-based application) does development on a newer Ensemble version than the version on which our software is released. We perform functional and performance testing against the older version. Most of the time, moving code from newer versions to older works just fine; when it does fail, it tends to be very obvious. The last class dictionary version change was in 2011.1, so that isn’t a concern for upgrades involving recent versions. These used to be much more frequent. Working this way provides a good sort of pressure for us to upgrade when we find new features that the old version doesn’t have or performance improvements on the newer version. It also eases concerns about future upgrades to the version on which the application has been developed.

For Caché-based internal applications at InterSystems, we have separate designated dev/test/live environments. Application code changes typically spend a relatively short time in dev, and a much shorter time in test, before going live. Upgrades are typically done in a short time frame and move through the environments in that order. It would be incredibly risky to upgrade the live environment first!  Rather, our process and validation for upgrades go through the same process as functional changes to these applications. The dev environment is upgraded first; this might take time if there are application issues found in testing after the upgrade. The test environment is upgraded next, typically a very short time before the live environment is upgraded. It's OK if code changes are made in dev before the test environment is upgraded, because changes will still be compiled and tested in the older Caché/Ensemble version prior to going live. Of course, if testing fails, the upgrade may become a prerequisite for the given change to the application. Additionally, we periodically clone the test environment for upgrades to and validation against field test versions. Using virtual machines makes this very easy.

0 0
Ben Spead · Mar 18, 2016

To add one more details to what Tim wrote about how we handle internal applications - we have a partnership with our QD department where at least one during every beta release period they will request a copy of each of our apps and verify compilation, automated unit tests and manual unit tests on that beta version of Caché.  This way if there is going to be any sort of breaking change, we know it before we upgrade our dev environment and we can schedule that upgrade more wisely so as not to halt other development work.  This practice is supported by the fact that we run all of our environments on VMs so it is fairly easy for us to get a new VM cloned off for them from Test and then we are not otherwise impacted in our development workflow.  

We have been able to move to new released much more quickly as a result of this practice

0 0
John Murray · Mar 16, 2016

That's certainly a useful tip. I wonder if it would have helped in a situation we assisted an Ensemble site with last year. They'd upgraded their DEV and QA environments from 2013.1 to 2015.1 but their LIVE was still on 2013.1. Then they amended an HL7 schema in DEV, exported it, imported it to QA, verified it worked, and finally loaded it to LIVE, where it broke the production.

The cause turned out to be a change made in 2014.1 and documented in a bullet point here. Yes, they had been unwise not to retain a QA or pre-live staging environment on the exact Ensemble version that LIVE was running.

0 0
Sebastian Patane · Mar 13, 2018

Our production systems use deployed obj code, so we can't upgrade cache versions without a new set of cache.dat's. We've been thinking a bit about how to best go about this as well.

Right now version jumps are a bit of a major effort and we're lagging behind on 2014.1. We're hoping to solve this with Docker for our linux sites - essentially by bundling cache with our product and distributing it all as one self-contained unit. Not too sure how to do this with windows yet (msi installer?).

That way our dev/test/prod upgrade process becomes "docker run $product:latest" or "msiexec /i $product-latest.msi". Docker/msi contain whatever version of cache the product version requires, rather than expecting the correct version to be installed (and potentially breaking if it isn't) .

Still yet to roll this out - I'll let you know how it goes (or has anyone else done something similar? ;)

OT: I couldn't add this as an "answer" (get the option to type it in, but the only buttons are Save & Preview). Not sure where this comment will go..

0 0