Purging Feeds seperately on Ensemble

Question

Question

Stuart Byrne · Oct 26, 2018

I'm currently working on a project to run our medical device integration through ensemble.

I've noticed a potential problem with the volume of messages . The feed generates around 474K messages daily( average of 1 message per minute per device from 300+ devices).

We currently keep all our other feeds message data for 90 days and purge automatically. From what I can see is that the purge functions in Ensemble are production wide and cannot set it at the Business Operation, Service or Router level.

Ideally we would like to have the new feed in the same namespace & production as our other feeds. With the current purge settings of 90 days, we would need to greatly increase our server specs to cope with the extra data.

We could create a new namespace with its own production and keep a small number of days of data, but would prefer not to do this, so we can keep the number of dashboards (productions) to a minimum.

I wonder if there is a way to separately set purging on each Business service, router and Operation?

Any thoughts would be appreciated.

Kind Regards

Stuart

Discussion (1)0

Log in or sign up to continue

Amir Samary · Oct 26, 2018

Hi!

It is not hard to build a task that will search for your message headers from a specific service, process or operation in a date range and delete the headers and the messages. You can start by searching on Ens.MessageHeader. You will notice that there are indices that will allow you to search by date range and source config item pretty quickly.

BUT:

I always recommend to my customers to create a production for each different business needs. That allow you to have different back up strategies, purging strategies and even maintenance strategies (some times you want to touch one production and be sure that the others will not be touched). You are also able to take these productions to other servers when you need them to scale independently (ops! now this new service is receiving a big volume of messages and is disturbing the others!). On different servers, you may be able to patch one production (one server) while the others are running. This is specially more feasible now with InterSystems IRIS. You can easily migrate from Ensemble to IRIS and have a production running on a VM with 2 cores, another on another VM with 4 cores, and so on. You don't have to start with 8 cores and scale by 4 like ensemble. So you can pay the same you are paying today, but with the possibility of having more governance.

PS: Mirroring on IRIS is paid differently from mirroring with Ensemble. With IRIS, you do have to pay for the mirrors as well while with Ensemble you don't have to.

So, sincerely I believe you will be happier having another production. It's a good start to have more governance.

Kind regards,

Amir Samary

Senior Sales Engineer - InterSystems

3 0