AllAccepted answers
Luca Ravazzolo · Jun 23, 2022

This is amazing to my js-dated eyes. Congrats @Sean Connelly.

Q: Are we back to a full & tight integration & experience that we had in Studio? With BP support and all? Posterity will cast their vote on that but I see a clean unadulterated architecture and I think the option to add Python support so I think we are on a winner :-)

All the best Sean and thanks for sharing it.

Luca Ravazzolo · Jan 19, 2022

Thanks @Mark Bolinsky to highlight those important KPI and links

@David Foard you can automate and have it all displayed and be free to improve the dashboards with System Alerting and Monitoring or SAM that it's free in its community edition.

https://github.com/intersystems-community/Sam

https://docs.intersystems.com/components/csp/docbook/DocBook.UI.Page.cls...
 

Hope this helps

Luca Ravazzolo · Nov 22, 2021

Hi @Trevor Strong 

The container image simply installs the standard Apache package in the container and adds the CSP add-on.

For any update on the Apache web server we should all keep an eye on https://www.cvedetails.com/product/66/Apache-Http-Server.html?vendor_id=45 and consider patching/upgrading/re-building as necessary and according to the security policies and best practices of the organizations we work for.

HTH

Luca Ravazzolo · Oct 12, 2021

Hi Ben,

The short answer is yes, you are correct.

The longer one :-)
Stating the obvious, from a tool point of view, to be able to rollback operations means understanding the present state of an instance and of course have a record of all the previous states. In order to be able to do that one needs the concept of a "release state". As soon as you get into maintaining state you quickly escalate the complexity of a solution. See Terraform for example and ICM itself that supports the replication of its state via Consul.

There are tools like Helm, ArgoCd, etc. that help in that, however that is left to the user. Enhancing InterSystems IRIS is an option but that is not available now. At present we rely on a GitOps approach. 

GitOps is a paradigm that incorporates best practices applied to the application development workflow all the way to the operating infrastructure of a system.

Embracing GitOps give us some benefits like:

  • Deploying faster and more often (with a DB we could argue those adjectives, nevertheless we can still appreciate the benefits)
  • Easier and quicker error handling and recovery
  • Self documenting deployments
  • Increased developer productivity and an enhanced experience for teams
  • Greater visibility on the lifecycle of developed features

However, GitOps itself is not the delivery & deployment panacea of this complex area. GitOps has issues too. There are shortcomings when auto-scaling and dynamic resources are implemented; there is no standard for managing secrets; observability is immature; rollbacks don't have a standard practice, etc.

The powerful CRUD operations that we can run with the CPF merge feature adds to the complexity. A solution needs to be implemented that may leverage one or more tools that organizations use in their automated provisioning pipeline, just like you would do when embracing the GitOps paradigm.

I think there are two ways to solve our rollback issue, at present.
The first one would be a programmatic approach, maybe a diff operation on the git hash declarations of (last_op_def) vs (last_op_def  -  1)
If last_op_def contains a Create-resource I then need to rollback that with a Delete-resource or Modify-resource. And even in this simple case how do you determine that? Human intervention is probably needed.

The second option, simpler and safer, would be to simply re-run the container, the base state we know, and apply configuration settings #1 and #2 only.

There are probably other options involving verifying the CPF file. However, the present CPF file does not hold all of an instance settings.
There are also other issues to these type of automations, like: what if you want to rollback after the creation of a database and data was written to it?

It's complex.

Luca Ravazzolo · Oct 11, 2021

Hi Ben,

Thanks for the pertinent question.

First of all the CPF merge feature is available for containerized and traditional tarball-based installations and configuration. I want to call that out as at times that is unclear in people's mind.

The CPF merge feature allows you to define and apply a delta to your present instance configuration (initial from an install or otherwise from day2). As we know, an instance configuration is described in the Configuration Parameter File or CPF that show us (output mainly) the parameters with which the instance is running.

The CPF merge feature supports CRUD operations and is idempotent. That is useful for three reasons IMO

  1. I can Delete or Update resource settings from the outset (day1). There might be settings that are not desired in a new installation
  2. I can Update a single property from day2. All I would need to do is simply add the modification to the same declaration and stamping it as v2.0
  3. It maps directly to InterSystems IRIS internal operational classes so we know the API is robust

Therefore, if a specific property or resource is not desired anymore, your #3 in your example, by modifying the Web Application Security with an Update/Modify operation or deleting it completely, you would return to your latest state of #2. IOW other settings are not affected.

Your single-instance, single source of truth is still the CPF file.

For an initial list of the CRUD operations see the [Actions] page. Please note how the official list will grow as we will be able to run more tests. 

Let me know if this is helpful.

Luca Ravazzolo · Jul 13, 2021

@Lorenzo Scalese great way of exposing the IRIS internal API! I like it and I like the way the community brings innovation and supports the needs of users. Great effort, Lorenzo!

I also wanted to draw attention to a utlity that InterSystems has been supporting for several versions. We call this feature the CPF merge feature.

Q: What is the CPF merge feature?

A: It's the capability to configure an instance dynamically from the outside. It can be used with any configuration management tool like Chef, Puppet, Ansible, Salt or simpler bash or any cloud service provider provisioning tool like AWS CloudFormation, Terraform or orchestrator like Kubernetes. A user can define the ultima state of an IRIS instance. The operation is executed idempotently and all you need is an environment variable called ISC_CPF_MERGE_FILE=the_file_that_holds_my_desired_config

The CPF merge file could have been JSON, YAML, TOML or whatever but we decided to go with the familiar format we know, for now. The CPF merge file provides a way to Create, Delete and Update instance resources.

The doc.

Some Examples - Note how the CPF merge feature does not only helps us in single instance configuration but also automates more complex cluster configurations like Mirror pairs and shard architecture topologies.

I hope this is useful to the reader who is seeking more elegant and easy ways to automate InterSystems IRIS clusters.

Luca Ravazzolo · Jul 2, 2021

I see both of them

 {
      "RepositoryName": "intersystems/arbiter",
      "Tags": [
        "2019.1.1.615.1",
        "2020.1.0.215.0",
        "2020.1.1.408.0",
        "2020.2.0.211.0",
        "2020.3.0.221.0",
        "2020.4.0.547.0",
        "2021.1.0.215.0"
      ]
    },
    {
      "RepositoryName": "intersystems/arbiter-arm64",
      "Tags": [
        "2020.4.0.547.0"
      ]
    },

--

Command used

docker run --rm carinadigital/docker-ls \
  docker-ls \
    -u luxabc \
    -p abcdefghijklmnopqrstuvxyz0987654321 \
    --registry https://containers.intersystems.com \
      repositories \
        --level 2 \
        --json

--