All
Accepted answers
Luca Ravazzolo · Apr 11, 2023 go to post

There are already many useful suggestions and tools on display here... amazing community! Thank you!

Luca Ravazzolo · Dec 20, 2022 go to post

Already liking Codespace a lot and the new AI direction Microsoft/Github is working with. I think that is the future as per ChatGPT shows & now with a well presented article for ObjectScript (thank @Dmitry Konnov!) what is there not to like? So many options!

Luca Ravazzolo · Sep 23, 2022 go to post

Hi Dmitry,

Again, thanks for pointing things out. Let me comment on a couple of things as you so strongly say that you do not recommend the facility provided.

The first thing is that the %Installer although it still work it is out of sync with internal classes. It is also cumbersome to feed it through in a declarative way to an IRIS instance. The hope with the CPF merge and its actions is to provide a more modern tool that fits with modern, declarative, gitops paradigms. The third is that containers forces us to think about the distinction between build and run phases and the CPF merge is definitely into the injection of configuration at the run phase. To that point, the actual ```iris merge``` was added later for organizations working with VMs that still want to supply a CPF Merge so that they can more adequately adhere to a declarative approach with tools like Ansible, CloudFormation etc. As I said, we use it daily in the cloud to configure just about anything we need. I know we can now go down a rabbit hole and chase my-use case vs yours so I shall leave it at it and I hope you can understand the intention.

I think we can probably fix the error code returned that comes from within IRIS just like anything else we call as we perform the merge.

It would be good to hear other feedback and what other features we should implement and we hope to hear from a wider audience too.

Luca Ravazzolo · Sep 22, 2022 go to post

Hi Dmitry,

Thanks for trying out the CPF merge utility. I am sorry to hear you found it not very useful because in many areas, especially cloud compositions we use it daily, it's of declarative nature and just aids in just about anything we do with containers.

The utility was created to be indempotent so if you have a database or namespace and you run it again it won't create new resources so, yes, the process run successfully it just did not bother you telling you you made a mistake by submitting the same request twice.

For the password use the following container:

containers.intersystems.com/intersystems/passwordhash:1.1

HTH

Luca Ravazzolo · Jun 23, 2022 go to post

This is amazing to my js-dated eyes. Congrats @Sean.Connelly.

Q: Are we back to a full & tight integration & experience that we had in Studio? With BP support and all? Posterity will cast their vote on that but I see a clean unadulterated architecture and I think the option to add Python support so I think we are on a winner :-)

All the best Sean and thanks for sharing it.

Luca Ravazzolo · Jan 13, 2022 go to post

Hi @David Foard I use Canonical's Multipass. I mean to open source the script that does it all with Mulltipass,  K3s & Longhorn on my Mac. I'll update this note when done.

Luca Ravazzolo · Oct 25, 2021 go to post

Also, please upgrade to the lastest container version available that is 2021.1

Thanks

Luca Ravazzolo · Oct 12, 2021 go to post

Hi Ben,

The short answer is yes, you are correct.

The longer one :-)
Stating the obvious, from a tool point of view, to be able to rollback operations means understanding the present state of an instance and of course have a record of all the previous states. In order to be able to do that one needs the concept of a "release state". As soon as you get into maintaining state you quickly escalate the complexity of a solution. See Terraform for example and ICM itself that supports the replication of its state via Consul.

There are tools like Helm, ArgoCd, etc. that help in that, however that is left to the user. Enhancing InterSystems IRIS is an option but that is not available now. At present we rely on a GitOps approach. 

GitOps is a paradigm that incorporates best practices applied to the application development workflow all the way to the operating infrastructure of a system.

Embracing GitOps give us some benefits like:

  • Deploying faster and more often (with a DB we could argue those adjectives, nevertheless we can still appreciate the benefits)
  • Easier and quicker error handling and recovery
  • Self documenting deployments
  • Increased developer productivity and an enhanced experience for teams
  • Greater visibility on the lifecycle of developed features

However, GitOps itself is not the delivery & deployment panacea of this complex area. GitOps has issues too. There are shortcomings when auto-scaling and dynamic resources are implemented; there is no standard for managing secrets; observability is immature; rollbacks don't have a standard practice, etc.

The powerful CRUD operations that we can run with the CPF merge feature adds to the complexity. A solution needs to be implemented that may leverage one or more tools that organizations use in their automated provisioning pipeline, just like you would do when embracing the GitOps paradigm.

I think there are two ways to solve our rollback issue, at present.
The first one would be a programmatic approach, maybe a diff operation on the git hash declarations of (last_op_def) vs (last_op_def  -  1)
If last_op_def contains a Create-resource I then need to rollback that with a Delete-resource or Modify-resource. And even in this simple case how do you determine that? Human intervention is probably needed.

The second option, simpler and safer, would be to simply re-run the container, the base state we know, and apply configuration settings #1 and #2 only.

There are probably other options involving verifying the CPF file. However, the present CPF file does not hold all of an instance settings.
There are also other issues to these type of automations, like: what if you want to rollback after the creation of a database and data was written to it?

It's complex.

Luca Ravazzolo · Oct 11, 2021 go to post

Hi Ben,

Thanks for the pertinent question.

First of all the CPF merge feature is available for containerized and traditional tarball-based installations and configuration. I want to call that out as at times that is unclear in people's mind.

The CPF merge feature allows you to define and apply a delta to your present instance configuration (initial from an install or otherwise from day2). As we know, an instance configuration is described in the Configuration Parameter File or CPF that show us (output mainly) the parameters with which the instance is running.

The CPF merge feature supports CRUD operations and is idempotent. That is useful for three reasons IMO

  1. I can Delete or Update resource settings from the outset (day1). There might be settings that are not desired in a new installation
  2. I can Update a single property from day2. All I would need to do is simply add the modification to the same declaration and stamping it as v2.0
  3. It maps directly to InterSystems IRIS internal operational classes so we know the API is robust

Therefore, if a specific property or resource is not desired anymore, your #3 in your example, by modifying the Web Application Security with an Update/Modify operation or deleting it completely, you would return to your latest state of #2. IOW other settings are not affected.

Your single-instance, single source of truth is still the CPF file.

For an initial list of the CRUD operations see the [Actions] page. Please note how the official list will grow as we will be able to run more tests. 

Let me know if this is helpful.

Luca Ravazzolo · Jul 28, 2021 go to post

Hi Johan,

When you say "Uber type application" what exactly do you refer to? Their DISCO system? The overall architecture? Their implementation of service oriented architecture? Their supply service or demand service?

They started with a monolith and Python and broke it up later...

Let us know & all the best with the new app!

Luca

Luca Ravazzolo · Jul 13, 2021 go to post

@Lorenzo Scalese great way of exposing the IRIS internal API! I like it and I like the way the community brings innovation and supports the needs of users. Great effort, Lorenzo!

I also wanted to draw attention to a utlity that InterSystems has been supporting for several versions. We call this feature the CPF merge feature.

Q: What is the CPF merge feature?

A: It's the capability to configure an instance dynamically from the outside. It can be used with any configuration management tool like Chef, Puppet, Ansible, Salt or simpler bash or any cloud service provider provisioning tool like AWS CloudFormation, Terraform or orchestrator like Kubernetes. A user can define the ultima state of an IRIS instance. The operation is executed idempotently and all you need is an environment variable called ISC_CPF_MERGE_FILE=the_file_that_holds_my_desired_config

The CPF merge file could have been JSON, YAML, TOML or whatever but we decided to go with the familiar format we know, for now. The CPF merge file provides a way to Create, Delete and Update instance resources.

The doc.

Some Examples - Note how the CPF merge feature does not only helps us in single instance configuration but also automates more complex cluster configurations like Mirror pairs and shard architecture topologies.

I hope this is useful to the reader who is seeking more elegant and easy ways to automate InterSystems IRIS clusters.

Luca Ravazzolo · Jul 2, 2021 go to post

I see both of them

 {
      "RepositoryName": "intersystems/arbiter",
      "Tags": [
        "2019.1.1.615.1",
        "2020.1.0.215.0",
        "2020.1.1.408.0",
        "2020.2.0.211.0",
        "2020.3.0.221.0",
        "2020.4.0.547.0",
        "2021.1.0.215.0"
      ]
    },
    {
      "RepositoryName": "intersystems/arbiter-arm64",
      "Tags": [
        "2020.4.0.547.0"
      ]
    },

--

Command used

docker run --rm carinadigital/docker-ls \
  docker-ls \
    -u luxabc \
    -p abcdefghijklmnopqrstuvxyz0987654321 \
    --registry https://containers.intersystems.com \
      repositories \
        --level 2 \
        --json

--