Article
Lorenzo Scalese · Apr 15, 2021 6m read

Environment setup with config-api

Hi Developers,

Writing a script for the application deployment can be very interesting to ensure rapid deployment without forgetting anything.
config-api is a library to help developers to write configuration scripts based on a JSON document.

Implemented features :

  • Set system settings.
  • Set security settings.
  • Enable services.
  • Configure namespaces, databases, mapping.
  • Export existing configuration.
  • All features are exposed with a RESTful API.

This library is focused on IRIS configuration to help applications deployment. So, config-api doesn't import\compile code feature, considering it should be the role of your application installer module or the client registry. config-api could be used with the ZPM client to configure IRIS settings on module deployment, we learn how to combine this library with ZPM in another article.

Install

zpm “install config-api”

If you aren't a ZPM user, download the latest version in XML format with dependencies release page import and compile.

First step

Let’s write a simple configuration JSON document to set a few system settings.
In this first document we :

  • Enable journal Freeze on error.
  • Set limit journal size to 256 MB.
  • Set SystemMode to development.
  • Increase the locksiz.
  • Increase the LockThreshold.
Set config = {
  "Journal": {                                /* Service class Api.Config.Journal */
       "FreezeOnError":1,
       "FileSizeLimit":256
   },
   "SQL": {                                    /* Service class Api.Config.SQL */
       "LockThreshold" : 2500
   },
   "config": {                                 /* Service class Api.Config.config */
       "locksiz" : 33554432
   },
   "Startup":{                                 /* Service class Api.Config.Startup */
       "SystemMode" : "DEVELOPMENT"
   }
}
Set sc = ##class(Api.Config.Services.Loader).Load(config)

configuration JSON document structure

The first level keys (Journal,SQL,config,Startup) are related to classes in %SYS namespace (using an intermediate class in Api.Config.Services package). It means Journal support all properties available in Config.Journal, SQLall properties in Config.SQL, etc...

Output :

2021-03-31 18:31:54 Start load configuration
2021-03-31 18:31:54 {
  "Journal":{
    "FreezeOnError":1,
            "FileSizeLimit":256
  },
  "SQL":{
    "LockThreshold":2500
  },
  "config":{
    "locksiz":33554432
  },
  "Startup":{
    "SystemMode":"DEVELOPMENT"
  }
}
2021-03-31 18:31:54  * Journal
2021-03-31 18:31:54    + Update Journal ... OK
2021-03-31 18:31:54  * SQL
2021-03-31 18:31:54    + Update SQL ... OK
2021-03-31 18:31:54  * config
2021-03-31 18:31:54    + Update config ... OK
2021-03-31 18:31:54  * Startup
2021-03-31 18:31:54    + Update Startup ... OK

Trick : The Load method is compatible with a string argument, in this case, the string must be a filename to a JSON configuration document (stream object is also permitted).

Create an application environment

In this section, we write a configuration document to create :

  • A namespace "MYAPP".
  • 4 databases (MYAPPDATA, MYAPPCODE, MYAPPARCHIVE,MYAPPLOG)
  • 1 CSP Web Application (/csp/zwebapp).
  • 1 REST Web Application (/csp/zrestapp).
  • Setup globals mapping.
Set config = {
    "Defaults":{
        "DBDIR" : "${MGRDIR}",
        "WEBAPPDIR" : "${CSPDIR}",
        "DBDATA" : "${DBDIR}myappdata/",
        "DBARCHIVE" : "${DBDIR}myapparchive/",
        "DBCODE" : "${DBDIR}myappcode/",
        "DBLOG" : "${DBDIR}myapplog/"
    },
    "SYS.Databases":{
        "${DBDATA}" : {"ExpansionSize":128},
        "${DBARCHIVE}" : {},
        "${DBCODE}" : {},
        "${DBLOG}" : {}
    },
    "Databases":{
        "MYAPPDATA" : {
            "Directory" : "${DBDATA}"
        },
        "MYAPPCODE" : {
            "Directory" : "${DBCODE}"
        },
        "MYAPPARCHIVE" : {
            "Directory" : "${DBARCHIVE}"
        },
        "MYAPPLOG" : {
            "Directory" : "${DBLOG}"
        }
    },
    "Namespaces":{
        "MYAPP": {
            "Globals":"MYAPPDATA",
            "Routines":"MYAPPCODE"
        }
    },
    "Security.Applications": {
        "/csp/zrestapp": {
            "DispatchClas" : "my.dispatch.class",
            "Namespace" : "MYAPP",
            "Enabled" : "1",
            "AuthEnabled": "64",
            "CookiePath" : "/csp/zrestapp/"
        },
        "/csp/zwebapp": {
            "Path": "${WEBAPPDIR}zwebapp/",
            "Namespace" : "MYAPP",
            "Enabled" : "1",
            "AuthEnabled": "64",
            "CookiePath" : "/csp/zwebapp/"
        }
    },
    "MapGlobals":{
        "MYAPP": [{
            "Name" : "Archive.Data",
            "Database" : "MYAPPARCHIVE"
        },{
            "Name" : "App.Log",
            "Database" : "MYAPPLOG"
        }]
    }
}
Set sc = ##class(Api.Config.Services.Loader).Load(config)

Output :

2021-03-31 20:20:07 Start load configuration
2021-03-31 20:20:07 {
  "SYS.Databases":{
    "/usr/irissys/mgr/myappdata/":{
      "ExpansionSize":128
    },
    "/usr/irissys/mgr/myapparchive/":{
    },
    "/usr/irissys/mgr/myappcode/":{
    },
    "/usr/irissys/mgr/myapplog/":{
    }
  },
  "Databases":{
    "MYAPPDATA":{
      "Directory":"/usr/irissys/mgr/myappdata/"
    },
    "MYAPPCODE":{
      "Directory":"/usr/irissys/mgr/myappcode/"
    },
    "MYAPPARCHIVE":{
      "Directory":"/usr/irissys/mgr/myapparchive/"
    },
    "MYAPPLOG":{
      "Directory":"/usr/irissys/mgr/myapplog/"
    }
  },
  "Namespaces":{
    "MYAPP":{
      "Globals":"MYAPPDATA",
      "Routines":"MYAPPCODE"
    }
  },
  "Security.Applications":{
    "/csp/zrestapp":{
      "DispatchClas":"my.dispatch.class",
      "Namespace":"MYAPP",
      "Enabled":"1",
      "AuthEnabled":"64",
      "CookiePath":"/csp/zrestapp/"
    },
    "/csp/zwebapp":{
      "Path":"/usr/irissys/csp/zwebapp/",
      "Namespace":"MYAPP",
      "Enabled":"1",
      "AuthEnabled":"64",
      "CookiePath":"/csp/zwebapp/"
    }
  },
  "MapGlobals":{
    "MYAPP":[
      {
        "Name":"Archive.Data",
        "Database":"MYAPPARCHIVE"
      },
      {
        "Name":"App.Log",
        "Database":"MYAPPLOG"
      }
    ]
  }
}
2021-03-31 20:20:07  * SYS.Databases
2021-03-31 20:20:07    + Create /usr/irissys/mgr/myappdata/ ... OK
2021-03-31 20:20:07    + Create /usr/irissys/mgr/myapparchive/ ... OK
2021-03-31 20:20:07    + Create /usr/irissys/mgr/myappcode/ ... OK
2021-03-31 20:20:07    + Create /usr/irissys/mgr/myapplog/ ... OK
2021-03-31 20:20:07  * Databases
2021-03-31 20:20:07    + Create MYAPPDATA ... OK
2021-03-31 20:20:07    + Create MYAPPCODE ... OK
2021-03-31 20:20:07    + Create MYAPPARCHIVE ... OK
2021-03-31 20:20:07    + Create MYAPPLOG ... OK
2021-03-31 20:20:07  * Namespaces
2021-03-31 20:20:07    + Create MYAPP ... OK
2021-03-31 20:20:07  * Security.Applications
2021-03-31 20:20:07    + Create /csp/zrestapp ... OK
2021-03-31 20:20:07    + Create /csp/zwebapp ... OK
2021-03-31 20:20:07  * MapGlobals
2021-03-31 20:20:07    + Create MYAPP Archive.Data ... OK
2021-03-31 20:20:07    + Create MYAPP App.Log ... OK

It works! The configuration is successfully loaded.

In the next article, we learn how to use config-api with ZPM to deploy your application.

The app is submitted for the contest, vote for it if you like it ;-) .

Thanks for reading.

30
1 0 16 150
Log in or sign up to continue

Replies

What a nice feature.

If I understand correctly, this module can be considered as a replacement for the Manifest installer?

If so, then this module has the benefit of exposing the configuration as JSON and API rather than a compiled class.

Exactly @Guillaume Rongier

I am a Manifest Installer user and It works fine. 

However, sometimes people which deploy applications aren't ObjectScript developers.

Manifest Installer, need to create a class, write an XML, compile. 
I would like a rupture between the configuration and the code. 

Moreover, everything is exposed in REST.  It could be interesting to integrate IRIS configuration from a tool in another language.

Thank you!

Wow. So now we can setup the environment using JSON and do more things than we can with %Installer, right?

Like users, roles, mappings, etc. Great feature!

Thank you @Evgeny Shvarov 
So, the main goals are REST expose and the rupture between config and code. 

%Installer allows to create, users, roles, mapping too, but it's not possible to configure SQL Privileges, SSL Configuration, SQL Connexion (config-api can do).

There are a few features existing in %Installer and not in config-api, but we have a good base to implement if needed.

What is the difference between SYS.Databases and Databases?

Got it. This is if I want special parameters like Expansion size, thanks.

And it looks like that if I want to create a database I need to have two entries:

SYS.Databases and

Databases.

Here is the minimal config to create IRISAPP namespace with one database IRISAPP in /irisapp folder.

{
    "Defaults":{
        "DBDIR" : "${MGRDIR}",
        "DBDATA" : "${DBDIR}irisapp/"
    },
    "SYS.Databases":{
        "${DBDATA}" : {}
    },
    "Databases":{
        "IRISAPP" : {
            "Directory" : "${DBDATA}"
        }
    },
    "Namespaces":{
        "IRISAPP": {
            "Globals":"IRISAPP"
        }
    }
    }

Works.

I use it in the following way to set up a docker container before loading the ZPM dev module into it.

Nice !

> I need to have two entries: SYS.Databases and Databases.

Yes, perhaps should I think to implement a short way to avoid two entries when we use default settings.

 

This is the correct usage.

To deploy a large application divided into several modules, using many namespaces, databases, web applications can be complex. The best way is to write a script to configure your environment and then deploy the code as you did in this simple example with zpm.

I tried your module as a replacement for the Manifest installer, see this example:

https://github.com/grongierisc/intersystems-iris-dev-template

However, I'm stuck for the part with ZPM.
The problem is, with your module, I can create a database, its rights, etc., but how to tell ZPM to load the classes in the freshly installed NameSpace?


What are the possible solutions?

We have today many options to load configuration IRIS, are they in competition?

How to federate these modules who are all complementary?

Hi @Guillaume Rongier ,

Thank you for this very interesting question.

Indeed, you can't tell directly zpm to load the classes in the freshly installed namespace.

You need to perform:

zn "irisapp"
zpm "install module-name"

I consider loading the code is the role of ZPM.
In my opinion congi-api shouldn't have an option to load the code.
We could add an option to call ZPM. Something like this :

{
    "ZPM" : {
        "IRISAPP" : {
            "install" : "module-name"
        }
    }
}

Execute zpm "install module-name" in namespace IRISAPP

Obviously this is subject to discussion. Community feedback would be appreciated.

About CPF module, I agree, my library has an overlap with all classes related to %SYS Config package.
For these operations the only difference is the REST expose. So, no added value if developers don't need REST expose.
However, config-api implement a part of %SYS Security package and also %Library SQLConnection.
You can configure users, roles, resources, ssl configuration, web application, enable services, sql connexions, set SQL Privileges and you don't with CPF module.

Hope answered to your questions.

This is an excellent application.
- It allows to create a base configuration and handle variants.
- And it's an excellent readable and easy-to-understand documentation.
- On top, it is JSON based and therefore well suited to source and version management

The original CachéParameterFile iris.cpf  does a mimic of versioning.
But it is buried deep into the  installation directory and as cryptic as the
Egyptian Book of the Dead. [reserved to the priest of IRIS cultus]  wink

I was waiting for something similar useful for decades!
I suggest: 

No one running and supporting more than 2 configurations should miss it!

Thank you @Robert Cemper .
Glad to hear that!

The original CachéParameterFile iris.cpf does a mimic of versioning.
But it is buried deep into the installation directory and as cryptic as the
Egyptian Book of the Dead. [reserved to the priest of IRIS cultus]

;-) Yes, I guessed that there is an existing feature for cpf versionning when I see classmethod parameters in Config.CommonSingleMethods :
classmethod Get(ByRef Properties As %String, ByRef CPFFile As %String = "", Flags As %Integer = $$$CPFSave+$$$CPFWrite+$$$CPFActivate) as %Status [ Language = objectscript ]