Ensemble supports several SSO options, but they generally require you to have an existing framework to use. For example, OpenAM and Kerberos are supported SSO options, but in both cases you would need the frameworks and central servers for them already in place.

You might want to start by figuring out which framework(s) your other applications use already, and then seeing if you can add Ensemble to those, rather than looking at what Ensemble supports and picking based on that.

The answers you've gotten so far are for the management portal, but you also mentioned Studio, and I wanted to follow up on that part, as it's configured quite differently.

Studio connects directly to the SuperServer, so you will need to configure SuperServer TLS on the server side if you haven't already. The documentation discusses how to do this here:

https://docs.intersystems.com/latest/csp/docbook/DocBook.UI.Page.cls?KEY...

Currently, I don't know of a way to enforce TLS for just the %Service_Bindings, so you will need to enforce it on all connections to the SuperServer if you want to require it. This means you'll need to configure TLS for any other types of connections to the SuperServer you use, including between the CSP gateway and the SuperServer.

Each machine which has Studio installed will also need to be configured to use TLS. I've written up how to do that here:

https://community.intersystems.com/post/configuring-cach%C3%A9-client-ap...

I wanted to add that you certainly can create an attribute to list a user's roles as described here, and some sites do, but it's not the only way to configure LDAP authentication.

Many administrators find the group-based behavior enabled by the "Use LDAP Groups for Roles/Routine/Namespace" option easier to configure, so you should consider that option if you're setting up LDAP authentication. If you do use that option, many of the steps here will be different, including at least steps 17-23 where the attribute is created and configured.

Your second call may be overwritting the value in the first.  I would suggest setting all values you want set in a single call, with the output going to a single file.  For example:

s status=connection.SetTraceMask(511,"/tmp/ssh.log")

Also, double check that you're running this before the connection is made.   I usually run it immediately after creating the new %Net.SSH.Session object.

There are a lot of details not included here which could be necessary.  For example:

Are you using a custom login page?  The "invalid password" message you state should never be returned by default Cache pages.  This message would leak information to an attacker by letting them know that they had found a valid username.   "Access denied" is the standard message returned by Cache when a login fails, for any reason.

Have you checked the audit log for login and/or loginfailure events?  You may need to enable auditing, and then the individual event types, then reproduce the problem.  The loginfailure event should give a reason for the failure to log in.  Depending on what's happening here, it may not be the same as the error returned to the user.   

OpenSSH format is the correct public key format, so your format should be ok.  The private key should be PEM encoded.

I've seen this happen sometimes when the key is in the correct format but the file has extra blank space in it, so I'd recommend making sure the key is all on one line with no white space or line breaks.  Whether this should matter is debated, but it's easy to to check.

This error is coming from the libssh2 library, so if you want to research it further, you could look into other people using that library who've seen it. 

This is true, and adding the user to the group to stop and start is the right approach.  However, I wanted to comment on making users members of the cacheusr group, as this has large security implications on most systems.

There are two groups used in a Cache install: one is the set of users allowed to start and stop the instance, the other is an internal tool for managing file access. 

By default, the internal group (the "Effective group for Caché processes") is cacheusr.  No real users should be a member of this group.  The documentation on this page says:  "On a secure system, no actual user should be a member of this group. By default, this group is cacheusr, but you can change the group during installation."  A user who is a member of the cacheusr group will generally be able to alter or delete Cache databases, config files, and binaries at the OS level.  This is a large ability, and one you may not want to hand to users for their standard logins, and instead reserve for cases where they specifically show they mean to do that.

During install, if you chose any setting other than 'minimal' for your initial security settings, you'll be prompted to pick the group to stop and start the instance.  I would recommend picking a group other than cacheuser in order to keep these two functions separate.   Then you can add the users who you want to be able to start and stop the instance to this different group.  Note that changing this group manually, after install, is not recommended.

Whenever you see a message in the log which makes you think the system may be hung or frozen, such as 'Freezing system', please run the CacheHung command before restarting the instance or OS.  This will collect information on how and why the system is hung, and may direct you to take further steps to collect information.

CacheHung is an OS script located in the bin directory under your install directory.  On a Linux system, you can run it with a command such as:

bash$ cd /InterSystems/Cache/bin

bash$ ./CacheHung.sh

You'll need to use your own path to the bin directory, of course.  There will be several prompts about what to collect, and the script will take a minute or two to run.  Depending on your configuration, you may need to be root to run it.

Once the data has been collected, you can open the generated file with a web browser and go to the "self-diagnosis" section to see whether more steps are suggested.

I agree with Andrew that this is complex enough that you'll get better answers by talking to the WRC.   In addition, I'd also recommend taking a Buttons or CacheHung report quickly, so that you have the cstat data to look at later if the problem clears up.  If it clears up before you collect any data, there likely won't be much anyone can say.

There isn't currently a way to avoid running ZAUTHENTICATE before password logins.  Running the routine is deliberate behavior because it's used by some sites as part of their login process.  You could ask for a change, such as a configuration option to control the behaviour, to be added but that won't fix your issue right now.

Have you considered changing the existing password users to being delegated users?  This would remove the extra login failure event for them.  You would have to update your ZAUTHENTICATE code to handle the accounts, or create matching accounts on your AD server.

You can change user account types by editing the user objects.  Here's an example of how you can do it:

%SYS>s status=##class(Security.Users).Get("Admin",.prop)

%SYS>w prop("Flags")  ; This is the property which states this is a password user account                                   
1
%SYS>s prop("Flags")=4   ; This value says this is a delegated user                 

%SYS>s status=##class(Security.Users).Modify("Admin",.prop)
 

This isn't one of the errors that easy to diagnose based on the client log, unfortunately.  Here are a couple of the most likely things it could be:

- This error can happen when the client and server don't support compatible versions of TLS.  For example, if you're configured to use only TLS v1.0 and the server wants TLS v1.2.  This is more likely if you're using an older version of Ensemble which doesn't support TLS v1.2.  Which versions of SSL/TLS have you enabled? (Note: trying to fix this by enabling all possible versions is a bad idea.  SSLv3 should not be used.) 

- This could also mean that the server requires server name indication (SNI) and the SNI info is not being sent by your client.  SNI is supported in Ensemble 2017.2. 

This isn't causing your failure, but I would recommend that you change the value of peer certificate verification from 'none' to 'required'.  As a client, you want to check that you're connecting to the server you think you are, not another server pretending to be that one.   You will need to set a certificate authority file for this to work.

That error is the generic failure, so it doesn't tell us what's wrong.  You can get more information by enabling the network debug flags:

%SYS>d INTSET^REDEBUG("FFFFFFFF")

and then trying again.  Detailed errors will be printed in the cconsole.log.  This level of debug info can fill the log quickly, so remember to set the level back to "FF" after you're done testing.

Also note that the test button in a configuration can't handle any protocols where there are messages before the TLS handshake.  This shouldn't be a problem for https, but it may be an issue for SMTP, FTPS, etc.