New post
Article
· May 10, 2018 3m read

Replacing ZEN - Part 2 - Strategic Issues

#Angular2 #CSP #JSON #Frontend #ZEN #Caché

Index to Articles

Published 2018-05-11 Last edit -

Hi All

I this article I detail some strategic issues that a new development UI will need to address - these are the ones that I can think of now - others may come to light during this journey.

See the webinar by Eduard Lebedyuk here from the last Global Summit describing  modern web development and Caché

And, as always, if I have missed something please comment....

Here's my list (so far):-

  • Choosing a Full Stack Development Environment With ZEN I need to know  ooDesign, COS, Studio (for editing), COS, HTML, Javascript, CSS, FOP !!!!!! And each of these is large in themselves. There is a good video  here that gives an overview of the choices - but it's dated as it does not include containers But the detailed choices huge -The video here details hundreds of different technologies/tools/environments  - it's awesome and frighting  The key decisions I need to make are
    • What client development environment to use 
    • What development environment (Caché Studio or Eclipse & Atelier - Which client editor to use)
    • What version control system that controls and moves both Caché classes and client files (CSS, JS, HTML or whatever)

As I said in part 1, I want to concentrate my intellectual effort to deliver complex business applications not the tools and always having to learn new stuff.

  • Leverage Existing ZEN code (link under construction) Again, as in the introduction I would want to leverage/translate the existing ZEN definitions to use going forward. This will:-
    • Speed up development
    • Reduce testing - end-user acceptance testing is a large overhead for a small/medium company  
  • Sessions or No Sessions (link under construction) A key concept for JSON communication between the client and the server is that each interaction is self contained, ie the server and the client need to know nothing about each other over the content of each restful call and the JSON response - ie no CSP sessions Is this correct or do we need to maintain sessions?  
  • Security (link under construction) Using JSON - how do you implement a logon? What are the licensing issues? How do you prevent users hacking restful calls that they have no access to? There is an post by Anna Kantola here that describes some problems. logging in via rest  
  • Screen Layout/Responsive App (link under construction) As I said in the introduction, responsive apps for a desktop business application is not a high priority However it would be great to have a single framework that can be used for both static screens on the desktop *and* for responsive apps.  
  • Localisation (multi-lingual pages)  (link under construction) ZEN/CSP has in-built localisation (csptext) that allows a single page definition to localise according to the browser language settings - it's a bit fiddly but works great allowing a single page to seamlessly appear in the user's preferred language .  What other options are there? Also date/time localisation.

Most of the above (will) link to a separate page to reduce the size of the article and to allow comments to be added to the appropriate heading

Index to Articles

Discussion (13)2
Log in or sign up to continue
Eduard Lebedyuk · May 11, 2018

Sessions or No Sessions 

REST mainly disallows sessions as a mechanism of data transfer. Stateless as stated in REST dissertation by Roy Fielding has three aims:

  • visibility
  • reliability
  • scalability

Visibility is improved because a monitoring system does not have to look beyond a single request datum in order to determine the full nature
of the request. In my opinion that's the most important one and it mainly deals with storing session data in between requests.

Let's say you provide a newsfeed API. There's a lot of news so users get them by pages of 10 elements. Clients can access news in two different ways:

  1. http://host/api/news/next - in this case the server remembers last page of news the client requested and returns the next one. That's not stateless.
  2. http://host/api/news/:pagenmumber - in this case client remembers last page of news he requested and asks for the next one by incrementing page number by 1. That's stateless.

Reliability is improved because it eases the task of recovering from partial failures. Where partial failure is defined as (from Waldo J,
Wyant G, Wollrath A, Kendall S. A Note on Distributed Computing.):

Partial failure is a central reality of distributed computing. Both the local and the distributed world contain components that are subject to periodic failure.

In the case of local computing, such failures are either total, affecting all of the entities that are working together in an application, or detectable by some central resource allocator (such as the operating system on the local machine).

This is not the case in distributed computing, where one component (machine, network link) can fail while the others continue. Not only is the failure of the distributed components independent, but there is no common agent that is able to determine what component has failed and inform the other components of that failure, no global state that can be examined that allows determination of exactly what error has occurred.

In a distributed system, the failure of a network link is indistinguishable from the failure of a processor on the other side of that link.

Sessions as an authentication/authorisation mechanism do not affect Reliability.

Scalability is improved because not having to store state between requests allows the server component to quickly free resources, and
further simplifies implementation because the server doesn't have to manage resource usage across requests. Not relevant in our case as Session still gets created, just destroyed immediately after the request is done.


To sum up: REST APIs can use sessions as authentication mechanism, but not as a data transfer mechanism.

1 0
Eduard Lebedyuk · May 11, 2018

Security (link under construction)
Using JSON - how do you implement a logon?

What are the licensing issues?

For password authenticated web applications it is possible by following these steps:

  1. All brokers effectively have Parameter UseSession = 1;
  2. REST web application and client web application allow only authenticated (i.e. password) access.
  3. REST web application and client web application have reasonable Session timeout (i.e. 900, 3600).
  4. REST web application and client web application have the same GroupById value.
  5. REST web application and client web application have the same cookie path.

If all these conditions are met, user would only consume one license slot per session and perform only one login.

How do you prevent users hacking restful calls that they have no access to?

Authentication as a start, SQL security for basic data access checks, app-level checks for the most specific cases

1 0
Peter Cooper · May 11, 2018

Hi Eduard

I had already seen your second comment elsewhere - but thanks for the additional detailed comment - most useful.

Just some quick thoughts......

  • All brokers effectively have Parameter UseSession = 1;
    But the default value in %CSP.REST is Parameter UseSession As BOOLEAN = 0;
    So the developer has to remember to override this every time (or sub-class)
     
  • Use same GroupById  
    I have seen this but have no understanding of what it's for - searching the docs only brings up a reference in the automatic deployment section only with no description
    If you have any more information on it's purpose please share 
     
  • Sharing the session between the CSP/ZEN app and Rest calls is a great feature
    But if it's a third party app then there is no CSP/ZEN app - the use case I have in mind is a 3rd party web developer is creating a complex shop system that needs to communicate with Caché
    I have no idea or interest in what technology they are using and it may be that their programming language does not easily support cookies so the CSPCHD (the session cookie) does not get passed.
    I am thinking that in this case the authentication needs to be passed with each Rest call - not an issue
    (or use OAUTH which I know little about)
     
  • On security
    Yes SQL security for sure - but there are traps here in the management of that security - I don't want a huge overhead in having to manage multiple roles each with it's own SQL security profile.
    And anyway it does not help if the Rest application is accessing the database via the object projection rather than SQL
    ​

As always, there are many options/scenarios my aim is not to give a definitive answer but to share my experience as a real world developer and how I implement things

Peter

0 0
Peter Cooper · May 11, 2018

Oh forgot this one......

Maintaining State between client calls...

I can think of a real world case where some sort of state between the client and the server is a good idea...

Consider a huge database where we need to query and produce the equivalent of the ZEN tablepane that displays rows of data with filtering and sorting functionality
I can see there are a number of widgets available for different frameworks where the whole result set is shipped to the client and filtering/sorting is done there.

However there are cases were....

  • We want the query to be run once on the server
  • We do not want  to ship the whole resultset  (too big there could be 100,000's rows) to the client and do pagination/sorting/filtering there
  • So the client component needs to get the data page by page from the server - and any filtering/sorting is done there
  • This is just how the ZEN tablepane works using ^CacheTemp(xxx to stash the data
  • Hence we are maintaining a state between the client and server

Unless you can suggest a better alternative that does not require state to be maintained

At the moment this is the only use case that I can think of where maintaining state is a useful thing 

Peter

0 0
Eduard Lebedyuk · May 11, 2018

Execute queries and show tables

State is not required (and in fact harmful) for that case. Here's why.

Users do not care for thousands of results.

That doesn't happen. User most often cares about one specific result, or a small group of them - dozen(s), rarely up to a hundred but to be extremely  generous let's say that user cares about 1000 individual records at the same time tops.

So, how do we work with that assumption?

There are several things to do.

  1. Add TOP 1000 to all of your queries, if the query actually returns 1000 results show 1000+ to the user and if he explicitly asks for more results reexecute the query (with new hidden condition to skip thousand of the results if possible) and show the next thousand. But don't forget to point him to docs page explaining how to refine search results before that. It's an extremely rare use case where user needs more than a thousand results - it should not happen and if that happens, that it's definitely something architects/product owners need to address.
  2. Why would the user scroll through thousands of the results? Because the results of  the query are irrelevant to him (low relevance). To solve that problem the user should be able easily, but precisely, express what does he need fom the system. This is a very complex problem, but some of the features that can be added are:
    • Automatic filtering by all (most) fields - if a new column appears, user should be able to sort/filter by it automatically, as in without developer spending time on adding the new field to the UI.
    • Filtering and sorting comparisons should be at least (dependent on datatype): equals, more, lest, between, contains ...
    • Filters could be combined via: AND, OR, NOT
    • High-relevancy fields should be easily accessible for the user. For example in one of our PoCs we've seen a problem with existing search - users overloaded it, by performing full-text search en mass. Examining the logs, we found that most of the search keywords were in a form of abc/xyz where abc and xyz are integers. That abc/xyz turned out to be values from one specific field (let's say ItemId), but fulltext search was the biggest search field and placed at the top of the search page. Our solution was redesign - placing ItemId search field at the top.

One other case where user might be actually interested in the exact number of the results is when he needs that number only. For example our user might be interested in the number of incidents per month and calculate it himself by filtering incidents by date and getting the results count. This requirement could be addressed in several ways:

  • The better solution would essentially be reports, maybe even some visual design, or by request. That is a preferable solution.
  • Add "Count results" button alongside "Search" button. When the user presses this button SQL query would contain only COUNT(1) select field and only that result would be shown.

tl;dr users are not interested in thousands of results, so we need to build systems where they get only the results they really need.

0 0
Peter Cooper · May 11, 2018

Hi Eduard

Food for thought !!!!

It's just that it's always been the way I have done it in ZEN for around 11 years - started off with "small" result sets and just carried on doing it that way
It's great to be challenged - Thank you!!!

= =

BTW I like the look of the way that you include source code in your posts
Do you care to share how you create the posts with the grey box around them - do you create/edit with the online editor or some other tool?

Peter

0 0
Eduard Lebedyuk · May 11, 2018

All brokers effectively have Parameter UseSession = 1;
But the default value in %CSP.REST is Parameter UseSession As BOOLEAN = 0;
So the developer has to remember to override this every time (or sub-class)

I recommend having an abstract broker, which does technical stuff like CORS, UseSession, Encoding, JSON transformation. All other brokers must extend it (and they don't do technical stuff, only business logic). More on that.

Use same GroupById

If two (or more) web applications share the same GroupBy value, then session opened in one application would be valid in all other applications with the same GroupBy value. This way user needs to login only once (or not at all if we have domain/SSO authentication configured). It's also documented (but hard to find), more docs.

But if it's a third party app then there is no CSP/ZEN app - the use case I have in mind is a 3rd party web developer is creating a complex shop system that needs to communicate with Caché

CSP app could contain only HTML/JS/CSS. So it could be an AngularJS web application, but also hosted with Caché (as a Caché CSP web application).


I have no idea or interest in what technology they are using and it may be that their programming language does not easily support cookies so the CSPCHD (the session cookie) does not get passed.

If you host a web-application via Caché (Ensemble, HealthShare, InterSystems IRIS) then your web application would be authorization-aware automatically. Browser sends relevant cookies/headers with each request so developer don't need to think about it.

I am thinking that in this case the authentication needs to be passed with each Rest call - not an issue
(or use OAUTH which I know little about)

You can do it like this:

  1. Client sends login/pass to some /login REST endpoint and receives token
  2. After that client only sends token with every request
  3. Token should be periodically refreshed

This way you pass login/pass only once instead of with every call. But I'd still recommend Caché security system.

1 0
Anil Mathew · Oct 9, 2019

Hi Peter,

I am not sure if it is appropriate to post my concern here.  Please let me know if I need to move this out this and post separately. 

The reason I posted it here is because similar posts have been redirected to this discussion.   This is wonderful discussion and we ourselves are probably on a similar journey butstill at exploratory stage.  Both you and Eduard seems to have traveled further down this road and you might be able to point me in the proper direction.  Below mentioned is my concern.

Is it mandatory for the application to be rendered using the CSP Server to maintain the CSP Session?

Would it be possible for html and javascript based application can connect to the REST service and maintain the session and the license?

If possible would someone be able post the login code and and the headers or parameters to be passed for the subsequent calls to the REST service. 

I made some trials with the Delegated login and ZAUTHENTICATE (Hardcoded the Username and Password ) for the REST application

I tried  all the below steps (only for the REST web application.  The client web application is not CSP)

  1. All brokers effectively have Parameter UseSession = 1;
  2. REST web application and client web application allow only authenticated (i.e. password) access.
  3. REST web application and client web application have reasonable Session timeout (i.e. 900, 3600).
  4. REST web application and client web application have the same GroupById value.
  5. REST web application and client web application have the same cookie path.

Still each call adds a new session. 


 

0 0