AppS.REST - a new REST framework for InterSystems IRIS

Article

Timothy Leavitt · Jun 4, 2020 3m read

Open Exchange

#API #Best Practices #Data Model #Framework #JSON #REST API #Security #InterSystems IRIS #InterSystems IRIS for Health #Open Exchange

Over the past year or so, my team (Application Services at InterSystems - tasked with building and maintaining many of our internal applications, and providing tools and best practices for other departmental applications) has embarked on a journey toward building Angular/REST-based user interfaces to existing applications originally built using CSP and/or Zen. This has presented an interesting challenge that may be familiar to many of you - building out new REST APIs to existing data models and business logic.

As part of this process, we've built a new framework for REST APIs, which has been too useful to keep to ourselves. It is now available on the Open Exchange at https://openexchange.intersystems.com/package/apps-rest. Expect to see a few more articles about this over the coming weeks/months, but in the meanwhile, there are good tutorials in the project documentation on GitHub (https://github.com/intersystems/apps-rest).

As an introduction, here are some of our design goals and intentions. Not all of these have been realized yet, but we're well on the way!

Rapid Development and Deployment

Our REST approach should provide the same quick start to application development that Zen does, solving the common problems while providing flexibility for application-specific specialized use cases.

Exposing a new resource for REST access should be just as easy as exposing it a a Zen DataModel.
Addition/modification of REST resources should involve changes at the level being accessed.
Exposure of a persistent class over REST should be accomplished by inheritance and minimal overrides, but there should also be support for hand-coding equivalent functionality. (This is similar to %ZEN.DataModel.Adaptor and %ZEN.DataModel.ObjectDataModel.)
Common patterns around error handling/reporting, serialization/deserialization, validation, etc. should not need to be reimplemented for each resource in each application.
Support for SQL querying, filtering, and ordering, as well as advanced search capabilities and pagination, should be built-in, rather than reimplemented for each application.
It should be easy to build REST APIs to existing API/library classmethods and class queries, as well as at the object level (CRUD).

Security

Security is an affirmative decision at design/implementation time rather than an afterthought.

When REST capabilities are gained by class inheritance, the default behavior should be to provide NO access to the resource until the developer actively specifies who should receive access and under what conditions.
Standardized implementations of SQL-related features minimize the surface for SQL injection attacks.
Design should take into consideration the OWASP API Top 10 (see: https://owasp.org/www-project-api-security)

Sustainability

Uniformity of application design is a powerful tool for an enterprise application ecosystem.

Rather than accumulating a set of diverse hand-coded REST APIs and implementations, we should have similar-looking REST APIs throughout our portfolio. This uniformity should lead to:
- Common debugging techniques
- Common testing techniques
- Common UI techniques for connecting to REST APIs
- Ease of developing composite applications accessing multiple APIs
The set of endpoints and format of object representations provided/accepted over REST should be well-defined, such that we can automatically generate API documentation (e.g., Swagger/OpenAPI) based on these endpoints.
Based on industry-standard API documentation, we should be able to generate portions of client code (e.g., typescript classes corresponding to our REST representations) using third-party/industry-standard tools.

Class NPM.REST.Handler Extends AppS.REST.Handler { ClassMethod AuthenticationStrategy() As %Dictionary.CacheClassname { Quit ##class(AppS.REST.Authentication.PlatformBased).%ClassName(1) } ClassMethod GetUserResource(pFullUserInfo As %DynamicObject) As AppS.REST.Authentication.PlatformUser { Quit ##class(AppS.REST.Authentication.PlatformUser).%New() } }

Class NPM.Model.Task Extends (%Persistent, %Populate, %JSON.Adaptor, AppS.REST.Model.Adaptor) { Parameter RESOURCENAME = "task"; Property RowID As %String(%JSONFIELDNAME = "_id", %JSONINCLUDE = "outputonly") [ Calculated, SqlComputeCode = {Set {*} = {%%ID}}, SqlComputed, Transient ]; Property TaskName As %String(%JSONFIELDNAME = "taskName"); /// Checks the user's permission for a particular operation on a particular record. /// <var>pOperation</var> may be one of: /// CREATE /// READ /// UPDATE /// DELETE /// QUERY /// ACTION:<action name> /// <var>pUserContext</var> is supplied by <method>GetUserContext</method> ClassMethod CheckPermission(pID As %String, pOperation As %String, pUserContext As AppS.REST.Authentication.PlatformUser) As %Boolean { Quit ((pOperation = "QUERY") || (pOperation = "READ") || (pOperation = "CREATE") || (pOperation = "UPDATE")) } }

Eduard Lebedyuk · Jun 4, 2020

Awesome!

1 0

Henrique Dias · Jun 4, 2020

@Timothy Leavitt this is amazing!

I'll be making use of it in my application :)

I was looking into the OpenExchange description, and in the Tutorial and User Guide, I think the links are broken. I got a "Not found" message when I try to access the URLs.

https://openexchange.intersystems.com/docs/sample-phonebook.md

https://openexchange.intersystems.com/docs/user-guide.md

2 0

Timothy Leavitt · Jun 5, 2020

Thank you for your interest, and for pointing out that issue. I saw it after publishing and fixed it in GitHub right away. The Open Exchange updates from GitHub at midnight, so it should be all set now.

Arto Alatalo · Jun 6, 2020

minimum platform version of InterSystems IRIS 2018.1

Porting old apps with a framework available on new version of the platform (IRIS) only, no contradictions here? :) Is there something fundamental preventing the framework from being used on Cache too?

0 0

Henrique Dias · Jun 6, 2020

Maybe I'm wrong, but the minimum requirement here it's because you don't have %JSON.Adaptor on Caché.

Timothy Leavitt · Jun 8, 2020

@Henrique Dias is right - that's the reason for the minimum requirement.

IMO, getting an old app running on the new version of the platform is a relatively small effort compared to a Zen -> Angular migration (for example).

Robert Cemper · Jun 7, 2020

The %JSON. Adaptor is missing in Caché but %JSON.Formatter was backported half a year ago. it is in OpenExchange available

Henrique Dias · Jun 17, 2020

Hi @Timothy Leavitt
I'm testing the AppS.REST to create a new application, following the Tutorial and Sample steps in Github I created a Dispatch Class:

And a simple persistent class:

But when I try the REST API using Postman GET: http://localhost:52773/csp/npm-app-rest/api/task/1

I'm getting a 404 Not Found message.

Am I doing something wrong or missing something?

Thanks

Timothy Leavitt · Jun 17, 2020

@Henrique Dias , do you have a record with ID 1? If not, you can populate some data with the following (since you extend %Populate):

Do ##class(NPM.Model.Task).Populate(10)

Yes, I already populated the class.

Eduard Lebedyuk · Jun 18, 2020

Give CSPSystem user access to the database with a REST broker.

Timothy Leavitt · Jun 18, 2020

@Eduard Lebedyuk is probably right on this. If you add auditing for <PROTECT> events you'll probably see one before the 404.

Henrique Dias · Jun 18, 2020

I added auditing on everything, and the <PROTECT> error never showed up. So, I started everything from scratch and found out a typo on Postman.

Thanks, @Eduard Lebedyuk @Timothy Leavitt

PS: Sorry, guys. I think not sleeping enough hours isn't good for health and cause this kind of mistakes

Richard Schilke · Aug 7, 2020

This is really cool, and we will be using this in a big way.

But I have encountered an issue I can't fix.

I took one of my data classes (Data.DocHead) and had it inherit from AppS.REST.Model.Adaptor and %JSON.Adaptor, set the RESOURCENAME and other things and tested using Postman and it worked perfectly! Excellent!

Due to the need to have multiple endpoints for that class for different use cases, I figured I would set it up using the AppS.REST.Model.Proxy, so I created a new class for the Proxy, removed the inheritance in the data class (left %JSON.Adaptor), deleted the RESOURCENAME and other stuff in the data class.

I used the same RESOURCENAME in the proxy that I had used in data class originally.

I compiled the proxy class, and get the message:

ERROR #5001: Resource 'dochead', media type 'application/json' is already in use by class Data.DocHead
> ERROR #5090: An error has occurred while creating projection RestProxies.Data.DocHead:ResourceMap.

I've recompiled the entire application with no luck. So there must be a resource defined somewhere that is holding dochead like it was still attached to Data.Dochead via a RESOURCENAME, but that parameter is not in that class anymore.

How do I clear that resource so I can use it in the proxy?

Timothy Leavitt · Aug 7, 2020

@Richard Schilke, I'm glad to hear that you're planning on using this, and we're grateful for your feedback.

Quick fix should just be: Do ##class(AppS.REST.ResourceMap).ModelClassDelete("Data.DocHead")

Background: metadata on REST resources and actions is kept in the AppS.REST.ResourceMap and AppS.REST.ActionMap classes. These are maintained by projections and it seems there's an edge case where data isn't getting cleaned up properly. I've created a GitHub issue as a reminder to find and address the root cause: https://github.com/intersystems/apps-rest/issues/5

Richard Schilke · Aug 8, 2020

That did the trick - thank you so much!

Best practice check: When I have a data class (like Data.DocHead) that will need multiple Mappings (Base, Expanded, Reports), then the recommended way is to use the proxy class and have a different proxy class for Data.DocHead for each mapping?

For example, RESTProxies.Data.DocHead.Base.cls would be the proxy for the Base mapping in Data.DocHead, while RESTProxies.Data.DocHead.Expanded.cls would be the proxy for the Expanded mapping in Data.DocHead, etc. (the only difference might be the values for the JSONMAPPING and RESOURCENAME prameters)? I'm fine with that, just checking that you don't have some other clever way of doing that...

Timothy Leavitt · Aug 10, 2020

@Richard Schilke , yes, having a separate proxy for each mapping would be best practice. You could also have Data.DocHead extend Adaptor for the primary use case and have proxies for the more niche cases (if one case is more significant - typically this would be the most complete representation).

@Timothy Leavitt, I've run into another issue.

The proxy is setup and working great for general GET access. But since my system is a multi-tenant, wide open queries are not a thing I can use, so I decided to try to use a defined class Query in the data class Lookups.Terms:

Query ContactsForClientID(cClientOID As %String) As %SQLQuery
{
SELECT *
FROM Lookups.Terms
WHERE ClientID = :cClientOID
ORDER BY TermsCode
}

Then I setup the Action Mapping in my proxy class RESTProxies.Lookups.Terms.Base:

XData ActionMap [ XMLNamespace = "http://www.intersystems.com/apps/rest/action" ]
{
<actions xmlns="http://www.intersystems.com/apps/rest/action">
<action name="byClientID" target="class" method="GET"
modelClass="Lookups.Terms" query="Lookups.Terms:ContactsForClientID">
<argument name="clientid" target="cClientOID" source="url"/>
</action>
</actions>
}

And I invoked this using this URL in a GET call using Postman (last part only):

terms_base/$byClientID?clientid=290

And the result:

406 - Client browser does not accept the MIME type of the requested page.

In the request, I verified that both Content-Type and Accept are set to application/json (snip from the Postman):

So what have I missed?

What's the MEDIATYPE parameter in Lookups.Terms (the model class)? The Accept header should be set to that.

Also, you shouldn't need to set Content-Type on a GET, because you're not supplying any content in the request. (It's possible that it's throwing things off.)

If you can reproduce a simple case independent of your code (that you'd be comfortable to share), feel free to file a GitHub issue and I'll try to knock it out soon.

Richard Schilke · Aug 10, 2020

I posted an issue with my source to Github.

Surfaced another issue this week-end. (I remember when I used to take week-ends off, but no whining!)

So I have a multiple linked series of classes in Parent/Child relationships:

DocHead->DocItems->DocItemsBOM->DocItemsBOMSerial

So if I wanted to express all of this in a JSON object, I would need to make the "Default" mapping the one that exposes all the Child Properties, because it looks like I can't control the Mapping of the Child classes from the Parent class.

This doesn't bother me, as I had already written a shell that does this, and your Proxy/Adaptor makes it work even better, but just wanted to check that the Parent can't tell the Child what Proxy the child should use to display its JSON. It's even more complicated than that, as sometimes I want to show DocHead->DocItems (and stop), while, in other Use Cases, I have to show DocHead, DocItems, and DocItemsBOM (and stop), while in other Use Cases, I need the entire stack.

Thanks for posting - I'm taking a look now. This issue is starting to ring a bell; I think this looks like a bug we fixed in another branch internally to my team. (I've had reconciling the GitHub branch and our internal branch on my list for some time - I'll try to at least get this fix in, soon.)

Re: customizing mappings of relationship/object properties, see https://docs.intersystems.com/healthconnectlatest/csp/docbook/Doc.View.cls?KEY=GJSON_adaptor#GJSON_adaptor_xdata_define - this is doable in %JSON.Adaptor mapping XData blocks via the Mapping attribute for an object-valued property included in the mapping.

Wow - I think that means I can handle all my Use Cases with that capability. Nice!

Thanks again!

Richard Schilke · Aug 12, 2020

@Timothy Leavitt , have you had a chance to see if this error I'm getting on Actions was resolved?

Timothy Leavitt · Aug 12, 2020

@Richard Schilke I'm planning to address it tomorrow or Friday. Keep an eye out for the next AppS.REST release on the Open Exchange - I'll reply again here too. (This will also include a fix for the other issue you reported; I've already merged a PR for that.)

@Timothy Leavitt , I will be looking for it.

I'm trying to do something with a custom Header that I want to provide for the REST calls. Do I have access to the REST Header somewhere in the service that I can pull the values, like a %request?

And in something of an edge case, we're calling these REST services from an existing ZEN application (for now as we start a slow pull away from Zen), so the ZEN app gets a %Session created for it, and then calls the REST service. It seems that Intersystems is managing the License by recognizing that the browser has a session cookie, and it doesn't burn a License for the REST call - that's very nice (but I do have a request in to the WRC about whether that is expected behavior or not so I don't get surprised if it gets "fixed"!). Does that mean your REST service can see that %Session, as that would be very helpful, since we store User/Multi-tenant ID, and other important things in there (the %Session, not the cookie).

@Richard Schilke - on further review, it's an issue with the Action map. See my response in https://github.com/intersystems/apps-rest/issues/7 (and thank you for filing the issue!). I'll still create a new release soon to pick up the projection bug you found.

Regarding headers - you can reference %request anywhere in the REST model classes, it just breaks abstraction a bit. (And for the sake of unit testing, it would be good to behave reasonably if %request happens not to be defined, unless your planning on using Forgery or equivalent.)

Regarding sessions - yes, you can share a session with a Zen application via a common session cookie path or using GroupById. You can reference this as needed as well, though I'd recommend wrapping any %session (or even %request) dependencies in the user context object that gets passed to CheckPermissions().

@Timothy Leavitt - thanks so much for the response. The Action worked perfectly with your corrections!

I will take your advice and work with the %session/headers in the context object, since that makes the most sense.

What are the plans (if any) to enable features in a resultset such as pagination, filters, and sorting?

Users are horrible, aren't they? No matter what good work you do, they always want more! I appreciate what you have done here, and it will save my company probably hundreds of hours of work, plus it is very elegant...

@Richard Schilke - great!

We have support for filtering/sorting on the collection endpoints already, though perhaps not fully documented. Pagination is a challenge from a REST standpoint but I'd love to add support for it (perhaps in conjunction with "advanced search") at some point. I'm certainly open to ideas on the implementation there. :)

Users are the best, because if you don't have them, it's all just pointlessly academic. ;)

@Timothy Leavitt - stuck again.

I'm in ClassMethod UserInfo, and found out some interesting things.

First off, I was wrong about the REST service using the session cookie from the Zen application when it is called from the Zen application. Displaying the %session.SessionId parameters for each call shows that they are all different, and not the same as the SessionId of the Zen application calling the REST service. So the idea that it holds a license for 10 seconds can't be correct, as it seems almost immediate. I run 20 REST calls to different endpoints in a loop, and I saw a single License increase.

You said I should be able to expose the session cookie of the Zen application, but I don't see a way to do that either.

I can't even find a way to see the header data in the UserInfo ClassMethod of the current REST call.

Sorry to be a pest...but since you''re giving answers, I'll keep asking questions!

Have a nice evening...

@Richard Schilke , you should be able to share a session by specifying the same CSP session cookie path for your REST web application and the web application(s) through which your Zen pages are accessed. Alternatively, you could assign the web applications the same GroupById in their web application configuration.

You likely also need to configure your REST handler class (your subclass of AppS.REST.Handler) to use CSP sessions (from your earlier description, I assumed you had). This is done by overriding the UseSession class parameter and setting it to 1 (instead of the default 0).

To reference header data in the UserInfo classmethod, you should just be able to use %request (an instance of %CSP.Request) and %response (an instance of %CSP.Response) as appropriate for request/response headers.

I'll also note - the only thing that really matters from the class query is the ID. If nothing else is using the query you could just change it to SELECT ID FROM ... - it'll constitute the model instances based on that. (This is handy because it allows reuse of class queries with different representations.)

Good to know and, yes, very handy!

Thank you!

Developer Commu... · Dec 11, 2020

💡 This article is considered as InterSystems Data Platform Best Practice.

Gerrit Henning · Sep 20, 2021

How would the AppS.REST Handler co-exist with a 'Spec-first' approach, where the dispatch class should not be modified manually - only by re-importing the API spec?

The AppS.REST user-guide states: 'To augment an existing REST API with AppS.REST features, forward a URL from your existing REST handler to this subclass of AppS.REST.Handler.' How would this work in practice with the above?

Thanks in advance.