Article
· Oct 6, 2020 4m read
Open Exchange

qewd-conduit: Showcasing QEWD's Unique integration of Node.js/JavaScript and IRIS

#JavaScript #Node.js #REST API #InterSystems IRIS

Those of you who are following the FullStack competition here in the Developer Community will know that I submitted an entry named qewd-conduit.  I wanted to summarise why I think it's something worth you taking a bit of time to check out.

qewd-conduit uses the Node.js-based QEWD framework alongside IRIS to implement the back-end REST APIs for something known as the RealWorld Conduit application:

https://github.com/gothinkster/realworld

This is a very cool initiative as it provides a platform for lots of different people to implement technically different solutions for both the back- and front-ends of a specified application.  So, qewd-conduit is just one of a pretty large number of solutions that conform to the same REST API back-end specification.  Similarly, you can try out any one of a large number of different front-end clients that implement the exact same UI/UX, and integrate via REST with any of the Conduit back-ends including qewd-conduit.

So it's a great way of being able to compare and contrast different frameworks and technologies, all of which, at the end of the day, do the exact same thing.

The RealWorld application is a nice balance between being relevant and non-trivial (ie it's not just another ToDo app!) and not too overwhelmingly complex, either in terms of UI/UX or back-end APIs.  As such it's a great way of showcasing and illustrating how a particular technology can be used to implement the RealWorld's specified functionality.

So in the case of qewd-conduit, I'm showcasing how you can use QEWD as a Node.js framework for implementing, entirely in JavaScript, a set of REST APIs that maintain the RealWorld Conduit application's data in IRIS. 

The really interesting and unusual things about QEWD are that not only is your access to IRIS from within Node.js/JavaScript via an extremely high-performance in-process connection, but also, as far as you are concerned, the IRIS database is abstracted as persistent JavaScript Objects.  What does that mean?  Well, I describe it as the JavaScript equivalent of the ObjectScript up-arrow which seamlessly and transparently differentiates in-memory and on-disk arrays.  In the case of QEWD's abstraction (known as QEWD-JSdb), you effectively not only have the normal in-memory JavaScript objects, but also objects that happen to reside physically on disk within the IRIS database.  So when you manipulate these QEWD-JSdb JavaScript objects, you're actually accessing and manipulating their properties directly within the IRIS database!

You can watch a video of a presentation I gave back in January on QEWD-JSdb to the London Node.js Users Group:

https://www.youtube.com/watch?v=1TlAKTw167s&list=PLam_80-FY3vSPW9apMaczTN_4dtke9GYM

One of my main incentives for putting together qewd-conduit was as a way of showcasing this QEWD-JSdb abstraction for a well-known and understood application and use-case - ie the RealWorld Conduit REST API and its associated data set.

With QEWD, then, you maintain and access data in an IRIS database entirely in JavaScript: qewd-conduit contains no ObjectScript code (and neither does any part of the QEWD stack), and all you need to know about IRIS is how to model and manipulate data using the QEWD-JSdb abstraction from within JavaScript.  The high-speed, in-process connection between Node.js and IRIS is provided by the Open Source mg-dbx interface (created and maintained by my colleague Chris Munt): this creates the required intimate relationship between JavaScript and the IRIS database, with a level of performance close to that of native ObjectScript.

If you're interested in finding out more about how I used QEWD-JSdb to model the data needed to support the REST APIs of the RealWorld Conduit application's back-end, I've included a document in the qewd-conduit repository:

https://github.com/robtweed/qewd-conduit/blob/master/QEWD-JSdb.md

If you then want to dig deeper, background information on QEWD-JSdb (including a detailed tutorial) can be found here:

https://github.com/robtweed/qewd-jsdb
 

For more information on the mg-dbx interface, see:

https://github.com/chrisemunt/mg-dbx

One final thing: qewd-conduit goes one step further than the RealWorld Conduit specification by also implementing those same RealWorld Conduit REST APIs as corresponding WebSocket messages - WebSocket support is seamlessly integrated into QEWD, and it made sense to expose the same back-end logic used for handling the REST APIs as WebSocket messages.  That way you have a unique opportunity to compare and contrast the use and relative performance of REST versus WebSockets doing the exact same job!  I'd like to thank Ward De Backer for putting together a WebSocket-enabled version of the Vue.js-based RealWorld client - see the main qewd-conduit documentation for more information.  (Spoiler alert: you should find the WebSocket messages quite a bit faster than their REST API equivalents!)

So that's a quick bit of extra background on qewd-conduit. As you'll have hopefully begun to realise, it's just the tip of a very interesting technical iceberg!

Anyway. if you've found this interesting and useful,  do please vote for qewd-conduit in the Fullstack competition!

Rob Tweed

Check the related application on InterSystems Open Exchange
Discussion (6)2
Log in or sign up to continue
Rob Tweed · Oct 9, 2020

QEWD-Conduit is a QEWD application, which means it's a Node.js application - 100% JavaScript.  As a QEWD application, it means it uses the mg-dbx connector to attach to an instance of IRIS or Cache.  For simplicity and to make it easy and convenient for readers to try out, the version I've made available for the competition assumes it's going to be run on a Windows machine on which IRIS is installed - that's the most common set up out there.

However, as a Node.js application, qewd-conduit is capable of running on any machine on which Node.js can be installed and run - so that means Linux, Unix, Mac OS and Windows.  Since the Dockerised version of IRIS is based on a Linux OS, then yes, of course, qewd-conduit could be installed and run on it, eg by modifying the Dockerfiles setup, or simply shelling into the running container, installing Node.js, copying in the qewd-conduit repo files, running npm install and npm start.

The key to modifying how/where qewd-conduit runs is just 2 configuration files:

- the Node.js-specific package.json file.  For Linux, Unix and MacOS systems, add "mg-dbx" as a dependency and make sure that the OS has a C++ compiler installed, since mg-dbx needs to be built from source during the npm install.  For Windows, leave mg-dbx out as a dependency - QEWD will automatically download and install a pre-built Windows binary of mg-dbx when it is first run.

- the QEWD-specific /configuration/config.json file which is used by QEWD to configure itself.  In particular, the "database" section defines how the connection is to be made to IRIS or Cache, so that's where you define the path, namespace etc as appropriate for the instance of IRIS you want to connect.

Note that the config.json file I've shipped assumes that you'll run qewd-conduit on the same physical machine as your instance of IRIS.  However, you can change the settings to tell mg-dbx to connect to a separate IRIS (or Cache) instance via a network connection.

Just explore the QEWD and mg-dbx documentation and you'll discover how to run qewd-conduit in any configuration you like on anything you like.  It's a shame there's no publicly-available version of IRIS for a Raspberry Pi, because you could run qewd-conduit there too.  You *could*, of course, run qewd-conduit on a Raspberry Pi and change the config.json to make a network connection to an IRIS system on a separate machine.

QEWD really does provide the ultimate in flexibility for implementing IRIS/Cache REST services

0 0
Rob Tweed · Oct 11, 2020

I've updated the repository on Github to include files for IRIS running on Mac OS X.  Here's how to get it working:

- replace the /package.json file with /package.json.iris-osx (ie rename it to package.json)

- replace the /configuration/config.json file with /configuration/config.json.iris.osx

- Edit the /configuration/config.json file to ensure the database connection parameters are correct for your IRIS system.  In particular you'll need to change the database.path property

Then just follow the same instructions as provided for Windows in the README.

HOWEVER!!  IMPORTANT:  When you start QEWD for the second (or subsequent) time, you need to have root privileges, so you need to do this:

  sudo npm start

This is due to the root permissions needed to connect to IRIS's C interface

It should then work according to the instructions I've provided for Windows.

0 0
Rob Tweed · Oct 21, 2020

So, step 2 along the road for qewd-conduit: this time focusing on the front-end.  If you've noticed, most of the available front-ends for the RealWorld Conduit application are a real pain to install and configure.  So, courtesy of the WebComponents and Module support that is now built-into modern browsers, I've created a version using my mg-webComponents framework.  No bundled JavaScript, no heavy-weight frameworks like React, Angular, Vue etc.  And a very simple configuration allowing you to change the Conduit back-end location in a matter of seconds.

Read all about it and try it out here:

https://github.com/robtweed/wc-conduit

Let me know if you have any issues / questions / comments etc.

1 0
Rob Tweed · Oct 29, 2020

A quick update:  

I've modified our WebComponent-based wc-conduit RealWorld Client to optionally use the alternative WebSocket API interfaces supported by the qewd-conduit back-end. 

Furthermore, to make it easy for you to try out the wc-conduit front end and compare the REST and WebSocket APIs,  a copy of the wc-conduit front-end is integrated into the qewd-conduit repository.  I've updated the documentation to explain how to use the WebSocket version of wc-conduit  if you want to try it out.  It's very simple - just load a different URL to launch the wc-conduit front-end and it will automatically do the rest (no pun intended!).

  http:xx.xx.xx.xx:8080/conduit-wc  (REST)

  http:xx.xx.xx.xx:8080/conduit-wc/index-ws.html  (WebSockets)

See if you notice any performance difference between the WebSocket and REST APIs.  The back-end literally runs the same handler code regardless of the interface you choose, and the front-end WebComponents operate identically, so any difference in performance you see is network overheads of HTTP versus WebSockets, and the back-end interface overheads of HTTP (handled by the Node.js Express module) versus WebSockets (handled by the Node.js socket.io module).

REST or WebSocket APIs with an IRIS back-end. What's not to like?

0 0