New post
Question
· Apr 19, 2017

WebSockets vs Long Polling vs Short Polling?

#Node.js #Tutorial #CSP #Caché

WebSockets look to be supported reasonably well in Cache. I have yet to use them in production so I am wondering how well it has worked for other developers.

In particular what happens when the browser does not support WebSockets, or when a firewall blocks the connection.

Have you had to write your own long polling fall-back?

I've read the documentation and found this interesting article...

https://community.intersystems.com/post/asynchronous-websockets-quick-tutorial

but no mention of long polling fall-backs that I can see.

Having tinkered around with sockjs and socket.io (with node) there is a realisation that these libraries require a great deal of fringe code to deal with a vast number of issues, such as fall-backs, disconnects and fringe errors, that I worry about using WebSockets in production.

Interestingly I have also read that facebook and twitter have used (and may still use) long polling instead of websockets because of various issues they have experienced. Considering the millions of users they have constantly connected, it makes me wonder if the fuss with long polling has more to do with how its implemented. If they can make it work for millions on technology x, then why not for Caché.

In particular, if the applications that I develop are intranet wide and not internet wide, then should I save the technical headache and just implement long polling instead for a smaller number of users. If I was using node.js then I would probably lean towards this decision based on my numbers of users. Except if I am using Caché only I wonder how I would implement this without each users connection creating a hung process until something happens.

I could write an intermediary message broker in node.js, but ideally I want a solution that is Caché only.

In the past I have developed short polling solutions in Caché. Requests that are known to have async operations are queued and the client is asked to periodically check back for the response. In benchmarks this is still able to handle a colossal number of messages, way more than my users ever need. It's simplicity is a grade trade off for the small lag that my users probably don't notice. But, removing that lag would still be good.

Any experience or advice would be appreciated. Is short polling working just fine for you? Does anyone have a clever Caché only long poll solution they can share that does not hang a process? Are you more than happy with websockets alone, or have you had to implement fall-backs as well?

Sean.

Discussion (15)0
Log in or sign up to continue
Sean Connelly · Apr 21, 2017

Hi Nikita,

Thanks for your detailed response. Good to hear a success story with WebSockets. Many of the problems that I have read about are very fringe. Most firewalls seem to allow the outgoing TCP sockets because they are running over 80 or 443, but it appears there are fringe cases of a firewall blocking the traffic. Also certain types of AV software can block. I suspect these problems are more prominent in the node.js community because node is more prevalent than Caché and that Caché is more likely to be inside the firewall with the end users.

The main problem I still have is that I work on Caché and Ensemble inside healthcare organisations in the UK and they are always behind on browser versions for various reasons. Only recently have I been able to stop developing applications that needed to work on IE6. Many are still on IE8 or IE9 (sometimes running in IE7 emulation mode). Either way websockets only work on IE10+. I can work around most browser problems with pollyfill's, but sockets require a client and server solution. That means you can't just drop in sockjs as an automatic fall-back library because there is no server side implementation for it on Caché.

Without any library built for Cache I am thinking what is needed is a simple native emulator of the client socket library that implements a simple long poll implementation with Cache. If I then hit a scalability problem then it would be time to put Node.JS in front of Cache with all the additional admin overhead. A nice problem to have all the same. Still, I suspect it would be a large number of users and sockets to swamp out Cache resources.

Your WebTerminal looks very good. Not sure why I have not seen that before, looks like something I would use. I'm not sure why we can't have a web based IDE for Cache when I see a complex working web application such as this. I even have my own web based IDE that I use for that other M database, not sure I can mentioned it here :) which I keep thinking to port to Cache.

0 0
Sean Connelly · Apr 21, 2017

Good to know QEWD has been using web sockets for a long time. Socket.io is a well maintained library with all of the fall back features that I worry are missing with Cache sockets alone. Makes it a lot easier when putting Node.JS in front of Cache. I guess I just want to avoid any additional moving parts as I bash out as many web applications per year as I have users on some of them. That's why I never use the likes of Zen, I just need things simple and fast and typically avoid web sockets for fear of headaches using them. But, it is 2017 and we can only hope the NHS masses will soon move on to an evergreen browser.

Do you have benchmarks for QEWD. I have my own Node to Cache binding that I have developed for an RPC solution and I found I could get twice as much throughput of marshalled TCP messages compared to CSP requests. But then I can never be sure with these types of benchmarks unless put into production environment.

0 0
Sean Connelly · Apr 21, 2017

I had some interesting results using just a TCP binding between node and cache.

With just one single node process and one single cache worker process I was able to process 1200 JSON RPC 2.0 messages per second. This included Cache de-serialising the JSON, calling its internal target method, writing and reading some random data and then passing back a new JSON object. Adding a second Cache process nearly doubled that throughput.

I was running node, cache and the stress test tool on the same desktop machine with lots of other programs running. I started to hit limits that seemed to be related to the test tool, so I wasn't sure how high I could take these benchmarks with this set up.

Interestingly when I bypass node and use a CSP page to handle the requests I could only get the same test set up to process 340 messages per second. This I couldn't understand. I am sure it was to do with the test tool, but could not work out how to get this higher. I would have expected Cache to spin up lots of processes and see more than the 1200 that were limited by one process.

It did make me wonder that no matter how many processes you have, you can only process 2 to 3 at a time per 4 CPU cores and that maybe Node was just much faster at dealing with the initial HTTP request handling, or that spreading the load between the two was a good symbiotic performance gain. Still I was not expecting such a big difference.

Now I would have thought, if you put Node.JS on one box and Cache on a second box so they don't compete for the resources they need most, that the TCP connection would be much more efficient than binding Node and Cache in the same process on the same box?

0 0
Sean Connelly · Apr 21, 2017

I can remember reading your previous post about the bottleneck before. I just wonder if such a fix would make the V8 less secure and as such would be unlikely to happen under the current architecture.

I did think about the problem for a while and I managed to get a basic AST JavaScript to Mumps compiler working. Its just a proof of concept and would require much more effort. Coercion missmatch is the biggest barrier. I came up with 1640 unit tests alone as an exercise to figure out how big the task would be. And that's just 1/20th of the overall unit tests it would require.

Effectively you would write JavaScript stored procedures inline that would run on the Cache side. I'm still not sure of the exact syntax, but it might be something like

db("namespace").execute( () => {
  //this block of code is pre-compiled into M
  //the code is cached with a guid and that's what execute will call
 //^global references are allowed because its not real JavaScript
  var data=[];
  for k1 in ^global {
    for k2 in ^global(k1) {
      data.push(^global(k1,k2));
    }
  }
  return data;
}).then( data => {
  //this runs in node, data is streamed efficiently in one go
  //and not in lots of costly small chunks
  console.log(data)
})


In terms of benchmarks, I would like to see the likes of Cache and ilk on here...

https://www.techempower.com/benchmarks/#section=data-r13&hw=ph&test=update
 

0 0
Nikita Savchenko · Apr 20, 2017

Hello Sean!

I am a developer of WebTerminal project, where I was playing around WebSockets in Caché every day. Also I am experienced in CSP / REST as I use them often, and NodeJS/JS are my daily drivers.

Regarding to WebSocket and CSP/REST in Caché I need to say that it is probably about of how InterSystems designed their tech. 1 dedicated process corresponds to 1 user, and I have no idea about doing something here, because historically (and actually) Caché takes account on a number of clients this way and performs licensing, so anything that interrupts to this mechanism is probably not available for the hi-level programming. But actually, each Caché process takes around 3MB of memory, so it is not too much losses here.

For Caché long-polling it's a normal practice to hang the Caché process for a while. Because one process is dedicated for each client, it won't affect other clients connected, and won't consume processor resources while hanging. I cannot tell more from the practical point of view here, as I used WebSocket whenever possible, which... Acts pretty much the same way. Also I haven't ever heard of firewall blocking WebSockets.

Yes, Facebook, Twitter and other big internet guys are using long polling, just because there is no reason to change their communications to WebSocket protocol, long-polling works well, and actually there is only one reason not to do it: around 6% of clients still not adopted to use them (IE9, Android 4.3 and lower, see here).

As for 2017, I will recommend using WebSockets wherever possible. If your clients may use outdated tech, which actually matters for Facebook, then I recommend just going with http long or short polling, depending on your application needs, without any WebSockets and http fallbacks.

But I cannot say that I am happy with the way how WebSockets implemented in Caché. For a single-client applications it's okay, but however implementing the clean pub-sub over WebSockets in Caché needs to deal with globals (which sometimes syncs with disk) or use interprocess communication (which is not effective sending big chunks of data), which creates additional hangs or lacks in flexibility.

To summarize. If ~5% of outdated clients is a big deal for you, I would recommend using short-polling for some of the applications, which have no need to deliver data simultaneously, and long-polling for primarily everything, without overloading the communication with WebSocket.

2 0
Sean Connelly · Apr 21, 2017

Hi Nikita,

Sounds like an interesting plan.

I've developed my own desktop grade UI widget library. I used to use ExtJS but got fed up with the price and speed. I've got it to the stage where I can build the shell of an IDE that could be mistaken for a thick client installation. If you right click the image below and open in a new tab you will see that it has all the features you would expect of an IDE. Split panel editing, drag panels, accordions, trees, menus, border layouts etc, and an empty panel that needs a terminal!

I have syntax highlighting working for a variation of M that I have been working on for a few years. I can get the syntax highlighting working for COS no problem (well long form at least).

The hard stuff would be getting things like the studio inspector working like for like. Lots of back end COS required etc.

I've still got a few months working on an RPC messaging solution for the UI but once thats done I would be open to collaborating on a back-end implementation for Cache.

Sean.

0 0
Rob Tweed · Apr 21, 2017

Chris Munt would be the person to ask, but I'm not aware that the lack of a resolution to the bottleneck issue is anything to with security - more that Google don't "get" the issue and see it as a priority, since it's an unusual requirement.

Anyway cache.node is an official InterSystems interface so why not just use it instead of rolling your own?  Ideally ISC should be pushing Google to fix the bottleneck.

0 0
Nikita Savchenko · Apr 22, 2017

Hi Evgeny!

Could you please create an issue on GitHub describing:

1. Did it happen before/with prevoius versions, symptoms;

2. Closing status and overal pooling time of the closing socket;

3. Which type of connection and Caché you use (https?);

4. The WebSockets frames dump will also be extremely useful (take a screenshot from Developer Tools -> Network -> Select "WS" and click on WebSockets channel -> Click "Frames".

0 0
Rob Tweed · Apr 21, 2017

QEWD (and EWD.js before it) has been using WebSockets for years with Cache at the back-end.  Of course, in this case, WebSocket support is provided by the Node.js socket.io module, so it avoids any COS-related issues.  It works extremely well and is very fast, stable and scalable.  Personally I'd always use WebSockets in preference to Ajax for browser to/from back-end communication.

However, one of the tricks QEWD pulls off is to allow you to transparently switch between Ajax and WebSockets as the transport for browser communication - for you, the developer, there's no difference in how you transfer messages, so you can choose the transport that is right for you via a simple configuration switch.  See:

https://www.slideshare.net/robtweed/ewd-3-training-course-part-14-using-...

0 0
Rob Tweed · Apr 21, 2017

See my recent article on QEWD for the rationale behind its architecture:

https://robtweed.wordpress.com/2017/04/18/having-your-node-js-cake-and-e...

As to benchmarks, the ewd-qoper8 module comes with a benchmarking app - even on a basic Linux VM setup, I can easily get in excess of 10,000 message round-trips per second.  Of course, once you add QEWD's access to Cache via cache.node, that will drop.  cache.node is a bottle-neck currently, for V8-related reasons described here:

https://groups.google.com/forum/#!searchin/v8-dev/bottleneck%7Csort:date...

It would be great if Google could fix this as we'd then get native COS performance via JS

As to moving parts, QEWD is entirely Node.js-based, so no extra moving parts other than Node.

1 0
Rob Tweed · Apr 21, 2017

Just to clarify - despite the bottleneck I've mentioned, cache.node will be a LOT faster than a TCP-based Node.js interface.  cache.node runs in-process with the Node.js process, and uses the Cache call-in interface.  As a result, QEWD is very fast and my guess is that it should outstrip an equivalent app using CSP.

Also, note that with QEWD, the socket.io connections are between the browsers and the QEWD master process, and NOT the child/worker processes that connect to Cache.  So Cache is decoupled from the web-socket and, indeed the HTTP(S) interfacing.

0 0
Rob Tweed · Apr 21, 2017

cache.node will handle in excess of 90,000 global sets/sec in its in-process mode.  If that V8 bottleneck was sorted out, we'd have COS performance for JS accessing a Cache database (ie > 1,000,000 global sets / sec)

With QEWD (actually ewd-qoper8), you find that for simple message handling, optimum throughput is when the number of Node.js processes (master + workers) equals the number of CPU cores.  That's a Node.js context switching issue, nothing to do with Cache.

QEWD allows you to configure cache.node to work in networked mode, in which case you can have Cache running on a separate server to the Node.js machine.  I've not compared performance, but it's a simple QEWD startup-file setting change to switch modes, so easy to compare if you want to give it  a try.

QEWD is very quick and easy to set up on a Windows machine where Cache is already installed.  See:

https://www.slideshare.net/robtweed/installing-configuring-ewdxpress

0 0