Consume data from websocket

Question

Question

Eduard Lebedyuk · Feb 4, 2019

I want to consume external websocket api, URL looks like this:

wss://site.com/ws/v2/?&token=<token>

Checked with external tool (Simple WebSocket Client) that websocket works and I can consume the data.

In Cache the relevant functionality is offered by %IO.Socket class.

set sock = ##class(%IO.Socket).%New()
set sock.SSLConfig = "MyEmptySSLConfig"
set sock.TranslationTable="UTF8"
do sock.Open("site.com/ws/v2/?&token=<token>","443", 10,.sc)

However I get this error:

ERROR #7109: Timed out after 10 seconds trying to open stream '|TCP|443|42881' [zOpen+41^%IO.Socket.1:IOT]

Any ideas on how Cache can consume websocket API are appreciated.

Discussion (17)1

Log in or sign up to continue

Robert Cemper · Feb 4, 2019

To me, this looks like your port doesn't like WS as initial connection protocol but expect the switch from HTTPS -> WSS or the port is just wrong.

several suggestions for the investigation to get it moving:

#1) verify your server from a normal web page (e.g. based websocketdem.csp in Samples) especially the port !!!!

#2) If you have control over your server then skip SSL and get running over HTTP -> WS first. You can add this once it works.

#3) If you have no server just use Caché / SAMPLES / Web.SocketTest.cls
I found it very useful to have control over both ends and now I own several variants for testing.

Staying tuned

0 0

Dmitry Maslennikov · Feb 5, 2019

And how it looks like just with telnet

$ telnet echo.websocket.org 80

Copy-Paste this data

GET / HTTP/1.1
Accept: */*
Host: echo.websocket.org
Connection: Upgrade
Upgrade: websocket
Sec-WebSocket-Key: 7BOhi3I1WkBoazaXv+MfWA==
Sec-WebSocket-Version: 13

After the empty line, it will show response

HTTP/1.1 101 Web Socket Protocol Handshake
Connection: Upgrade
Date: Tue, 05 Feb 2019 10:37:17 GMT
Sec-WebSocket-Accept: /gSfI5y+P3MMhONARUXNHG5vrHc=
Server: Kaazing Gateway
Upgrade: websocket

0 0

Dmitry Maslennikov · Feb 5, 2019

You should not forget that WebSockets is still used the HTTP protocol, so, you should send some headers first.

Look at this my code, it uses plain OPEN, and I'm not sure if my example 100% correct, but works.

  set securityKey = $SYSTEM.Util.CreateGUID()
  set securityKey = $SYSTEM.Encryption.MD5Hash(securityKey)
  set securityKey = $SYSTEM.Encryption.Base64Encode(securityKey)

  set host = "echo.websocket.org"
  set url = "/"
  set port = 80

  set device = "|TCP|"_port
  Open device:(host:port:"SCWD"::8192:8192:/TCPNOXY)
  Use device

  Write "GET ",url," HTTP/1.1",!
  Write "Accept: */*",!
  Write "Host: ",host,!
  Write "Connection: Upgrade",!
  Write "Upgrade: websocket",!
  Write "Sec-WebSocket-Key: ",securityKey,!
  Write "Sec-WebSocket-Version: 13",!
  Write !,*-3
  
  Use device:(::"A":$char(13))
  Set fullResponse = ""
  Do {
    Set response = ""
    Read response:1
    Quit:'$test
    Set fullResponse = fullResponse_response_$char(13)
  } While $test

  Use 0
  Close device

  Write !!,fullResponse

In this case, it only reads the first response, which actually should be with HTTP headers as well. Something like this.

HTTP/1.1 101 Web Socket Protocol Handshake
Connection: Upgrade
Date: Tue, 05 Feb 2019 10:51:37 GMT
Sec-WebSocket-Accept: qU2IAmlBvnSoEctnti8lcbc4bVA=
Server: Kaazing Gateway
Upgrade: websocket

It does not contain the first portion of data, which some WebSocket servers may send after initial connect. But if your server sends it, you should see it at the and of response. If you have to send something before, you should do it after the first response, which says that connection established and you can send any data. But not any, it should be in binary format, more details you can find here. Any responses also decoded.

3 0

Alexey Maslov · Feb 6, 2019

This is well-known classic approach how asynchronous behavior can be simulated in "synchronous only" language. It potentially suffers of two problems caused by the need to check if something has dropped to TCP connection:

1) Read x:timeOut (where timeOut>0) causes delays up to timeOut seconds which are more or less acceptable for background job but not acceptable for foreground (e.g., some UI handling).

2) Read x:0 is too unreliable.

Agree with Dmitry: it helps that "not all those tasks have to be asynchronous".

0 0

Robert Cemper · Apr 28, 2021

There is an example of the WebSocket Server in namespace SAMPLES on Caché only:
in addition there are 4 examples for Caché on Open Exchange

https://openexchange.intersystems.com/package/WebSocketsSample

https://openexchange.intersystems.com/package/WebSockets-Tutorial

https://openexchange.intersystems.com/package/WebSocketClient-js

0 0

Robert Cemper · Feb 5, 2019

OK.

I tried it and found: sock.Open( does not accept "/ws/v2/?&token=<token>" only the server name

Now you have the connection BUT no server to take care of it as you see:

USER>do sock.WriteLine("/csp/samples/Web.SocketTest.cls",1,.sc) zw sc sc=1 USER>write sock.ReadAny(32000,5,.rSC) zw rSC HTTP/0.9 404 Stream Not Found Content-Type: text/html; charset=utf-8 Date: Tue, 05 Feb 2019 10:45:16 GMT Expires: Tue, 05 Feb 2019 11:45:16 GMT rSC=1

Investigating the browser it became clear what's going on:
- you begin with HTTP to start your WebSocketServer
- an then initiate the WS connection.

This is the output from TcpTrace:

>>>>> GET /csp/samples/websocketdemo.csp HTTP/1.1 >> .......

<<<<< HTTP/1.1 200 OK << Date: Tue, 05 Feb 2019 10:55:01 GMT << Server: Apache <<<<...

>>>>> GET /csp/samples/Web.SocketTest.cls HTTP/1.1 >> ... >> Upgrade: websocket

<<<<< HTTP/1.1 101 Switching Protocols << Upgrade: websocket << Connection: Upgrade << Sec-WebSocket-Accept:

That's my point:
With the actual approach starting the requres WS Server is missing.

1 0

Robert Cemper · Feb 8, 2019

see my 2 alternative solutions to use external adapters

Client for WebSockets based on CSP

Client for WebSockets based on Node.js

and thanks again for the inspiration.
It was just in time for a business project.

2 0

score 0 · Answer 1 · 2019-02-04T20:59:00-05:00

#) I know websocket server works from the browser client, just not from Cache client.

#2) I do not have control (or even dev access) to the websocket server. It's not Cache-based through.

score 0 · Answer 2 · 2019-02-05T06:19:30-05:00

Robert Cemper · Feb 5, 2019

Great you confirmed my investigations !

0 0

score 0 · Answer 3 · 2019-02-05T11:01:33-05:00

Web Socket is an asynchronous protocol by its nature. How would you implement asynchronous client in ObjectScript?

score 0 · Answer 4 · 2019-02-05T12:40:13-05:00

I don't know exact task of Eduard, but I think, some time when you only one way to communicate with the server, and you need to do some task, why not to use it. But not all those tasks have to be asynchronous. Like here, you can connect, get response and disconnect, as well as with plain HTTP. Or connect, send request for some data, get response and disconnect. Websockets should keep connection alive. That's how it works, when you connected, you send any data, and get response, at any time, or get only response.

score 0 · Answer 5 · 2019-02-05T15:37:17-05:00

Eduard Lebedyuk · Feb 5, 2019

You can have a device which holds the connection and read from it when you want to.

0 0

score 0 · Answer 6 · 2020-08-26T07:07:02-04:00

Dmitriy,
1. I tried the open socket script, you wrote, and it works OK.

2.now I want to send some string messages on the opened socket,

how I do it ?

I have to add on each "MyMessage" string some header before ?

can you write exact script how I implement using send message(s), and handle get response on each

message.

did not find any docs about

score 0 · Answer 7 · 2021-04-26T04:25:06-04:00

Eduard Lebedyuk · Apr 26, 2021

Note that InterSystems IRIS supports WebSockets natively as a client and as a server.

0 0

score 0 · Answer 8 · 2021-04-28T03:38:26-04:00

Hi Eduard

I use cache 2018.1.4

IO.SocketClass

1. how to implement send JSON Array message via SocketClass ?

2. message that include file data(binary or base64) in one of the JSON Array "index" like:

["name", MyBinaryData, "12345"]

is the class methods limited by Cache String Max ?

if yes, how to bypass this obstucle ?

code examples on send / receive will help

score 0 · Answer 9 · 2021-04-28T03:51:26-04:00

Eduard Lebedyuk · Apr 28, 2021

Consider updating to InterSystems IRIS.

0 0

score 0 · Answer 10 · 2019-02-05T19:09:07-05:00

Out of curiosity, I have built a more exotic solution that is easier to understand. (At least for me)
I don't like so much this protocol upgrade stuff and encrypting and simulation of a browser.

My personal workaround:

start browser over ZF(-1, "start chrome http://.................my csp......")
the page gets my request passed with the URL. very simple just as a Hash
the page does all the WS stuff via JavaScript
the reply is returned using Hyperevent #call(....)

It works fine and is rather "classic CSP" style.