Question
Kevin B Lavallee · Sep 30, 2022

%ToJSON not detecting open websocket as current device

Hello,

I am trying to use the %ToJSON method on my dynamic object, calling it with a "DO" and not passing in outstrm parameter.

We are trying to avoid MAXSTRING errors that we get with some of our abnormally large messages.  In order to do this, I am trying to update my code to not call the socket's "Write" method after converting the dynamic object to a JSON string using the %ToString method's output.  Per the documentation on the %ToJSON method:
    If outstrm is not specified and the method is called via DO, the JSON string is written to the current output device

The previous code (extending from %CSP.WebSocket of course) was doing this:

Method Send(message As %Library.DynamicObject) As %Integer
{
    do ..Write(message.Msg.%ToJSON())
    quit 1
}

The updated method is doing this:

Method Send(message As %Library.DynamicObject) As %Integer
{
    do message.Msg.%ToJSON()
    quit 1
}

When I execute the new code it does not write the JSON object directly to the websocket.  It is as if the function does not see the websocket as being the current device.

Any suggestions would be greatly appreciated.

Product version: IRIS 2022.1
$ZV: IRIS for UNIX (Red Hat Enterprise Linux 8 for x86-64) 2022.1 (Build 209_0_21727U)
1
0 143
Discussion (9)2
Log in or sign up to continue

The documentation of the %ToJSON() method is correct and yes, you can do 

 do obj.%ToJSON()
 

merely, this works only "on devices without protocol" like terminal, (sequential) file, etc. Everywhere, wehere the data bytes goes direct to the target. WebSocket isn't such a device. There is a "header part", with information fields like the command, masking, the length of the data, etc.

You have two possibilities, a) you ask WRC for a "WriteStream()" method or b) you handle the whole WebSocket by hand (is not impossible) or c) you change your application logic and send the those messages in chunks.

Thank you Julius for explaining why this does not work on websockets and for the suggestion about the WriteStream method.  I did use Nicholai's sample method to implement the changes to work around this issue.  Much appreciated.

Why do not pass a stream to %ToJSON, and then Write the stream content?

Set stream = ##class(%Stream.TmpCharacter).%New()
Do message.Msg.%ToJSON(stream)

But, I'm sure that you will not be able to send the whole big stream as a whole message. You still have to split it and receive it knowing that it's not complete.

According to WebSocket protocol, the maximum payload size is (2**(8*8))-1 octets, if I recall it right.

Protocol and its implementation

I've looked at the realization in IRIS, and it does not support any streams, So, any call of Write, is like a complete message and it accepts only string.

Hence I wrote to OP, quote from my answer, "you ask WRC for a 'WriteStream()' method" 

The %ToJSON method sends JSON text to a device or %Stream one element at a time.  Starting in IRIS 2019.1.0 it broke up long JSON strings into blocks of 5460 characters so that strings exceeding the maximum length could be sent to the device.  Make sure you are using an IRIS 2019.1.0 or later if you are getting a <MAXSTRING> signal.

In a future IRIS release (now out in a preview release) a change was made such that sequences of many small items would be blocked together and sent to the device in a larger buffer.  This is being done to improve the performance of %ToJSON method when sending many small elements to a %Stream.

I think what you'll need is a write stream method below (I haven't had a chance to test this so the code comes with no guarantees). This will work for Non-Shared web-socket connections since I don't have the chance to test with shared pools at the moment.

Method Send(message As %Library.DynamicObject) As %Integer
{
    Set stream = ##class(%Stream.TmpCharacter).%New()
    Do message.Msg.%ToJSON(stream)
    Set sc = ..WriteStream(stream)
    If $$$ISERR(sc) $$$SysLog(2,"WebSocket","[Write] Error WRITE Stream on JSON Command",sc)
    quit 1
}

Method WriteStream(data As %Stream.Object) As %Status
{
	Set $ZTrap="WriteError"
	If i%WSClassProtocolVersion > 1 & i%WSDataFraming = 1 {
		Set head=$ZLChar(data.Size)
		If i%BinaryData = 1 {
			Set head=head_"8"
		} Else {
			Set head=head_"7"
		}
	} Else {
		Set head=""
	}
	#; As there is activity on this session reset the session timeout
	Set sc=$$updateTimeout^%SYS.cspServer(i%SessionId) If $$$ISERR(sc) $$$SysLog(2,"WebSocket","[Write] Error updating session timeout",sc)
	#; Only return an error status if there's an issue with the write itself.
	Set sc=$$$OK
	If (i%SharedConnection = 1) {
        // If you want to try and play with the shared pools and large payloads, move the commented code to the loop below

		#; Set sc=$$CSPGWClientRequest^%SYS.cspServer3(i%GWClientAddress,"WSW "_i%WebSocketID_" "_head_data1,-5,.response)
		#; If $$$ISERR(sc) $$$SysLog(2,"WebSocket","[Write] Error sending request",sc)
        $$$SysLog(2,"WebSocket","[Write] Shared Connections Don't Support Stream Sizes over 3.6MB",sc)
        Quit $$$Error($$$GeneralError, "[Write] Shared Connections Don't Support Stream Sizes over 3.6MB")
	} else {
        SET sc = $$$OK
        // Write header
		Write head
        // Rewind Stream
        Do data.Rewind()
        // Loop through stream and write 64kb chunks
        while (data.AtEnd = 0) {
            // Set Buffer to 64KB
            Set BUFSZ = 65536
            // Read Buffer
            SET BUFBLOCK = data.Read(.BUFSZ)
            // Convert UTF8 if we aren't using Binary Web Sockets
            If i%BinaryData '= 1 {
                Try {
                    Set BUFBLOCK=$zconvert(BUFBLOCK,"O","UTF8")
                } Catch exp {
                    $$$SysLog(2, "WebSocket", "[Write] Exception: "_exp.DisplayString(), data)
                    Set BUFBLOCK=BUFBLOCK
                }
            }
            // Use DEVICE WRITE method
            Write BUFBLOCK
            // Quit when done with stream
            BREAK:(BUFSZ < 65536)
        }
        // Send a flush command mnow
        Write *-3
        Quit sc
	}
	Quit sc
WriteError	
	#; No interrupts during cleanup or error processing
	Do event^%SYS.cspServer2("WebSocket Write Error: "_$ZError)
	$$$SetExternalInterrupts(0)
	Set $ZTrap="WriteHalt"
	Hang 5
	Close 0
WriteHalt 
	Halt
    
}

Thank you Nicholai for this method.  I made a few modifications to the code but I very much appreciate the great starting point.