Working Around SOAP Service Timeouts

Some of our applications provide SOAP services that use “DSTIME”-based SQL queries that return records that have recently been added or changed. Since the records don’t change often, these queries usually return a small number of records and therefore take little time.

However, we sometimes make a table change that affects all records in that table. When that happens, on the next SOAP request from a SOAP client the service will run its query which will take an extra-long time because all records are included (for our apps, the queries return hundreds of thousands of records in this case).

The amount of time for producing the results therefore sometimes exceeds the default “timeout” specified in the CSP gateway connection for the instance hosting the SOAP service. This results in the connection being closed before the client gets its requested data and the client instead gets an error message like the following:

ERROR #5002: ObjectScript error: InvokeClient+208^%SOAP.WebClient.1

We can confirm the error is due to a timeout by viewing SOAP error details on the client using the “InterSystems IRIS SOAP Log” feature described here.

The following output from the log file confirms the timeout error:

Input to Web client with SOAP action = http://www.intersystems.com/user/SOAPTest.SlowSOAPService.GetSlowService

ERROR #5922: Timed out waiting for response
string**** SOAP client return error. method=GetSlowService, action=http://www.intersystems.com/user/SOAPTest.SlowSOAPService.GetSlowService
     ERROR #5922: Timed out waiting for response

To address this error, we added code to set a connection timeout to the instance of the SOAP client before calling the SOAP method that uses that one query. We had run the query on the Management Portal’s SQL page, writing the output to a csv file, and noted that it took over 6 minutes to complete. We therefore added this code to increase the timeout to 10 minutes before calling the service (the names were changed to protect the innocent 😊):

    #dim result As %XML.DataSet
    set wc = ##class(SlowSOAPService.WebClient.SlowSOAPServiceSoap).%New()
    set sc = wc.TimeoutSet(600)
    set sc = wc.GetSlowService(.result)

The %SOAP.WebService class provides the “TimeoutSet” method that only changes the connection timeout value for this one request. It doesn’t affect the CSP gateway connection’s timeout value, which by default is a short 30 seconds (to incentivize writing efficient server-side code).

If fact, to preserve that incentive, we can ensure that the extended timeout is only used in the rare cases we need it by wrapping the client code in conditional code that checks a global, as follows:

    #dim result As %XML.DataSet
    set wc = ##class(SlowSOAPService.WebClient.SlowSOAPServiceSoap).%New()
    if (^reallySlowSvcExpectedTime > 0) {
        set sc = wc.TimeoutSet(^reallySlowSvcExpectedTime)
    }
    set sc = wc.GetSlowService(.result)

The global acts as both a flag (greater than 0? Set the timeout) and the number of seconds to wait. For the issue described at the start of this article, we would notify clients that they should temporarily set a global using an IRIS terminal connected to their instance as follows:

    set ^reallySlowSvcExpectedTime = 600

Once we’re again expecting the service to quickly return a small amount of data, clients can be reset to use the default timeout by setting the global back to 0:

    set ^reallySlowSvcExpectedTime = 0