No, JVM settings are not the reason.

It seems to me the reason is the way how the XDBC Java wrapper is implemented. There are two things I don't really like about the code:

1. Statements created by an instance of XDBC Connection objects are being put in a List which keeps growing and growing. Even if I call Statement.close(), it will not be removed from the list immediately. The list is cleared only when the Connection.close() method is called.

2. Less likely, but this could also be the cause of the problem: no ResultSet.close() calls. The specification says:

https://docs.oracle.com/javase/7/docs/api/java/sql/Connection.html#close()

It is strongly recommended that an application explicitly commits or rolls back an active transaction prior to calling the close method. If the close method is called and there is an active transaction, the results are implementation-defined.

Modern JDBC drivers try to close dependent objects cascadely on Connection.close() but it is not always the case, and it is recommended to always close them explicitly.

In particular, someone complained about the similar issue with the Postgres JDBC driver here. And in the XDBC code I see that it tries to close and release all Statement instances but not ResultSets. On the other hand, Postgres driver maintainers reported that namely this issue was fixed.

1. It is officially declared that XDBC is the most modern remote data access technology

2. I use remote data sources very extensively and have tried everything - linked tables, Ens.* business operations and adapters, foreign tables, and XDBC. XDBC looks like the most simple and attractive data access technology. Using foreign servers and tables you need to describe both the server and the tables. Using XDBC you work with any ODBC or JDBC data source transparently.
In particular, I have had bad experience with both Postgres linked tables and with Postgres foreign tables, especially when remote/Postgres tables contain columns of type `text` or `bytea`. In this case, IRIS often silently returns an empty dataset without any errors even if a remote table is actually not empty.

Hi Enrico,

  1. I'm aware of that warning "FOR INTERNAL USE" in the source code, but the way I create XDBC connections is officially documented

  2. The OS is Ubuntu, IRIS is containerized, the memory is consumed by the IRIS Java language server

  3. Regarding memory leaks, I suspect that ResultSets are not closed properly - method %XDBC.Gateway.JDBC.ResultSet::Close() is empty and does nothing but it seems to be it should call ..%externalResultSet.close(). But I'm not sure it is the only reason

  4. UPD: calling Java GC really helps:

Set gateway = $system.external.getGateway("%Java Server")
Do gateway.invoke("java.lang.System", "gc")

```

Sure, here it is:

Class User.TestService Extends EnsLib.Kafka.Service
{

Method OnProcessInput(pInput As %Net.Remote.Object, Output pOutput As %RegisteredObject) As %Status {
    Set tSC = $$$OK
    Try {
        Set tMsg = ##class(EnsLib.Kafka.Message).%New()
        Do tMsg.FromRemoteObject(pInput)

        #Dim row as Test.Stat
        Set row = ##class(Test.Stat).%New()
        Set row.Topic = tMsg.topic
        Set row.Key = tMsg.key
        // row.Ts - generated automatically in the Test.Stat constructor
        Do row.%Save()
        $$$LOGINFO("Saved")
    }
    Catch (ex) {
        Set tSC = ex.AsStatus()
    }
    Quit tSC
}

}

I finally managed to solve the problem in Python. It's not perfect but it works:

Class User.Timer Extends %RegisteredObject
{

Property Executor [ Private ];

Method Initialize(maxWorkers As %Integer = 4) [ Language = python ]
{
import concurrent.futures
import time
import threading

self.Executor = concurrent.futures.ThreadPoolExecutor(max_workers=maxWorkers)
}

Method Close() [ Language = python ]
{
if self.Executor:
    self.Executor.shutdown()
}

Method Greet(name)
{
    Write "Hello ", name, !
}

Method OnCallback0(methodName As %String) [ Private ]
{
    Do $METHOD(instance, methodName)
}

Method OnCallback1(instance As %RegisteredObject, method As %String, arg1) [ Private ]
{
    Do $METHOD(instance, method, arg1)
}

Method OnCallback2(instance As %RegisteredObject, method As %String, arg1, arg2) [ Private ]
{
    Do $METHOD(instance, method, arg1, arg2)
}

Method OnCallback3(instance As %RegisteredObject, method As %String, arg1, arg2, arg3) [ Private ]
{
    Do $METHOD(instance, method, arg1, arg2, arg3)
}

Method OnCallback4(instance As %RegisteredObject, method As %String, arg1, arg2, arg3, arg4) [ Private ]
{
    Do $METHOD(instance, method, arg1, arg2, arg3, arg4)
}

Method OnCallback5(instance As %RegisteredObject, method As %String, arg1, arg2, arg3, arg4, arg5) [ Private ]
{
    Do $METHOD(instance, method, arg1, arg2, arg3, arg4, arg5)
}

Method InternalRun(delayMs As %Integer, wait As %Boolean, instance As %RegisteredObject, method As %String, args... As %List) [ Internal, Language = python ]
{
import time
import iris

if not self.Executor:
    raise Exception("The 'Initialize' method has not been called.")

def worker_function():
    time.sleep(delayMs / 1000)
    if len(args) == 0:
        self.OnCallback0(instance, method)
    elif len(args) == 1:
        self.OnCallback1(instance, method, args[0])
    elif len(args) == 2:
        self.OnCallback2(instance, method, args[0], args[1])
    elif len(args) == 3:
        self.OnCallback3(instance, method, args[0], args[1], args[2])
    elif len(args) == 4:
        self.OnCallback4(instance, method, args[0], args[1], args[2], args[3])
    elif len(args) == 5:
        self.OnCallback5(instance, method, args[0], args[1], args[2], args[3], args[4])
    else:
        raise Exception("Too many arguments.")
    return 0


future = self.Executor.submit(worker_function)

# wait == 0 means fire-and-forget
try:
    if (wait == 1):
        rv = future.result()

except Exception as e:
    print(f"{e}")
}

/// delayMs  - the parameter specifies the timer delay in milliseconds
/// wait     - if the parameter is false, the process will not wait for the Future result to be returned (fire-and-forget)
/// instance - any object which method should be called with a delay
/// method   - specifies the object's callback method name
/// args     - the callback method arguments (up to 5)
Method Run(delayMs As %Integer, wait As %Boolean, instance As %RegisteredObject, method As %String, args... As %List)
{
    Do ..InternalRun(delayMs, wait, instance, method, args...)
}

ClassMethod Test()
{
    Set obj = ##class(Timer).%New()
    Do obj.Initialize()
    Do obj.Run(1000, 0, obj, "Greet", "John")
    Do obj.Run(2000, 0, obj, "Greet", "Jessica")
    Write "If 'wait == 0' this line will be printed first", !
    Do obj.Close()
}

}