· Dec 15, 2023

%PARALLEL and Per Process Memory issue

HI *,

So last night I was doing a deployment which had a query to select from a large dataset of joined tables, to iterate through and populate data in a new table.

This was done in a classmethod using embedded SQL and the %PARALLEL keyword.
On a testing environment with the same dataset, this worked perfectly a few days earlier.

It was going to be a long-running process, so we started it from a csession using the JOB command.
Then get the PID with $zchild, go to Processes and monitor it there.
It would terminate after a minute or so with a <STORE> error.
The per process memory is set at 1GB. While the query is executing, even before starting to loop through the cursor, the memory used will just continue increasing until it reaches the limit, and then terminates.

Many other processes running on the system every day use %PARALLEL to work through large datasets, without any issues, which baffled me a bit.

Eventually, I removed the %PARALLEL from the query and started the process, and it was completed without issues. I just tried it as a last resort before contacting support.

What I did notice when not using %PARALLEL, is that PPGs are being used during the query execution, as where with the %PARALLEL it does something to the particular process' memory which I do not understand.

Does this have something to do with starting the process from csession?
Is ti something specific to %PARALLEL which I should consider, to ensure my other processes do not break?
It will be greatly appreciated if anyone can provide me with some insight on why this might have happened.

The product is Ensemble 2018.1.
Database Cache: 102400 MB
Cache for UNIX (Red Hat Enterprise Linux for x86-64) 2018.1.8 (Build 766_0_22335U)

Thank you,

Product version: Caché 2018.1
Discussion (2)1
Log in or sign up to continue