%PARALLEL and Per Process Memory issue
HI *,
So last night I was doing a deployment which had a query to select from a large dataset of joined tables, to iterate through and populate data in a new table.
This was done in a classmethod using embedded SQL and the %PARALLEL keyword.
On a testing environment with the same dataset, this worked perfectly a few days earlier.
It was going to be a long-running process, so we started it from a csession using the JOB command.
Then get the PID with $zchild, go to Processes and monitor it there.
It would terminate after a minute or so with a <STORE> error.
The per process memory is set at 1GB. While the query is executing, even before starting to loop through the cursor, the memory used will just continue increasing until it reaches the limit, and then terminates.
Many other processes running on the system every day use %PARALLEL to work through large datasets, without any issues, which baffled me a bit.
Eventually, I removed the %PARALLEL from the query and started the process, and it was completed without issues. I just tried it as a last resort before contacting support.
What I did notice when not using %PARALLEL, is that PPGs are being used during the query execution, as where with the %PARALLEL it does something to the particular process' memory which I do not understand.
Does this have something to do with starting the process from csession?
Is ti something specific to %PARALLEL which I should consider, to ensure my other processes do not break?
It will be greatly appreciated if anyone can provide me with some insight on why this might have happened.
The product is Ensemble 2018.1.
Database Cache: 102400 MB
Cache for UNIX (Red Hat Enterprise Linux for x86-64) 2018.1.8 (Build 766_0_22335U)
Thank you,
Comments
- PPG is a nice feature if your local data might break partition limits.
- Without knowing details on the internals of %PARALLEL:
- It is given that it runs in more than a single process
- but sharing PPG over multiple processes is not known
- so a lot of data may accumulate in some of the parallel processes.
I have found some information on this, which was highlighted by another issue we had.
gmheap | Configuration Parameter File Reference | InterSystems IRIS Data Platform 2023.3
The gmheap setting was too small.
Someone else might find this helpful in the future.