Is there some tricks to make $system.obj.load() faster ?

Question

Question

Norman W. Freeman · Oct 26, 2024

I'm using $system.obj.load() to synchronize a folder contains many classes (CLS) in UDL format. I use "-d" as flag (compilation is done later one). It's already done in parallel to make it as fast as possible but I was wondering if there was way to make this even faster, for example using some tradeoffs (eg: by turning off some features before loading them all (that would be re-enabled later on)). It's currently taking 250 seconds to import 3000 classes. MAC file and GBL import is much faster.

Product version: IRIS 2021.1

$ZV: IRIS for Windows (x86-64) 2021.1 (Build 215U) Wed Jun 9 2021 09:39:22 EDT

Discussion (5)2

Log in or sign up to continue

score 1 · Answer 1 · 2024-10-27T13:29:56-04:00

Use XML format and you get 6x faster load than using UDL.

My guess the difference is due to the parsing needed by UDL import.

My estimate in your case you can get as low as 42 seconds, I'm curious to see the actual numbers.

score 0 · Answer 2 · 2024-10-27T15:58:50-04:00

I tried that and what happen is weird : the CPU usage of IRISDB.exe processes (4 of them used as workers) fall back to 0-1% while before it was peaking 25% (on a 4 cores machine, so 100% of the CPU was used). Despite this, it takes as much time as before, if not even more. There might be some bottleneck. I don't think it's I/O because importing MAC file is definitely faster (and they just as big as CLS files).

score 1 · Answer 3 · 2024-10-27T16:22:32-04:00

.MAC and much more .INT are an almost 1:1 image of the runtime .OBJ code
they get implicitely compiled by ZSAVE command
while any .CLS gets first stored with a lot of internal parameters where XML is faster than UDL
then it's translated to .INT(at least) including all extra params,
and inherited pieces and other features for comfort
and then compiled as any other .INT for the final .OBJ

So comparing .CLS to .MAC/.INT is like comparing a goat to the final cheese you buy.

score 0 · Answer 4 · 2024-10-28T04:50:52-04:00

If some of those 3000 classes are divided to different packages, I would try to do the load in "segments" (using the queue manager, to have this done in parallel). This might speed things up a bit.

score 0 · Answer 5 · 2024-10-28T14:19:56-04:00

Thanks for the suggestion. I have tried to group CLS files to be loaded into clusters of 256 items, each cluster is then sent to a worker (instead of worker getting one CLS at a time). This increase chance of worker working exclusively on one package. In the end it's roughly same time. I don't wanna load them by package as packages are not balanced (some have 10 classes, some 500).