Interoperability <STORE> error with a File inbound adapter BS

Question

Question

Yaron Munz · Aug 10, 2021

#Business Service #Interoperability #Ensemble

Hello,

We have a simple BS that Extends Ens.BusinessService with an ADAPTER = "EnsLib.File.InboundAdapter";
The "incoming" folder has more than 4M files, so we get the following error:
ERREUR <Ens>ErrException: <STORE>zFileSetExecute+38 ^%Library.File.1

Is there any simple workaround this ?

Product version: IRIS 2020.1

$ZV: IRIS for Windows (x86-64) 2020.1 (Build 215_0_20763U) Mon Apr 12 2021 10:08:45 EDT

Discussion (7)2

Log in or sign up to continue

Pravin Barton · Aug 12, 2021

Here are a couple of ways to avoid <STORE> errors by increasing the per-process memory available to IRIS processes:

Increase the 'bbsiz' parameter, either by editing the CPF file or in the System Management Portal under System Administration > Configuration > System Configuration > Memory and Startup.
In code in the specific process throwing the <STORE> error, set the $zstorage special variable to increase the memory available to that process.

1 0

score 0 · Answer 1 · 2021-08-10T09:03:08-04:00

Not sure if it could be a simple workaround, but I created a FileControl mechanism where I create my own queue of files for the business service to process. I add files to the queue through an Outbound File Adapter so they can get processed by modified Inbound Adapters

score 0 · Answer 2 · 2021-08-10T09:08:18-04:00

Hi

Did you check if the disk is full? Or may its a O.S. problem:

"You can put 4,294,967,295 files into a single folder if drive is formatted with NTFS (would be unusual if it were not) as long as you do not exceed 256 terabytes (single file size and space) or all of disk space that was available whichever is less. For older FAT32 drives the limits are 65,534 files in a single folder and max file size of 4 Gigabytes and 2TB of total space or all of disk space that was available or whichever is less."

https://answers.microsoft.com/en-us/windows/forum/all/how-many-files-can....

score 0 · Answer 3 · 2021-08-10T09:32:57-04:00

Mario,

The OS is totally irrelevant here !

The problem is with the IRIS %Library.File class that uses a $ZSEARCH to scan the directory, and gives a <STORE> error due to being unable to fully scan the directory

score 0 · Answer 4 · 2021-08-10T09:41:02-04:00

Oliver Wilms · Aug 10, 2021

Can you add more memory?

0 0

score 0 · Answer 5 · 2021-08-10T12:32:00-04:00

Hi, Yaron
It's me, NOT MARIO.

Analyzing the %Library.File code, I think that is not a $ZSEARCH problem, but a local variable, highlighted in the image below. Maybe, the consume all memory space.
I recommend you open a official bug ticket at InterSystems to proper response.

score 0 · Answer 6 · 2021-08-13T11:10:53-04:00

I agree with Pravin that <STORE> errors generally indicate a lack of process private memory which you can try to address by raising that limit. Documentation on <STORE> errors here: https://docs.intersystems.com/irislatest/csp/docbook/DocBook.UI.Page.cls?KEY=RCOS_vstorage#RCOS_C132257

The other practical side of things though - can you use smaller subfolders rather than one that needs to host so many files? Do you really want your process to have to scan through that many files?