SAX XML Parser-error <STORE> on large xml import

Question

Question

Arto Alatalo · May 30, 2018

%XML.Reader.OpenFile fails with SAX XML Parser-Error: <STORE> on importing large (1.5 GB) xml file. Any workaround for it?

My goal is one-time import of the file using classes generated by its XML schema.

Discussion (8)3

Log in or sign up to continue

Timothy Leavitt · May 30, 2018

One option is to increase $zstorage (see documentation) to let the process use more memory; since it's just a one-time import, this may be simplest. Another option is to set the UsePPGHandler property to true (1) on the instance of %XML.Reader - this has a negative impact on performance.

1 0

score 0 · Answer 1 · 2018-05-30T15:03:50-04:00

Arto Alatalo · May 30, 2018

Thanks, $zstorage worked fine

0 0

score 0 · Answer 2 · 2018-05-30T08:53:14-04:00

Timothy Leavitt · May 30, 2018

[edit: moved to Answer rather than comment]

0 0

score 0 · Answer 3 · 2018-05-30T15:02:36-04:00

Arto Alatalo · May 30, 2018

Thanks but I must use xml-to-object classes generated by the xsd

0 0

score 0 · Answer 4 · 2018-05-30T15:09:58-04:00

Understand. I haven't tried this myself yet but in theory you should be able to initialize the Reader with an XML.Document object. This Document can be initialized (%OnNew) with a global of your choice in the same manner that TextReader can.

score 0 · Answer 5 · 2018-05-30T15:20:44-04:00

As I see it, xml-to-objects classes need DOM while the whole purpose of TextReaded is to read the xml without DOM creation

score 0 · Answer 6 · 2018-05-30T15:32:54-04:00

Correct. The alternative using TextReader was to "avoid" increasing memory but if you have that option then the answers provided by Timothy are the way to go.

score 0 · Answer 7 · 2018-05-30T10:18:05-04:00

For large files I use %XML.TextReader.ParseFile(). It allows me to pass a "concrete" global instead of using Cache defaults. Usually CacheTemp which is more limited than a concrete global. With this I'm able to process larger files. Granted, it's a little bit of more work (code) as you have to traverse the results (element by element) but it gets the job done.