Stream Compression

Question

Question

Paul Riker · Nov 7, 2022

Does anyone have insight into how the Stream Global Compression works in 2022? I'm particularly interested in the XDSb repository on the Edges. As I understand it anything new will be compressed. Is there a compression method that is used that can be utilized by other code? How are the retrievals aware that the contents are compressed in order to de-compress them?

Product version: IRIS 2022.1

Discussion (8)4

Log in or sign up to continue

score 2 · Answer 1 · 2022-11-07T16:05:29-05:00

Hi Paul,

IRIS use gzip/zlib to compress and decompress data. You can see more information in the documentation:

https://docs.intersystems.com/iris20222/csp/docbook/Doc.View.cls?KEY=RCOS_vzeos

https://docs.intersystems.com/iris20222/csp/docbook/DocBook.UI.Page.cls?KEY=GIOD_tcp#GIOD_tcp_openusekeywords

https://docs.intersystems.com/irislatest/csp/docbook/DocBook.UI.Page.cls?KEY=GOBJ_propstream#GOBJ_propstream_gzip

score 1 · Answer 2 · 2022-11-08T05:25:26-05:00

Hello Paul,

The compression is using "zstd" type of compression, comming from %occStream.inc
The function that is used is: $System.Util.Compress(%data,"zstd")

score 0 · Answer 3 · 2022-11-15T10:28:27-05:00

In regards to by XDSb portion of my question, I found this video from Mark Bolinsky that states that in order to compress existing streams, you just write a routine that reads then saves each existing stream.

https://www.youtube.com/watch?v=ZiGrlslPpQU&t=380s

score 0 · Answer 4 · 2023-02-23T05:25:59-05:00

Akshay Pandey · Feb 23, 2023

Hi @Paul Riker
Did you managed to make this work?

I don't find it in HS 2022.1 or 2

0 0

score 0 · Answer 5 · 2023-02-23T07:31:52-05:00

Eduard Lebedyuk · Feb 23, 2023

Check out this repo.

0 0

score 0 · Answer 6 · 2023-03-02T02:57:38-05:00

I have already check out this repo not working in my case

I have scanned PDF stored in repo edge in Stream binary

score 0 · Answer 7 · 2023-02-27T09:26:41-05:00

We haven't attempted yet. May be a while. I was told that IRIS actually performs the compression.

https://docs.intersystems.com/iris20222/csp/docbook/Doc.View.cls?KEY=GOB...

score 0 · Answer 8 · 2023-02-27T14:44:17-05:00

https://docs.intersystems.com/hs20222/csp/docbook/DocBook.UI.Page.cls?KE...

"19. Beginning in version 2022.2, HealthShare offers stream compression for your Edge Gateways, and provides a conversion utility to compress existing streams in the ECR database at each Edge Gateway. The conversion process is designed to not interfere with regular Edge Gateway data processing, and should complete within a day. ..."