File size on disk bigger than expected with file stream
Hello,
we use a NAS to archive stream from database to files. If we look at the file size on disk, it is bigger than expected.
For example :
File size is 127K
ls -alh | grep write-chupmb-pdf 127K 2020-07-02 10:43 write-chupmb-pdf
Size on disk is 192K
du -sh * | grep write-chupmb-pdf 192K write-chupmb-pdf
Or expected size is 128K (with 4K blocs)
I tried with %Stream.FileBinary, %FileBinaryStream and OPEN/USE/WRITE and with multiple files and file type (PDF, JPG, PNG) and I have the same problem each time.
For example :
du -sh * | grep write-chupmb-pdf 192K write-chupmb-pdf 192K write-chupmb-pdf-copyfrom 224K write-chupmb-pdf-raw 192K write-chupmb-pdf-tmpstream
And the code to generate these files
//stream variable come from database (%GlobalBinaryStream)
//write-chupmb-pdf-copyfrom
SET fileNAS = ##class(%Stream.FileBinary).%New()
DO fileNAS.FilenameSet("\\nas-chup-7d\TSarchives\2018\TEST\write-chupmb-pdf-copyfrom")
WRITE fileNAS.CopyFromAndSave(stream),!
DO stream.Rewind()
//write-chupmb-pdf
SET fileNAS = ##class(%Stream.FileBinary).%New()
DO fileNAS.FilenameSet("\\nas-chup-7d\TSarchives\2018\TEST\write-chupmb-pdf")
WHILE 'stream.AtEnd {
DO fileNAS.Write(stream.Read())
}
WRITE fileNAS.%Save(),!
DO stream.Rewind()
//write-chupmb-pdf-tmpstream
SET tmpStream = ##class(%Stream.GlobalBinary).%New()
DO tmpStream.CopyFrom(stream)
SET fileNAS = ##class(%Stream.FileBinary).%New()
DO fileNAS.FilenameSet("\\nas-chup-7d\TSarchives\2018\TEST\write-chupmb-pdf-tmpstream")
WRITE fileNAS.CopyFromAndSave(stream),!
DO stream.Rewind()
// write-chupmb-pdf-raw
SET fileNASRAW = "\\nas-chup-7d\TSarchives\2018\TEST\write-chupmb-pdf-raw"
OPEN fileNASRAW:("WSN")
USE fileNASRAW
WHILE 'stream.AtEnd {
WRITE stream.Read()
}
CLOSE fileNASRAWI also compared with streams written by other technologies than Caché and we have no problem with them.
Has anyone ever encountered this kind of problem ?
Caché version is : Cache for Windows (x86-64) 2017.2.2 (Build 865U) Mon Jun 25 2018 10:33:28 EDT
Thank you
Yann
Comments
What's your file I/O default?
zn "%SYS"
do ^NLS
Choose 4) Display loaded settings
Choose 3) Display system defaults for I/O tablesHere's what I got:
-------------- System defaults for I/O tables --------------
Table Name
----------------- ---------------
Process RAW
Cache Terminal UTF8
Other terminal UTF8
File UTF8
Magtape UTF8
TCP/IP RAW
System call RAW
Printer CP1251Here's what I got
-------------- System defaults for I/O tables -------------- Table Name ----------------- --------------- Process RAW Cache Terminal UTF8 Other terminal UTF8 File RAW Magtape RAW TCP/IP RAW System call RAW Printer RAW
Thank you
As a rule of thumb disks allocate files in blocks larger than the I/O block. This is called the allocation size or cluster size. I believe the default for large Windows disks is 64 kio. This page has some explanation: <https://docs.microsoft.com/en-us/Exchange/plan-and-deploy/deployment-re…;, search for NTFS allocation unit size.