How to force binary hash algorithm in Ensemble production?
TL;DR: If I set an Ensemble Production Service based on EnsLib.File.PassthroughService to a Binary charset encoding, it breaks the file handling. Any ideas?
Full long post:
All,
I set up an Ensemble production to transfer files via SFTP, which works fine sending the files to my Linux server. Then I was informed that we needed to save a hash of the file, for comparison with the destination to verify the file arrived unmodified. I looked at the base64 encoded hash on my linux server:
G7QWAP6FcLInFWP8ECRL/EI2NfKdaf6TtrpwYuvSOEc=
and I created a class to extend the StreamContainer to add a hash value:
{ Property Hash As %String(MAXLEN = 256); }
which seems to work fine. But when the production runs, the hash is listed as:
Vn0ygZZN8DewD8KfKKYF8BTV9K1LsuFBEXnAkqXBEFs=
I narrowed that down to Ensemble automatically choosing the stream type - in this case, it sees it as a FC type stream - File Character - but Linux sees it as a Binary encoding. I verified the issue with this bit of code:
New bstream,cstream,sc,bhash,chash,zfile
Set zfile="d:\Patch\Test_File_1.CSV"
Set bstream=##class(%Stream.FileBinary).%New()
Set sc=bstream.LinkToFile(zfile)
Do bstream.Rewind()
Set bhash=$SYSTEM.Encryption.SHAHashStream(256,bstream)
W "Binary: "_$SYSTEM.Encryption.Base64Encode(bhash),!
Do bstream.%Close()
Set cstream=##class(%Stream.FileCharacter).%New()
Set sc=cstream.LinkToFile(zfile)
Do cstream.Rewind()
Set chash=$SYSTEM.Encryption.SHAHashStream(256,cstream)
W "Char: "_$SYSTEM.Encryption.Base64Encode(chash),!!
Do cstream.%Close()
Q
And when run, here's the output:
ENSDEMO>D ^HASHCHECK Binary: G7QWAP6FcLInFWP8ECRL/EI2NfKdaf6TtrpwYuvSOEc= Char: Vn0ygZZN8DewD8KfKKYF8BTV9K1LsuFBEXnAkqXBEFs=
I found a setting in the EnsLib.File.PassthroughService to manually set the charset to Binary, and when I set that, the production will give me the correct hash and sends the file - but then recreates the file with a randomized OriginalFilename, waits 5 seconds (or whatever's set in the Retry Interval parameter), and sends it again... and again... and again.
When I look at the message viewer, the <Stream></Stream> is empty:
<?xml version="1.0" ?> <!-- type: SFTP.HashFile id: 281 --> <HashFile> <OriginalFilename>C:\Export\aPvlvsYdsoNpng.CSV </OriginalFilename> <Stream></Stream> <Type>FB </Type> <Attributes> <AttributesItem AttributesKey="1" xsi:nil="true"></AttributesItem> </Attributes> <Hash>G7QWAP6FcLInFWP8ECRL/EI2NfKdaf6TtrpwYuvSOEc= </Hash> </HashFile>
Even though file is sent intact successfully. (You can see an example of the 'random' filename above.)
Any ideas that would be easier than rewriting a custom EnsLib.File.PassthroughService and/or Ens.StreamContainer to only handle binary encoded streams?
Thanks for any and all input!