.Stream files stored on server when using file streams

Question

Question

Julian Matthews · May 3, 2019

Hi all.

I have a process/operation within Ensemble that grabs a PDF from a webservice, and then saves it to an external location. All is well, except I am noticing that a lot of .Stream files are being stored on the Cache Server.

Does anyone know what causes these to be stored and what would trigger their removal? Would I be right to assume these would be removed if I ran Ens.Util.Tasks.PurgeMessageBank and the creation dates fell within the purge range?

Edit: I answered my own question on if the purge would clear them out. The answer is yes.

So my real question I guess is: is it possible to do what I'm doing in such a way where I only hold on to the files during the time I'm handling them and then that is it?

I thought about using the temp file stream, but the description sounds like it's not possible to save it elsewhere. So I think I'm looking at either a custom purge task that only clears out the activity in question, or I farm this out to a new namespace where I can then have the purging more restrictive.

Discussion (4)3

Log in or sign up to continue

Stylianos Chalkiotis · May 5, 2019

Hi

One approach to handle files in Ensemble without storing them in the database could be to store the file to an external directory and pass the file path as a string message property instead of the stream between the business hosts.

When you finish what you need to do with the file, then using the file path you can delete the file.

If you follow this approach and depending on what you need to do with your interface, you have to consider that you may not be able to resend messages or view the files because your files may not exist when you resend your messages, unless your interface is retrieving the files every time you resend a message. In this case, the purge tasks will not have to delete the external files, but deleting the external files has to be handled properly by your code.

I hope it helps.

Regards

Stelios

0 0

David Underhill · May 9, 2019

If you want to store the files in the database rather than the filesystem then you can use the Global* streams instead of File*, these also do not create the .stream files in the filesystem.

0 0

score 0 · Answer 1 · 2022-10-24T16:51:24-04:00

This is exactly my question too. I'm creating and sending milloins of xml files and they get stored into the stream -folder in the namespace. The filename is obviously contained in the StreamContainer object, but is there really no clean way to delete those files? In the case of retry within the BusinessOpeation, the file has to be there, but in the case of return message, I should be able to delete it using %File.Delete()? This will slow the process down and I would like that while performing the Management data purge (including the bodies), would also purge those files, yes?

score 0 · Answer 2 · 2023-04-14T05:06:08-04:00

Hi all,

I am facing actually a similar issue, any hint would be highly appreciated: I developed an operation that downloads a document (from a REST Service as well) and stores it to a local folder on the server. If saving locally fails, e.g. due to insufficient file permissions, the .stream-files do remain in the stream-folder for the namespace.

Before using the Save-Method in %FileBinaryStream, the payload is copied from HttpResponse.Data (CopyFrom).

I purged the trigger-messages for the operation as well, but without an effect. Can I simply delete those .stream-files? Maybe they were created temporarily before trying to copy them into the mentioned directory, or as a backup? Is there any way how to check if they are related to any database objects?

Thanks!