New post
Question
· Jan 25, 2023

Checking that file did arrived on remote SFTP

#Ensemble

Hi guys, 
I've setup an FTP passthrough service and operation to move two files from one SFTP server to another. 
The files are moved and i can confirm with filezila that the whole operation went well. 
However, i need to raise errors and send email if one the sftp servers is down for whatever reason, and if the trasnfer didn't go well. 
I ticked the alert on error box for the passthrough BS and BO and the errors are indeed forwraded to an alert management system. I have real problems with the check on the SFTP servers. 
I'm stuck after trying to design several solutions that i didn't manage to implement. 
I'm currently trying to use a BP that does those checks for me but can't figure out how to set it up. 
I gathered some code to check for a file on a disk such as 
Set fileName="a_test_filename.txt" /* Check for existence of a file- Return Value: 0 - filename does not exist; 1 - filename does exist */
Set fileExists=##class(%File).Exists(fileName)
If ('fileExists) // do the processing for when a filename does not exist

I jsut can't figure out how to set it up. I'm not very good at Ensemble yet. Any pointers to help me sort this out ? 
Please and thank you

Product version: Ensemble 2018.1
Discussion (7)2
Log in or sign up to continue
Cedric Montanuy · Jan 25, 2023

Hi Stefan, 
The whole thing looks like this:
Files are dropped on server A:/origin_folder by some piece of software. The passthrough service is listening to this server A location and get triggered whenever the files are dropped. The files are then move to server B:/reception_folder with the passthrough operation. 
I then want Ensemble to check on server B:/reception_folder that the files are indeed present. 
If the transfer didn't go well, i need an email to be sent to some recipients. 
The BP in my design is here to call the passthrough BO, and handle the potentials problems. 
Does that make sense ?

0 0
Stefan Cronje · Jan 25, 2023

So this code runs on Server A?

Server A handles the passthrough and then needs to check that the file landed on Server B, correct?

If that is the case, the solution is not going to be this simple. You will have to have another operation called by the BP to check if the file is on Server B. In short

  • A custom operation that uses the FTP Outbound Adaptor to FTP to Server B and pull a list of files.
  • The operation will need the filename and should be provided by the Ensemble message from the BP.
  • The operation should then check if the filename received in the Ensemble message is in the File List on Server B. You might be able to filter for the filename directly with FTP, I will have to confirm that.
  • The operation should then return a message to the BP that contains a "result" of the check. The BP can then act on that by creating an Alert Message and sending it to Ens.Alert.

I hope this helps. Let me know if I am misunderstanding the requirement.

0 0
Cedric Montanuy · Jan 25, 2023

That's about it (not exactly, but close enough, the check on server B is done from another server where Ensemble is running, so same difference i think) 
I'll give it a shot.
Thank you, Stefan ! 
PS:
Not sure abou the long connection thingy you mentionned in the comment below, i don't think it's relevant, but i'll keep it in mind if i run into troubles. It is currently set to -1 (never disconnected) so this might gets me in trouble eventually.
Thanks for the heads up. 
 

0 0
Stefan Cronje · Jan 25, 2023

Wtih -1 the connection probably "dies" on the other server.

The Passthrough Operation does not know it died, so FTP is going to fail, immediately. Then it will retry if E=R is set on the Action and within the timeout period, it will reconnect and then transfer the file, but soon the next file will come and same will happen if the connection died.

I recommend changing that setting to 0 first and monitoring what happens.

UPDATED:
To ensure delivery

  • Stay Connected: 0
  • Reply Code Actions: E=R
  • Failure Timeout: -1
0 0