· May 21, 2018 10m read

Adding your own provider to MFT

Managed File Transfer (MFT) feature of InterSystems IRIS enables easy inclusion of a third-party file transfer service directly into an InterSystems IRIS production. Currently, DropBox, Box, and Kiteworks cloud disks are available.

In this article, I'd like to describe how to add more cloud storage platforms.

Here's what we're going to talk about:

  • What is MFT
  • Reference: Dropbox
    • Connection
    • Interoperability
    • Direct access
  • Interfaces you need to implement
    • Connection
    • Logic
  • Installation

What is MFT

MFT provides bidirectional file transfer, you can both download files from a cloud storage and upload files there. 

Check out this video on for an introduction to MFT.

Reference: Dropbox

Before we start writing our own MFT adapter, let's run an existing one. I chose Dropbox.

Basic access

First of all, we need to configure Dropbox access:

  1. Register new Dropbox account.
  2. Create new Dropbox App.
  3. Create a new SSL Configuration (if you don't have one)
  4. Create a new MFT configuration at: http://localhost:57772/csp/sys/sec/%25CSP.UI.Portal.MFT.Connection.zen?i...
    • Remove "Use SSL" flag (unless your web server is available over https).
    • Email address field should match your Dropbox email address.
  5. Press Get Access Token button (on MFT connections list page) to authorize your application.
  6. If everything went alright, the status of the new MFT connection should be Authorized.


Next, we need to configure Interoperability production to use the new connection

To receive files from Dropbox into a local directory:

  1. Create EnsLib.MFT.Service.Passthrough service.
    • MFT Connection Name: Dropbox
    • MFT Source Folders: the path to Dropbox folder, from which the files would be downloaded (i.e. /inbox/)
    • Target Config Names: BP or BO to send files to, for example, EnsLib.File.PassthroughOperation
  2. Create EnsLib.File.PassthroughOperation operation.
    • File Path:  the path to a local directory to which the files from Dropbox would be downloaded (i.e. C:\\temp\in\)
    • Charset: Binary
    • File Name: %f

    That's all. After starting the production, files from the specified Dropbox folder would be downloaded into the local directory.

    To upload files to Dropbox, the process should be done in reverse:

    1. Create  EnsLib.MFT.Operation.Passthrough service.
      • MFT Connection Name: Dropbox
      • Default MFT Folder: the path to Dropbox folder, into which the files would be uploaded (i.e. /sent/)
    2. Create EnsLib.File.PassthroughService operation.
      • File Path:  the path to a local directory from which the files would be uploaded to Dropbox (i.e. C:\\temp\out\)
      • Target Config Names: BP or BO to send files to, in our case EnsLib.MFT.Operation.Passthrough
      • Charset: Binary

    And with this, files placed into C:\\temp\out\ folder would be uploaded into /sent/ Dropbox folder.

    For a more comprehensive guide refer to MFT First look.

    "Direct" access

    Interoperability  BS/BO are wrappers over a direct access. When writing your own adapter it's better to use direct access to check if things work as expected. Here's a sample method to get file information

    ClassMethod dropboxInfo(file = "/1.txt")
        // Establishing Dropbox connection
        set mftConnectionName = "Dropbox"
        set mftConnection = ##class(%MFT.API).GetConnection(mftConnectionName, .sc)
        write:$$$ISERR(sc) $System.Status.GetErrorText(sc)
        // Getting information about file
        // Some other methods: GetFileInfo GetFolderInfo CreateFolder DeleteFile GetUser ShareFolder UnshareFolder DownloadStream UploadFile
        set sc = $classmethod($$$EnsCoreMFTAPIClass,"GetFileInfo", mftConnection, file, .itemInfo)
        write:$$$ISERR(sc) $System.Status.GetErrorText(sc)
        // Displaying information about file
        #dim itemInfo As %MFT.ItemInfo
        zw itemInfo


    MFT interface

    MFT provider consists of two parts. To create your own provider you need to implement them.

    Technical API  (connection)

    Connection is responsible for sending requests and parsing results. It should extend %SYS.MFT.Connection.Base and implement the following methods:

    • DefaultURL that returns the root of service API
    • CreateClient that creates OAuth client
    • RevokeToken to log out of the application

    Other methods could be overloaded as required bit these three are a must. Additionally, for Yandex I created these three methods:

    • MakeRequest - to execute normal API requests
    • MakeDownloadRequest - to download files
    • MakeUploadRequest - to upload files

    These methods structure are completely dependent on target API architecture. In my case, a majority of requests were similar, with different requests required for downloading and uploading files so I ended up with three methods.

    Logical API

    Is responsible for consuming the API, it uses the connection to execute requests and it should extend %MFT.API and reside in the %MFT package (we'll talk about working around that requirement later). The methods it must overload could be separated into four categories:

    Info Create/Delete Sharing Load
    • GetFileInfo
    • GetFolderInfo
    • GetFolderContents
    • CreateFolder
    • DeleteFolder
    • DeleteFile
    • ShareFolder
    • UnshareFolder
    • UnshareFolderAll
    • UploadStream
    • DownloadStream

    These methods seem self-explanatory. But please ask in comments if the purpose of these methods seems unclear. There's also documentation in %MFT.API class.  I recommend implementing these methods left to right, you can also skip Sharing methods. That brings us to 8 methods that must be implemented.

    Additionally, if your cloud disk provider has user management (i.e. teams and so on), you can manage that too via:

    • GetUser
    • GetUserById
    • GetUserList
    • CreateUser
    • DeleteUser
    • DeleteUserById

    As Yandex.Disk does not have user management I skipped these methods (they should still be implemented, but left effectively empty).

    Finally GetRequestId method should be implemented to convert paths into ids (that could be faster), otherwise, it should return path.

    Along with some other methods that are about the same for all providers here's a code snippet you can start writing your adapter with:

    /// Get the form of id for a file or folder that is most efficient for subsequent calls.
    /// GetRequestId will return either an id of the form "id:<id>" or a full path depending on which is more efficient.
    /// This method is included to allow the id for future requests to be saved in the most efficient form.
    ClassMethod GetRequestId(connection As %SYS.MFT.Connection.Base, itemInfo As %MFT.ItemInfo) As %String
        Quit itemInfo.GetPath()
    /// Retrieve the %MFT.UserInfo for current user
    ClassMethod GetUser(connection As %SYS.MFT.Connection.Base, username As %String, Output userInfo As %MFT.UserInfo) As %Status
        Set userInfo = ##class(%MFT.UserInfo).%New()
        Set userInfo.Username = connection.Username
        Quit sc
    /// Retrieve the %MFT.UserInfo specified by the service defined Userid.
    /// If the user does not exist, then $$$OK is returned as status and userInfo is returned as "".
    ClassMethod GetUserById(connection As %SYS.MFT.Connection.Base, userid As %String, Output userInfo As %MFT.UserInfo) As %Status
        Quit ..GetUser(connection, userid, .userInfo)
    /// Return the list of all currently defined users for this team or enterprise.
    ClassMethod GetUserList(connection As %SYS.MFT.Connection.Base, Output userList As %MFT.UserList) As %Status
        Set sc = $$$OK
        Set userList = ##class(%MFT.UserList).%New()
        Set sc = ..GetUser(connection, "", .userInfo)
        Quit:$$$ISERR(sc) sc
        Do userList.Users.Insert(userInfo)
        Quit sc
    /// Create a new user.
    /// Unable to do it in Yandex
    ClassMethod CreateUser(connection As %SYS.MFT.Connection.Base, userInfo As %MFT.UserInfo) As %Status
        Quit $$$ERROR($$$MFTBadConnection)
    /// Delete new user.
    /// Unable to do it in Yandex
    ClassMethod DeleteUser(connection As %SYS.MFT.Connection.Base, username As %String) As %Status
        Quit $$$ERROR($$$MFTBadConnection)
    /// Delete the user that is specified by the id.
    ClassMethod DeleteUserById(connection As %SYS.MFT.Connection.Base, userid As %String) As %Status
        Quit $$$ERROR($$$MFTBadConnection)
    /// Unshare a folder from everyone, user is ignored
    ClassMethod UnshareFolder(connection As %SYS.MFT.Connection.Base, path As %String, user As %String) As %Status
        Quit ..UnshareFolderAll(connection, path)
    /// MountFolder is a Dropbox specific method to mount a shared folder that was shared by a different user.
    /// MountFolder is treated as a NOP for all other services.
    ClassMethod MountFolder(connection As %SYS.MFT.Connection.Box, folderName As %String) As %Status
        // A NOP if not Dropbox
        Quit $$$OK
    /// UnmountFolder is a Dropbox specific method to unmount a shared folder that was shared by a different user.
    /// UnmountFolder is treated as a NOP for all other services.
    ClassMethod UnmountFolder(connection As %SYS.MFT.Connection.Box, folderName As %String) As %Status
        // A NOP if not Dropbox
        Quit $$$OK
    /// Update the specified remote file with the contents of the specified local file.
    /// filePath must be a file path.  An id may not be specified.
    /// If replace is true, then an existing file of the same name will be replaced.  
    /// The default is to return an error if a replacement is attempted.
    ClassMethod UploadFile(connection As %SYS.MFT.Connection.Base, localFilePath As %String, filePath As %String, replace As %Boolean = 0, Output itemInfo As %MFT.ItemInfo) As %Status
        Set stream=##class(%FileBinaryStream).%New()
        Set stream.Filename=localFilePath
        Quit ..UploadStream(.connection,stream,filePath,replace,.itemInfo)
    /// Download the specified remote file and store at the location given by localFilePath.
    /// filePath may be a file path.
    ClassMethod DownloadFile(connection As %SYS.MFT.Connection.Base, filePath As %String, localFilePath As %String) As %Status
        Set stream=##class(%FileBinaryStream).%New()
        Set stream.Filename=localFilePath
        Quit ..DownloadStream(.connection,filePath,stream)

    Additional data

    Some other classes are used to transmit information, they are:

    • %MFT.ItemInfo is a detailed description of a file or folder - you'll need it.
    • Other classes are %MFT.UserInfo for user information and  %MFT.FolderContents/%MFT.UserList to list several items or users respectively.

    Note, that these classes, despite being % classes, store the data in the namespace they are called from. They are storing data in ^MFT.* globals.


    As our API should be in a %MFT package we would use mapping to map code from the specialized database into our target namespace (or %SYS). That way InterSystems IRIS can be safely updated and our work wouldn't be overwritten. An Interesting thing about it is that we need to load classes only into target namespace - %SYS in our case.

    First of all, we need to download our code, to do that:

    1. Download and import Installer into any Interoperability-enabled namespace.
    2. Execute: write $System.Status.GetErrorText(##class(MFT.Installer).Install())

    It would:

    • Create MFTLIB database
    • Add mapping of %MFT.Addons and %SYS.MFT.Connection.Addons packages into %SYS namespace from MFTLIB database
    • Download the rest of the code from GitHub, correctly importing % and non % classes

    Next, we need to configure Yandex application:

    1. Register on Yandex.
    2. Create Yandex App
    3. Execute: write $System.Status.GetErrorText(##class(MFT.Yandex).Install(Login, ID, Pass, Host, Port, UseSSL))
      • Login - your Yandex email
      • Host, Port - same as the callback
      • UseSSL - use SSL for callback? Your server needs to support https
    4. Open http://Host:Port/csp/sys/sec/%25CSP.UI.Portal.MFT.ConnectionList.zen
    5. Press Get Access Token and complete authorization.
    6. If everything went fine the Status would be Authorized.
    7. Execute: write $System.Status.GetErrorText(##class(MFT.Yandex).ConfigureProduction(yandexSource, fileDestination, fileSource, yandexDestination))
      • yandexSource and fileDestination - Yandex.Disk folder to download files from, they are stored in a local destination folder.
      • fileSource and yandexDestination - local folder from which files are uploaded to Yandex.Disk.
      • Important: Yandex.Disk folder names should end with / (i.e. out in a disk root would be /out/)
    8. Open production MFT.Production and start it.
    9. Add file(s) to yandexSource and fileSource to see how it works.

    Note, that unlike Dropbox, I'm creating service automatically, like this:

    ClassMethod Install(username As %String, clientId As %String, clientSecret As %String, host As %String = "localhost", port As %Integer = {$get(^%SYS("WebServer","Port"), 57772)}, useSSL As %Boolean = {$$$NO})
        New $Namespace
        Set $Namespace = "%SYS"    
         Do:'##class(Security.SSLConfigs).Exists(..#SSLConfig) ##class(Security.SSLConfigs).Create(..#SSLConfig)
        Set sys = ##class(%SYS.MFT.Connection.Addons.Yandex).%New()
        Set sys.Name = "Yandex"
        Set sys.Service = "Addons.Yandex"
        Set sys.ApplicationName = "Yandex"
        Set sys.SSLConfiguration = ..#SSLConfig
        Set sys.Username = username
        Set sys.URL = ..#URL
        $$$QuitOnError(##class(%SYS.MFT.Connection.Addons.Yandex).CreateClient(sys.Name, ..#SSLConfig, clientId, clientSecret, ,host, port,,useSSL))
        Quit sys.%Save()

    Note the Service value: Addons.Yandex. Service name, when appended to %MFT. should yield the name of the logical API class. Since we can't modify %MFT package directly we can add a subpackage to it.


    MFT is a useful technology and can be easily extended to support cloud providers you need.


    Discussion (5)1
    Log in or sign up to continue

    Minor Update for:

    Managed File Transfer - setup for DropBox

    For DropBox you can use the free 2GB DropBox service.

    Dropbox setup:

    1. Create your DropBox account
    2. Create Your Dropbox App
      1. Click Create App
        1. Status - Development
        2. Permission Type: Full Dropbox
        3. Redirect URIs:  http://localhost:52773/csp/sys/oauth2/
      2. Take note of the App Key and App Secret
    3. Create new SSL Configuration - give it a name (eg  MFTTLSConfig )
    4. Create new MFT Configuration: http://localhost:52773/csp/sys/sec/%25CSP.UI.Portal.MFT.Connection.zen?isNew=1
      1. Connection Name: Dropbox
      2. File Management Service: Dropbox
      3. SSL/TLS - pick the one from step above ( MFTTLSConfig )
      4. Email Address: for the Dropbox account
      5. Base URL:
      6. OAuth 2.0 application name: The Name of the Dropbox App you created
      7. OAuth 2.0 client ID: <App Key> from Dropbox App
      8. OAuth 2.0 client secret: <App Secret> from Dropbox App
      9. OAuth 2.0 redirect URL: If you used the URI link above, then this should match
        1. Use TLS/SSL: UNCHECK
        2. Host Name: localhost
        3. Port: 52773
    5. From the List http://localhost:52773/csp/sys/sec/%25CSP.UI.Portal.MFT.ConnectionList.zen you should now see the new Dropbox Connection Name
      1. Click - Get Access Token
      2. Status should now be "Authorized"

    Interoperability setup:


    Create new Business Operations:

    1. For Documents FROM Dropbox
      1. Operation Class:  EnsLib.File.PassthroughOperation
      2. Operation Name: BO.StoreLocalFile
      3. File Path: <put the local path>
      4. File Name: leave the default - as it ensures it is unique %f_%Q%!+(_a)
    2. For Documents TO Dropbox:
      1. Operation Class: EnsLib.MFT.Operation.Passthrough
      2. Operation Name: BO.StoreOnDropbox
      3. MFT Connection Name: <Connection Name from Above>
      4. Default MFT Folder: /yourfolderhere (note: it will create this folder if it does not exist)

    Create new Business Services:

    1. For Documents FROM Dropbox:
      1. Service Class: EnsLib.MFT.Service.Passthrough
      2. Service Name: BS.GetDropboxFiles
      3. MFT Connection Name: <Connection Name from Above>
      4. MFT SOURCE FOLDERS: /yourfolderhere (note: this folder has to exist or it errors)
      5. Target Config Names: BO.StoreLocalFile
    2. For Documents TO Dropbox
      1. Service Class: EnsLib.File.PassthroughService
      2. Service Name: BS.SendFilesToDropBox
      3. File Path: <put local path here>
      4. File Spec: *
      5. Target Config Names: BO.StoreOnDropbox

    Hey Peter, minor fix: the redirect URL should end in .cls of course.

    Also, I am trying this out and found yours and Eduard's how-tos very clear. However I am seeing permissions issues despite having checked every option in my Dropbox account. The error I am getting is this (on file retrieval)

    ERROR #8921: Error response: ERROR 401: {"error_summary": "missing_scope/..", "error": {".tag": "missing_scope", "required_scope": ""}}.

    I created a Dropbox folder called ToIRIS and have the MFT source folder set to /ToIRIS.

    I think I am missing a step. Any ideas?