Moving API data into external SQL instance without audits/persistence
Recently our team have been getting requests to pickup a large amount of data from API nightly (e.g. using ODATA to loop through pages) and placing that into MS SQL databases.
My question is, what is considered the best practice to get data from API and route to external SQL database WITHOUT persisting any messages/traces etc?
At the moment my thinking would be to write a service that triggers an action once a day to an EnsLib.HTTP.OutboundAdapter, then map the HTTP response into a linked table (via method to keep the operation clean that managed the gateway on the way in and out of the method).
What bothers me about this its using EnsLib.HTTP.OutboundAdapter to do something else its not originally intended for. And the gateway management of link tables is a bit of pain (especially when there is patching that affects SQL failover cluster because IRIS seems to lose sight of the linked connection until a process/operation is restarted). You also get none of the adapter benefits of EnsLib.SQL.OutboundAdapter.