Join the Early Access Program for Foreign Tables

Last week at the InterSystems Global Summit, we announced our new Foreign Tables capability, which was introduced as an experimental feature with the 2023.1 release earlier this year. We're now inviting you to join the Early Access Program for Foreign Tables and kick the tires on this new capability, so you can let us know whether it suits your needs and what capabilities we should prioritize next.

So what are Foreign Tables, really? -- In case you don't have the time or popcorn to watch this nice overview video, Foreign Tables help you project data to IRIS SQL that is physically stored elsewhere, be it in files or remote databases. A Foreign Table presents itself to SQL as a regular IRIS table that can be used in any valid SQL statement, including as part of a JOIN with other regular or foreign tables. At query time, we'll figure out what you need from the foreign server and retrieve it for you, and in case we know that server is a relational database, we'll try to push down eligible query predicates to that server to minimize the data that needs to be pulled across the network.

Why wouldn't I just load the data? -- That's an excellent question! There are use cases in which data is managed externally, and when you query it in IRIS some time after loading it, it might already have changed in the source system. Rather than set up costly synchronization mechanisms, projecting it as a foreign table ensures you always go get the current data at query time. Other use cases include big file-based data that would otherwise just sit there and consume IRIS database storage, or generally data you'll only read once anyway. And by the way, foreign tables' syntax and capabilities are fully aligned with what you know from the LOAD DATA command we introduced in 2021.2.

Didn't we already have Linked Tables for this? --  True. That capability has been around for quite a while and saw good adoption across our customer base. However, implementations for the JDBC and ODBC variants came about at different times and have slightly differing features and limitations, which has made maintenance and enhancements complicated. Since their original implementation, several internal improvements to the IRIS SQL internals (including Table Valued Functions) enabled a more practical approach with a clearer separation of the foreign side and the IRIS side. This is what led us to implementing Foreign Tables as a new capability, and make sure it aligns with the official ANSI SQL standard. 

We intend to make Foreign Tables the successor for Linked Tables, and if you're currently using them, we'd love to know which of its features are important to your use case so we can properly prioritize the next implementation round for Foreign Tables and we can quickly onboard more users and get feedback from experiences outside the lab.

Where do I sign up? -- Check out the form at intersystems.com/early-access-programs and we'll get you started with a tutorial and demo repository. We'll reach out periodically to hear what you think and update you on new capabilities we're adding.

Looking forward to hear from you!