Great stuff! This was really easy to get going with once I got the right OpenAI API key.

I was curious how it'd do on a different (more complex) schema and it seemed to hold up quite well. Two things I noted so far (but I'm not done with poking around) is that it seems to add LIMIT n to queries every now and then, which is not valid IRIS SQL syntax (and likely you can suggest that in the prompt), and that it does weird things when you try to put your own SQL in the execute box (it restores the previous query - just a UI thing I guess).

You guys really earned that first place in the contest!!

Please note that documentation link is for InterSystems Caché. In IRIS, we've gotten more strict around the deprecation of those older classes and that section of the documentation has been removed, and the associated classes are marked as deprecated and/or internal to further encourage you to take advantage of %SQL.Statement

Equally curious in the feedback here, as we're currently working on a long-awaited project to collect table statistics automatically in the background.

See also this earlier article, with some discussion at the bottom.

As for #4: if you'd need to clear your cached queries in order for new stats to be picked up, that's a bug. Please let us know if that is what you'd find ;-)

another question to ask yourself is whether you really want that CSV file data to be in IRIS tables. Maybe if it's rarely accessed projecting it as a foreign table is more cost effective?

Nice article! 

as a complement to the section on Tune Table, I'd like to refer to this article I wrote about a bit of automation we put in the product in 2021.2 (that we intend to enhance this year), and also this one on caveats wrt packaging statistics

something odd is going on with that new_embedding_str parameter value you're adding. Rather than taking the value, it's taking the parameter name, hence the "field not found" error. Try removing that * before parameters in your call to exec(), or just inline the parameter value into the statement as you're not reusing it anyway.

Hi @Vadim Aniskin ,

while putting together a sample of the new feature, I found out that very unfortunately this change did not make it into 2024.1.0 after all. It passed our internal testing a few months ago and was promoted through project and integration branches using our existing automation, and as such was added to the draft list of features we wanted to describe in the release notes. However, because of overlapping changes it did not get promoted into the main release branch automatically (as those earlier steps), and ended up in a manual queue. That took a little longer than expected, and I did not perform a final check before publishing the draft release notes. To my knowledge, this is the first time we had a fully-greenlit feature miss out on the automation, but that's no excuse and we've learned to do a manual check right before release. 

In short, this change is now on its way to 2024.2, and the 2024.1 release notes will be updated shortly.

I'm sorry about the confusion this may have caused,
benjamin

Please open an iService ticket to request assistance, referencing your AWS account details.

FYI - the new Vector Search capability, and some other smaller items advertised in the release notes, are not yet part of this first developer preview. We're working hard to make them part of the next drop!

Hi @Iryna Mykhailova , I'm sorry your students had a bad experience setting up a deployment in the preview environment. We have indeed found some glitches along the way, and have prioritized fixing them in the main code branch that's on its way to GA, rather than patching the Developer Access Program environment.

Great to hear though you're thinking of promoting this to your students. When we are GA, it would be great to see them test this out, not least the Cloud IntegratedML piece that's quite unique to IRIS.

as @Ben Spead pointed out, we are currently having issues with eu-central-1. In fact, the Developer Access Program should only have shown the us-east-1 region but at some point in syncing with the portal for our GA cloud services that option slipped back in. This said, the DAP environment is a preview system and we're getting close to releasing a GA version of InterSystems IRIS Cloud SQL and Cloud IntegratedML, based on feedback and experiences from that preview, including those at the hackathon.

Yay, nice work Tim, and exciting collaborations that made this possible!

We're also trying to phrase the individual items in a more "actionable" way so they're easier to check off, rather than just describe what changed. On that aspect as well, are your feedback is much appreciated!

indeed, indices for vectors are not straightforward at all. Even though our %FunctionalIndex structure allows you to hook into filing and build pretty much any data structure you want, leveraging it in SQL is hard because the corresponding %SQL.AbstractFind is for filtering (in a WHERE clause) and not a good fit for the combination of filtering and ranking that is the common expectation when doing a vector search.

Both the indexing techniques and a proper fit in a relational model are the subject of ongoing academic research. Other vendors such as SingleStore have focused on ensuring the dot product (or other distance function) can be executed very efficiently so they just need to throw a lot of compute at it to make up for the lack of an index.

Nice to see what you were able to pull together here 👍

FYI - We have an internal research project on making vectors a first-class datatype, leveraging the same internals that columnar storage uses, which will be a better fit than $list (which offers flexibility we don't need here). Hopefully we'll be able to share more details on that later this year!