In the previous article, we saw different modules in IRIS AI Studio and how it could help explore GenAI capabilities out of IRIS DB seamlessly, even for a non-technical stakeholder. In this article, we will deep dive into "Connectors" module, the one that enables users to seamlessly load data from local or cloud sources (AWS S3, Airtable, Azure Blob) into IRIS DB as vector embeddings, by also configuring embedding settings like model and dimensions.

4 2
2 320
Article
· May 15, 2024 2m read
Retrieve images using vector search (1)

Hi Community,

In this article, I will introduce my application iris-image-vector-search.
The image vector retrieval demo uses IRIS Embedded Python and OpenAI CLIP model to convert images into 512 dimensional vector data. Through the new feature of Vector Search, VECTOR-COSINE is used to calculate similarity and display high similarity images.

3 0
1 209

Hi Community,

In this article, I will introduce my application iris-VectorLab along with step by step guide to performing vector operations.

IRIS-VectorLab is a web application that demonstrates the functionality of Vector Search with the help of embedded python. It leverages the functionality of the Python framework SentenceTransformers for state-of-the-art sentence embeddings.

Application Features

  • Text to Embeddings Translation.
  • VECTOR-typed Data Insertion.
  • View Vector Data
  • Perform Vector Search by using VECTOR_DOT_PRODUCT and VECTOR_COSINE functions.
  • Demonstrate the difference between normal and vector search
  • HuggingFace Text generation with the help of GPT2 LLM (Large Language Model) model and Hugging Face pipeline

2 0
0 440

Hey Community,

We have more exciting news! The new InterSystems online programming contest dedicated to Generative AI, Vector Search and Machine Learning is starting very soon!

🏆 InterSystems Vector Search, GenAI and ML Contest 🏆

Duration: April 22 - May 19, 2024

Prize pool: $14,000

6 9
2 1.3K
Article
· May 4, 2024 3m read
Using VECTORs in ObjectScript

Most examples I've seen so far in OEX or DC left the impression that VECTORs
are just something available with SQL with the 3 Functions especially around VECTOR_Search.
* TO_VECTOR()
* VECTOR_DOT_PRODUCT ()
* VECTOR_COSINE ()

There is a very useful summary hidden in iris-vector-search demo package.
From there you find everything you need over several links and corners.

8 1
1 319
Article
· Apr 26, 2024 3m read
Geo Vector Search #2

Technical surprises using VECTORs
>>> UPDATED

Building my tech. example provided me with a bunch of findings htt I want to share.
The first vectors I touched appeared with text analysis and more than 200 dimensions.
I have to confess that I feel well with Einstein's 4 dimensional world.
7 to 15 dimensions populating the String Theory are somewhat across the border.
But 200 and more is definitely far beyond my mathematical horizon.

3 4
0 287

With the advent of Embedded Python, a myriad of use cases are now possible from within IRIS directly using Python libraries for more complex operations. One such operation is the use of natural language processing tools such as textual similarity comparison.

14 4
4 570

Hi Developers!

Here're the technology bonuses for the InterSystems Vector Search, GenAI, and ML contest 2024 that will give you extra points in the voting:

  • Vector Search usage - 5
  • IntegratedML usage - 3
  • Embedded Python - 3
  • LLM AI or LangChain usage: Chat GPT, Bard, and others - 3
  • Questionnaire - 2
  • Docker container usage - 2
  • ZPM Package deployment - 2
  • Online Demo - 2
  • Implement InterSystems Community Idea - 4
  • Find a bug in Vector Search, or Integrated ML, or Embedded Python - 2
  • First Article on Developer Community - 2
  • Second Article On DC - 1
  • First Time Contribution - 3
  • Video on YouTube - 3
  • Suggest a new idea - 1

See the details below.<--break->

3 0
0 323

Hey Community,

Watch the latest video on InterSystems Developers YouTube:

Using Vector Search for Generative AI

https://www.youtube.com/embed/IapbO3j1Qmw
[This is an embedded link, but you cannot view embedded content directly on the site because you have declined the cookies necessary to access it. To view embedded content, you would need to accept all cookies in your Cookies Settings]

6 0
1 196

As you have seen in the latest community publications, InterSystems IRIS has included since version 2024.1 the possibility of including vector data types in its database and based on this type of data vector searches have been implemented. Well, these new features reminded me of the article I published a while ago that was based on facial recognition using Embedded Python.

7 2
2 452

Using VECTOR_COSINE() in SQL query to perform a text similarity search on existing embeddings in a %VECTOR column.

Code is below.

Commented out sql query returns this error: SQLCODE: -29 Field 'NEW_EMBEDDING_STR' not found in the applicable tables^ SELECT TOP ? maxID , activity , outcome FROMMain .AITest ORDER BY VECTOR_COSINE ( new_embedding_str ,

Sql query as written returns ERROR #5002: ObjectScript error: <PYTHON EXCEPTION> *<class 'OSError'>: isc_stdout_write: PyArg_ParseTuple failed!

0 10
0 251

Hi Community!

As an AI language model, ChatGPT is capable of performing a variety of tasks like language translation, writing songs, answering research questions, and even generating computer code. With its impressive abilities, ChatGPT has quickly become a popular tool for various applications, from chatbots to content creation.
But despite its advanced capabilities, ChatGPT is not able to access your personal data. So we need to build a custom ChatGPT AI by using LangChain Framework:

Below are the steps to build a custom ChatGPT:

  • Step 1: Load the document

  • Step 2: Splitting the document into chunks

  • Step 3: Use Embedding against Chunks Data and convert to vectors

  • Step 4: Save data to the Vector database

  • Step 5: Take data (question) from the user and get the embedding

  • Step 6: Connect to VectorDB and do a semantic search

  • Step 7: Retrieve relevant responses based on user queries and send them to LLM(ChatGPT)

  • Step 8: Get an answer from LLM and send it back to the user

For more details, please Read this article

3 6
0 372

InterSystems announces its fourth preview, as part of the developer preview program for the 2024.1 release. This release will include InterSystems IRIS®, InterSystems IRIS® for HealthTM, and HealthShare® Health Connect.

1 2
0 294

InterSystems announces its second preview, as part of the developer preview program for the 2024.1 release. This release will include InterSystems IRIS®, InterSystems IRIS® for HealthTM, and HealthShare® Health Connect.

5 9
0 503

InterSystems announces its third preview, as part of the developer preview program for the 2024.1 release. This release will include InterSystems IRIS®, InterSystems IRIS® for HealthTM, and HealthShare® Health Connect.

1 0
0 194
Article
· Sep 18, 2023 7m read
Vectors support, well almost

Nowadays so much noise around LLM, AI, and so on. Vector databases are kind of a part of it, and already many different realizations for the support in the world outside of IRIS.

Why Vector?

  • Similarity Search: Vectors allow for efficient similarity search, such as finding the most similar items or documents in a dataset. Traditional relational databases are designed for exact match searches, which are not suitable for tasks like image or text similarity search.
  • Flexibility: Vector representations are versatile and can be derived from various data types, such as text (via embeddings like Word2Vec, BERT), images (via deep learning models), and more.
  • Cross-Modal Searches: Vectors enable searching across different data modalities. For instance, given a vector representation of an image, one can search for similar images or related texts in a multimodal database.

And many other reasons.

So, for this pyhon contest, I decided to try to implement this support. And unfortunately I did not manage to finish it in time, below I'll explain why.

10 7
3 1.1K

InterSystems announces its first preview, as part of the developer preview program for the 2024.1 release. This release will include InterSystems IRIS®, InterSystems IRIS® for HealthTM, and HealthShare® Health Connect.

2 3
0 313

It seems like yesterday when we did a small project in Java to test the performance of IRIS, PostgreSQL and MySQL (you can review the article we wrote back in June at the end of this article). If you remember, IRIS was superior to PostgreSQL and clearly superior to MySQL in insertions, with no big difference in queries.

8 6
3 856

1. IRIS RAG Demo

IRIS RAG Demo

This demo showcases the powerful synergy between IRIS Vector Search and RAG (Retrieval Augmented Generation), providing a cutting-edge approach to interacting with documents through a conversational interface. Utilizing InterSystems IRIS's newly introduced Vector Search capabilities, this application sets a new standard for retrieving and generating information based on a knowledge base.
The backend, crafted in Python and leveraging the prowess of IRIS and IoP, the LLM model is orca-mini and served by the ollama server.
The frontend is an chatbot written with Streamlit.

17 3
2 998

As an AI language model, ChatGPT is capable of performing a variety of tasks like language translation, writing songs, answering research questions, and even generating computer code. With its impressive abilities, ChatGPT has quickly become a popular tool for various applications, from chatbots to content creation.
But despite its advanced capabilities, ChatGPT is not able to access your personal data. So in this article, I will demonstrate below steps to build custom ChatGPT AI by using LangChain Framework:

4 0
1 12.8K
Article
· Feb 13, 2023 4m read
When to use Columnar Storage

With InterSystems IRIS 2022.2, we introduced Columnar Storage as a new option for persisting your IRIS SQL tables that can boost your analytical queries by an order of magnitude. The capability is marked as experimental in 2022.2 and 2022.3, but will "graduate" to a fully supported production capability in the upcoming 2023.1 release.

The product documentation and this introductory video, already describe the differences between row storage, still the default on IRIS and used throughout our customer base, and columnar table storage and provide high-level guidance on choosing the appropriate storage layout for your use case. In this article, we'll elaborate on this subject and share some recommendations based on industry-practice modelling principles, internal testing, and feedback from Early Access Program participants.

14 2
2 644
Article
· Jan 10, 2023 4m read
Columnar Storage in 2022.3

As you may well remember from Global Summit 2022 or the 2022.2 launch webinar, we're releasing an exciting new capability for including in your analytics solutions on InterSystems IRIS. Columnar Storage introduces an alternative way of storing your SQL table data that offers an order-of-magnitude speedup for analytical queries. First released as an experimental feature in 2022.2, the latest 2022.3 Developer Preview includes a bunch of updates we thought were worth a quick post here.

9 2
3 735

Fixing the terminology

A robot is not expected to be either huge or humanoid, or even material (in disagreement with Wikipedia, although the latter softens the initial definition in one paragraph and admits virtual form of a robot). A robot is an automate, from an algorithmic viewpoint, an automate for autonomous (algorithmic) execution of concrete tasks. A light detector that triggers street lights at night is a robot. An email software separating e-mails into “external” and “internal” is also a robot. Artificial intelligence (in an applied and narrow sense, Wikipedia interpreting it differently again) is algorithms for extracting dependencies from data. It will not execute any tasks on its own, for that one would need to implement it as concrete analytic processes (input data, plus models, plus output data, plus process control). The analytic process acting as an “artificial intelligence carrier” can be launched by a human or by a robot. It can be stopped by either of the two as well. And managed by any of them too.

6 0
0 364

This is the third post of a series explaining how to create an end-to-end Machine Learning system.

Training a Machine Learning Model

When you work with machine learning is common to hear this work: training. Do you what training mean in a ML Pipeline?
Training could mean all the development process of a machine learning model OR the specific point in all development process
that uses training data and results in a machine learning model.

4 10
2 357