Retrieve images using vector search (2)

You need to install the application first. If not installed, please refer to the previous article

Application demonstration

After successfully running the iris image vector search application, some data needs to be stored to support image retrieval as it is not initialized in the library.

Image storage

Firstly, drag and drop the image or click the upload icon, select the image, and click the upload button to upload and vectorize it. This process may be a bit slow.

This process involves using embedded Python to call the CLIP model and vectorize the image into 512 dimensional vector data.

ClassMethod LoadClip(imagePath) As %String [ Language = python ]
{
import torch
import clip
from PIL import Image

device = "cuda" if torch.cuda.is_available() else "cpu"
model, preprocess = clip.load("ViT-B/32", device=device)

image = preprocess(Image.open(imagePath)).unsqueeze(0).to(device)

with torch.no_grad():
    image_features = model.encode_image(image)
return str(image_features[0].tolist())
}

View vector data

Viewing Vector Data in the Management Portal

Vector query

 The image vector query process uses VECTOR_COSINE to retrieve the similarity of vector data, which can be achieved through the following SQL.

select top ? id Image,VECTOR_COSINE(TO_VECTOR(ImageVector,double),(select TO_VECTOR(ImageVector,double) from VectorSearch_DB.ImageDB where id= ? )) Similarity from VectorSearch_DB.ImageDB order by Similarity desc

After uploading multiple images, select query and the image with the highest similarity to the left will be displayed on the right. Click next to view the next page. The similarity between the two images will gradually decrease with each click.

This application is a demo of image vector retrieval. If you are interested, you can check Open Exchange or visit Github to see more information.