2
382

Written by

Yuri Marx
Software Architect at Visum
Follow
Article Yuri Marx · Jan 31, 2022 8m read
Open Exchange

Object Detection using Embedded Python and IRIS

#InterSystems IRIS #Embedded Python #Python

From IRIS 2021.2 is possible write Class Methods using the Python Language. I used this new feature to detect persons and objects into images, using ImageAI (https://github.com/OlafenwaMoses/ImageAI). The ImageAI creator defines it as: "An open-source python library built to empower developers to build applications and systems with self-contained Deep Learning and Computer Vision capabilities using simple and few lines of code." In this article you will learn how to apply AI Computer Vision to detect object and persons inside images.

Steps to analyze images using ImageAI

  1. Go to https://openexchange.intersystems.com/package/AI-Image-Object-Detector and click Download button to go to the Git project.
  2. Clone/git pull the repo into any local directory
$ git clone https://github.com/yurimarx/image-analyzer.git
  1. Open a Docker terminal in this directory and run:
$ docker-compose build
  1. Run the IRIS container:
$ docker-compose up -d
  1. Go to your Postman (or other similar REST client) and config the request like this image:

Request Image Analysis input

  1. Click send and get a response with detected objects like this:

Request Image Analysis output

Behind the scenes - the source code

The Dockerfile

It is an import step. To use python libraries you need to install it before, but you must pay attention to install inside the correct Python IRIS folder (/usr/irissys/mgr/python):

 

Dockerfile

FROM intersystemsdc/iris-community

 

USER root

 

ENV DEBIAN_FRONTEND noninteractive

 

# install libraries required ImageAI/OpenCV to process images and videos
RUN apt-get -y update \
    && apt-get -y install apt-utils \
    && apt-get install -y build-essential unzip pkg-config \
        zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev \
        libssl-dev libreadline-dev libffi-dev wget \
    && apt-get install -y ffmpeg libsm6 libxext6  

 

# use pip3 (the python zpm) to install imageai and the imageai dependencies
RUN pip3 install --upgrade pip setuptools wheel
RUN pip3 install --target /usr/irissys/mgr/python tensorflow==2.4.0
RUN pip3 install --target /usr/irissys/mgr/python keras==2.4.3 numpy==1.19.3 pillow==8.1.1 scipy==1.4.1 h5py==2.10.0 matplotlib==3.3.2 opencv-python keras-resnet==0.2.0
RUN pip3 install --target /usr/irissys/mgr/python imageai --upgrade

 

USER root  
WORKDIR /opt/irisbuild
RUN chown ${ISC_PACKAGE_MGRUSER}:${ISC_PACKAGE_IRISGROUP} /opt/irisbuild
USER ${ISC_PACKAGE_MGRUSER}

 

WORKDIR /opt/irisbuild
COPY  python python
COPY input input
COPY output output
COPY models models
COPY  src src
COPY module.xml module.xml
COPY iris.script iris.script

 

# download the trained model used to detect objects and persons inside images

 

USER root
RUN chmod -R 777 input
RUN chmod -R 777 output
RUN chmod -R 777 models

 

USER ${ISC_PACKAGE_MGRUSER}

 

RUN iris start IRIS \
    && iris session IRIS < iris.script \
    && iris stop IRIS quietly
  • pip3 is the python tool used to install python libraries, like imageai, opencv, tensorflow and other.
  • target parameter was used to install the python libraries where IRIS requires, the folder /usr/irissys/mgr/python.
  • ImageAI uses the following python libraries:
    • TensorFlow: "is an open source library for numerical computation and large-scale machine learning. TensorFlow bundles together a slew of machine learning and deep learning (aka neural networking) models and algorithms and makes them useful by way of a common metaphor. It uses Python to provide a convenient front-end API for building applications with the framework, while executing those applications in high-performance C++" (source: https://www.infoworld.com/article/3278008/what-is-tensorflow-the-machin…)
    • Keras: "Keras is a deep learning API written in Python, running on top of the machine learning platform TensorFlow. It was developed with a focus on enabling fast experimentation. Being able to go from idea to result as fast as possible is key to doing good research." (source: https://keras.io/about/)
    • OpenCV: "(Open Source Computer Vision Library) is an open source computer vision and machine learning software library. OpenCV was built to provide a common infrastructure for computer vision applications and to accelerate the use of machine perception in the commercial products" (source: https://opencv.org/about/).
  • ImageAI is a framework that combines TensorFlow, Keras and OpenCV to allows you train and/or use market deep learning models easily to:
    • Image prediction;
    • Object Detection;
    • Video object detection and traking;
    • Video analysis;
    • Custom model training (imagine train a model to detect risk of patient falling out of bed in hospital ICU, for example).
  • ImageAI works with the 3 main Computer Vision models used in the market for object detection:
    • RetinaNet (Size = 145 mb, high performance and accuracy, with longer detection time)
    • YOLOv3 (Size = 237 mb, moderate performance and accuracy, with a moderate detection time)
    • TinyYOLOv3 (Size = 34 mb, optimized for speed and moderate performance, with fast detection time)
  • The input folder stores the images to be analyzed;
  • The output folder stores the results (images with visual result tags);
  • The models folder stores the trained models to be used with imageai.
  • This sample uses RetinaNet model (restnet50)

Embedded Python implementation

The source code implementation is simple, and a few lines we do object detection (because ImageAI). If tensorflow was used directly with OpenCV and Keras, many more lines of code would have been written. See the commented code:

 

ImageAI implementation

/// Detect objects and persons into images and videos
Class dc.imageanalyzer.Detector
{

 

/// Detect person and objects inside image parameter
ClassMethod GetImageFeatures(Image) [ Language = python ]
{
    # import imageai and json libs
    from imageai.Detection import ObjectDetection
    import json
   
    # instantiate imageai
    detector = ObjectDetection()

 

    # set retinanet as model to detect objects
    model_path = "/opt/irisbuild/models/resnet50_coco_best_v2.1.0.h5"
    # set folder to receive the image to be processed
    input_path = "/opt/irisbuild/input/" + Image
    # set folder to stores the results (visual tags inside image processed)
    output_path = "/opt/irisbuild/output/" + Image

 

    # instantiate retina model
    detector.setModelTypeAsRetinaNet()
   
    # set trained detection model - retina
    detector.setModelPath(model_path)
    # load the model
    detector.loadModel()
    # do the object image detection
    detection = detector.detectObjectsFromImage(input_image=input_path, output_image_path=output_path)
    # return json with the results of the processing
    # if you want you can see the result image inside output folder
    return json.dumps(detection)
}

 

/// Get JSON
ClassMethod GetJSON(pythonObj) [ Language = python ]
{
    import json
    return json.dumps(pythonObj)
}

 

/// Description
ClassMethod GetImageAI(Image) As %Status
{
    Set sc = $$$OK
    Write ..GetImageFeatures(Image)
    Return sc
}

 

}

The IRIS API to expose the Object Detection as an IRIS Object Detection Microservice

It is very easy to call the python classmethod, is it similiar to call any objectscript classmethod, see:

 

IRIS microservice to detect objects inside images

Class dc.imageanalyzer.ImageAnalyzerRESTApp Extends %CSP.REST
{

 

Parameter CHARSET = "utf-8";

 

Parameter CONVERTINPUTSTREAM = 1;

 

Parameter CONTENTTYPE = "application/json";

 

Parameter Version = "1.0.0";

 

Parameter HandleCorsRequest = 1;

 

XData UrlMap [ XMLNamespace = "http://www.intersystems.com/urlmap" ]
{
<Routes>

 

<!-- Detect objects inside an image -->
<Route Url="/analyzeImage" Method="POST" Call="AnalyzeImage" />

 

</Routes>
}

 

// Detect objects inside an image

 

ClassMethod AnalyzeImage() As %Status
{
    Set tSC = $$$OK
   
    try {
        // get the file from the multipart request
        Set source = %request.GetMimeData("file")
       
        // save the file to the input folder, to be processed with imageai
        Set destination=##class(%Stream.FileBinary).%New()
        Set destination.Filename="/opt/irisbuild/input/"_source.FileName
        set tSC=destination.CopyFrom(source) //reader open the file
        set result=destination.%Save()
       
        Set %response.ContentType = ..#CONTENTTYPEJSON
        Set %response.Headers("Access-Control-Allow-Origin")="*"

 

        //call embedded python classmethod to detect objets and write the results as json content
        Write ##class(dc.imageanalyzer.Detector).GetImageAI(source.FileName)

 

        Set tSC=$$$OK
   
    //returns error message to the user
    } catch e {
        Set tSC=e.AsStatus()
        Set pOutput = tSC
    }

 

    Quit tSC
}


 

}

You can do many things with Embedded Python and ImageAI. Know more about ImageAI into: https://github.com/OlafenwaMoses/ImageAI.

This sample app using imageai will be participate in the Python contest, if you liked, vote it. Thanks!

Discussion (0)1
Log in or sign up to continue