Facial recognition with Embedded Python and IRIS

Welcome, dear members of the community!

In this article we are going to demonstrate the great potential that IRIS/HealthConnect makes available to all its users with the use of Embedded Python and we are going to do it by developing a small production that will allow us to recognize and identify the faces present in a JPG file from some images that we will use as a reference.

Project configuration:

Let us begin! We have published the entire application in OpenExchange so that you only need to download and deploy it in Docker as you can read in the associated README file. As you will see, its name is Nomenclator, referring to the slaves who in Ancient Rome, during festivals and social events, stayed with their masters and were in charge of remembering the names of all those they met.

I am not an expert in Python so I have based the development on this article (in spanish) on the web, it is not necessary that we reinvent the wheel.

Let's take a look at the project sources, starting with the docker-compose.yml file:

version: "2.2"
services:
  # iris
  iris:
    init: true
    container_name: iris
    build:
      context: .
      dockerfile: iris/Dockerfile
    ports:
      - 52773:52773
      - 51773:51773
    command: --check-caps false
    volumes:
    - ./shared:/shared

As you can see it is extremely simple, we only need an instance of IRIS running. Let us explain above the most relevant files that are part of the project:

• Dockerfile: file in charge of project configuration, it will install the necessary Python libraries in the container and deploy the code and production.
• mobilenet_graph.pb: file with the pre-built model for the recognition of existing faces in JPG files.
• facenet_keras_weights.h5: pre-built model for identifying faces from images stored in our repository
• \shared: folders defined as volumes that will be accessible from Visual Studio Code and from the Docker container.
  • \JPG: folder where we will paste the image that we want to identify.
  • \knowns: folder that will function as a repository of our identified faces, our image will be compared with the ones we have in this folder, if there is a match we will show a message indicating the file with which it matches. For our example we have included a few photos of actor John Travolta.
  • \results: folder to store the result of the validations, an image will be included with the faces found within a box, this box will be green if the face matches one of those present in the \shared\knowns folder and red if it does not match.
  • \test: folder with 2 images that we will use for our tests, one of John Travolta (travi.jpg) and another of Nicolas Cage (nicolas.jpg).
  • \unknowns: folder to which we will move the image received in \shared\JPG for processing from the Python method.
• Installer.cls: ObjectScript class that will create the NOMENCLATOR namespace and the associated database in our IRIS instance. 
• Nomenclator.xml: xml file containing the production and business service code.
• requeriments.txt: file with the Python libraries needed to run the project.

Previous warning:

This project is for educational purposes only and is not intended for direct use in any production environment.

Python code:

Let's briefly explain how our Python code works. Let's start with the code that is executed when we call the python method:

isMatch = ''
	# Embeddings reference
	known_embeddings = []
	for name in os.listdir(DIR_KNOWNS):
		if name.endswith('.jpg'):
			image = load_image(DIR_KNOWNS,name)
			bboxes = detect_faces(image)					
			face = extract_faces(image,bboxes)
			if len(face) > 0 :
				known_embeddings.append(compute_embedding(facenet,face[0]))				
					
	# Searching matches for knowns faces		
	for name in os.listdir(DIR_UNKNOWNS):
		if name.endswith('.jpg'):
			image = load_image(DIR_UNKNOWNS,name)
			bboxes = detect_faces(image)
			faces = extract_faces(image,bboxes)
      
			# Computing embedding for each face
			img_with_boxes = image.copy()
			for face, box in zip(faces,bboxes):
				emb = compute_embedding(facenet,face)

				puntuacion, reconocimiento = compare_faces(known_embeddings,emb)

				if any(reconocimiento):							
					isMatch = isMatch + ' ' + name + ' match!'
					img_with_boxes = draw_box(img_with_boxes,box,(0,255,0))
				else:
					img_with_boxes = draw_box(img_with_boxes,box,(255,0,0))
            
			cv2.imwrite(f'{DIR_RESULTS}/{name}',cv2.cvtColor(img_with_boxes,cv2.COLOR_RGB2BGR))
	
	return isMatch

Alright, let's get started, we have our isMatch variable into which we will load our method's response to the entered image. Next we have a for loop over all the files we have in the known images directory (what we have called our repository) in which we will first detect all the faces for the known images (detect_faces) using the pre-built model present in the mobilenet_graph.pb file, we will extract them from the file in the format we need (extract_faces) and we will extract their embeddings (compute_embedding) using the model defined by keras present in the facenet_keras_weights.h5 file, the embedding will not be more than a vector of eigenvalues of every face found in the known image files.

Once we have obtained the embeddings for all the known faces that we have, we will repeat the process with the unknown images that we want to validate, we will first detect the faces present in the images, we will extract them and finally we will calculate their embedding individually. With the calculated embedding we can compare it with the embeddings we obtained with the familiar faces.

Let's take a look at the Python function that performs the comparison:

def compare_faces(embs_ref, emb_desc, umbral=1.1):
		distancias = []
		for emb_ref in embs_ref:
			distancias.append(np.linalg.norm(emb_ref-emb_desc))
		distancias = np.array(distancias)
		return distancias, list(distancias<=umbral)

Here we see how we go through all the known embeddings that we have and compare it with the one belonging to the unknown image that we have, if the "distance" of both embeddings is less than 1.1 we will consider that they are the same person, otherwise we will not have a coincidence. The 1.1 value is something that you can fine-tune on your own by raising or lowering it to suit your needs.

Once the comparison process is finished, we will create a file that we will leave in the \shared\results folder with the original image and a green frame on the face if there is a match or red if there is not.

Demonstration:

Let's see an example of how it would all work together.

First let's copy the images we have in our \shared\test folder to \shared\JPG

Let's see what our production has done in IRIS:

Perfect! We have captured the two images from our /shared/JPG directory, we have processed them and we have obtained a match for the travi.jpg file and a divergence for the nicolas.jpg file. Let's look at the files in our /shared/results folder:

Here's good old Nicolas sad because he didn't get a match with our known faces. Let's see the other image.

Brilliant! John is more than happy to have been recognized for our IRIS production.

As I said before this is a simple example of the potential of IRIS with Embedded Python, you could improve this example by storing the embeddings of each known image in the IRIS database and we would save the step of calculating it every time we have a new image to test, since we would only have to load them from the database.

Well this would be all. I hope you find it useful and if you have any questions or suggestions do not hesitate to write a comment.

PS: a choco-point for those who have discovered the reference to the film Face off.