The Invisibility Cloak
"The Cloak of Invisibility," he finished, enclosing both line and circle in a triangle." Xenophilius Lovegood
Chromakeying, sometimes known as color keying, is the process of singling out a particular color in an electronic image and then using computer software to make that color transparent. This allows another image, which can be just about anything you can imagine, to show through.
For example, when a meteorologist stands in front of a green screen, television producers use the chromakey special effects technique to isolate the particular shade of green used on the green screen and make that shade of green effectively disappear, allowing another image — such as the animated weather map — to show through instead!
Our invisibility cloak works on the same principle!
- An ordinary cloak of some colour(preferably red)
- Somewhere to run your python scripts (any IDE, Jupyter notebook etc..)
- Import these python modules
import cv2 import time import numpy as np
This is the algorithm we'll follow.
# 1. Capture and store the background frame # 2. Detect the red colored cloth using color detection algorithm # 3. Segment out the red colored cloth by generating a mask # 4. Generate the final augmented output to create the Invisibility cloak
- For making our invisibility cloak, we need to store a frame of the background since we'll be replacing the current frame pixels corresponding to the cloth with the background pixels to generate the effect of an invisibility cloak. So let's do that.
# Creating a VideoCapture object for image acquisition later in the code. cap = cv2.VideoCapture("video.mp4") # We give some time for the camera to warm-up time.sleep(3) background=0 # capture background with for loop to stabilise the parameters and reduce noise for i in range(30): ret,background = cap.read() # ret stores True/False if frame is read correctly # Laterally invert the image / flip the image. background = np.flip(background,axis=1)
- Since we are using a red colour cloth to convert it into an invisibility cloak we will focus on detection of red colour in the frame. However we should transform the colour space of our image from RGB to HSV because HSV is defined in a way that is similar to how humans perceive colour. Due-to shadow, Red channel values of the corresponding pixels are quite low in RGB colour scheme.
# Capturing the live frame ret, img = cap.read() # Laterally invert the image / flip the image img = np.flip(imgaxis=1) # converting from BGR to HSV color space hsv = cv2.cvtColor(img,cv2.COLOR_BGR2HSV) # Range for lower red lower_red = np.array([0,120,70]) upper_red = np.array([10,255,255]) mask1 = cv2.inRange(hsv, lower_red, upper_red) # Range for upper range lower_red = np.array([170,120,70]) upper_red = np.array([180,255,255]) mask2 = cv2.inRange(hsv,lower_red,upper_red) # Generating the final mask to detect red color # Combining masks - basically doing an OR operation pixel-wise mask1 = mask1+mask2
- Let's refine this mask and then use it for segmenting out the cloth from the frame.
mask1 = cv2.morphologyEx(mask, cv2.MORPH_OPEN, np.ones((3,3),np.uint8)) mask1 = cv2.morphologyEx(mask, cv2.MORPH_DILATE, np.ones((3,3),np.uint8)) #creating an inverted mask to segment out the cloth from the frame mask2 = cv2.bitwise_not(mask1) #Segmenting the cloth out of the frame using bitwise and with the inverted mask res1 = cv2.bitwise_and(img,img,mask=mask2)
- Replace the pixel values of the detected red colour region with corresponding pixel values of the static background and finally generate an augmented output which creates the magical effect, converting our cloth into an invisibility cloak.
# creating image showing static background frame pixels only for the masked region res2 = cv2.bitwise_and(background, background, mask = mask1) #Generating the final output final_output = cv2.addWeighted(res1,1,res2,1,0) imshow("magic",final_output) cv2.waitKey(1)
Yay! That was super fun. See you next time! 👋