AI based Face swapping is one of the most popular concepts. There are a number of Apps and techniques by which we can swap the face like Draw Something, Dubsmash, Flappy Bird, FatBooth and (in the earliest days of Apple’s App Store) virtual pint-drinking and lightsaber. We can say that it has become a mobile craze since 2019.

These applications normally work like:

  1. It lets us switch faces, for example: we can select any funny pictures and swap with someone else in real time.
  2. We can record video or even with live stream a face swapping can be done, unlike static pictures.

Techniques for Face Swapping

There are a number of face swapping techniques available. Basic Face swapping techniques are mentioned below:

1. Landmark Detection

It is defined as the process which identifies the point of interest in an image of the human face. In opencv dlib is used for facial landmark detection, it extracts some unique fashion landmarks for the whole face. In this method, multiple iterations happened. After the first iteration of the algorithm, an initial value “0” is given and this value increases linearly such that at the end of the iteration it gets the value 10. After completing the iterations an  image evolved with the  ‘ground truth’, it means iteration can stop now. After identifying landmarks, another image can be blended over it.

2. Convex HUll

 The convex hull is a set of points defined as the smallest convex polygon, which encloses all of the points in the set. This means that for a given set of points, the convex hull is the subset of these points such that all the given points are inside the subset. To find the face border in an image, we need to change the structure a bit. The structure is first passed to the convex hull function with return points to false, this means that we get an output of indexes. Brzęczkowski then exhibited the face border in the image in blue color using the find_convex_hull.py function.

3. Triangulation

This technique is based on the inputs obtained after triangulation [1].  Hence can we make some assumption about the 2D image to approximate 3D information of the face. One simple way is to triangulate using the facial landmarks as corners and then make the assumption that in each triangle the content is planar (forms a plane in 3D) and hence the warping between the triangles in two images is affine. Triangulating or forming a triangular mesh over the 2D image is simple but we want to triangulate such that it’s fast and has an “efficient” triangulation. One such method is obtained by drawing the dual of the Voronoi diagram, i.e., connecting each two neighboring sites in the Voronoi diagram. This is called the Delaunay Triangulation and can be constructed in O(nlogn) time. We want the triangulation to be consistent with the image boundary such that texture regions won’t fade into the background while warping. Delaunay Triangulation tries the maximize the smallest angle in each triangle

There are a number of Issues in these techniques like Angle of Face is not detectable and closing is difficult. However we have used Position Map Regression Network [2] for face swapping.

The main features are:

Tools Used:

Applications

Results

  1. Following is the image used as the input  and output

NOTE: The input images are obtained from myntra.com

References:

[1] https://cmsc733.github.io/2019/proj/p2/

[2] https://github.com/YadiraF/PRNet

Leave a Reply

Your email address will not be published. Required fields are marked *