Canny Edge Detection

Face Detection and Recognition for iOS with OpenCV

I had to write a thesis in my last year of engineering, and the aim was to build a facial detection and recognition system which would do the computation on the device itself instead of on a remote server.

In other terms, quiet opposite to the way works/worked (I don’t know what’s going on with them ever since they were bought by Facebook).

So I started doing some research work on this topic and came across OpenCV. This “Open Computer Vision” was started by Intel some years ago and is now currently backed by big guns like Google, Microsoft, Yahoo etc..

OpenCV, as the name suggests, gives “vision” to your computer. It enables the computer to analyze images and look for features, shapes, colors etc.. It is a C/C++ based library and can be ported to virtually any platform capable of compiling and executing C/C++ code.

OpenCV Featureset

OpenCV Feature set

How do we compile OpenCV for iOS and include it in our Xcode project ?

Well, luckily there are 2 ways to do that. First, as the instructions on the OpenCV site suggest, do some terminal magic (which, btw, I suck at) and compile your own .framework file. And the second, is to search the web for an already compiled .framework file which you can drag and drop into your Xcode project and get going.

Since I was short on time, I took the easy way out. While doing my research, I found a nice article which had a sample Xcode project with an already compiled .framework file and step-by-step instructions on how to import, what all standard frameworks to include, and what all changes to be made to some header files.

This was a sheer blessing !

Once I had my project ready, I went on to build a sample app to check if OpenCV was working correctly. I built a small test app which changed the color scheme, detect edges in an image, detect lines in an image, and also detect faces in a static image. This gave me a level of assurity that things were working as suppose to.


Canny Edge Detection

Detecting edges in an image


Offline Facial Detection

Detecting faces in a static image

Since I was able to detect faces from a static image, It was time to move to recognizing faces. The approach I took was simple. I created an app with two tabs. The first tab was a database creation tab. I clicked 5 pictures each of 3 people to create a “database of faces”. In the second tab, I click a picture of a person and it should match against the database and give the result.


Facial Recognition

Recognizing faces on the iPhone


Now once all the heavy lifting of the project was done, I decided to do some “extra credit” work. I thought that if I could detect faces in a static image, then why not try implementing facial detection in a continuous camera feed (trust me on this one, I did this 2 months before the iPhone 4S came out which featured facial detection in the camera app).

Live Facial Detection

Detecting faces in the live camera feed.

Since, a lot of my code is based on other open source code, I decided to give back to the community. I decided to release my entire thesis as open source. All I want is a link back to my work and citing my name in all other work derived from it. The community has full access to the thesis document, which explains everything I have done, and the Xcode projects which implement everything I have explained in the document. All of this can be found here.

Mohit AthwaniFace Detection and Recognition for iOS with OpenCV

Comments 4

  1. Aman

    thanks for this tutorial.Sir I need your help. Could you please tell me the code for accessing camera on iphone using C++. I’ve tried searching on many sites. Please sir help me in this.

    1. Post
      Mohit Athwani

      Hey Aman,
      I’m sorry I don’t know of a way to invoke the Camera via C++ and as far as I know you could do this only by the methods suggested by the iOS SDK.

      However, since ObjC and C++ code can all be written in the same file, you can mix the two to achieve what you want !

      All the best !

Leave a Reply to Mohit Athwani Cancel reply

Your email address will not be published. Required fields are marked *