Android OpenCV :Face Localization using Haar Cascade Classifier

Pi19404
April 1, 2013

Contents

Contents
Android OpenCV :Face Localization using Haar Cascade Classier
0.1 0.2 0.3 0.4 Introduction . . . . . . . . . . . . . . . . . . . Processing Raw Camera Data . . . . . . . . Haar Cascade Classifiers . . . . . . . . . . . Haar Cascade Classifier : Implementation 0.4.1 Native Code . . . . . . . . . . . . . . . 0.4.2 Java Code . . . . . . . . . . . . . . . . 0.4.3 Project Setup . . . . . . . . . . . . . . 0.4.4 Compilation Setup . . . . . . . . . . . 0.4.5 Compilation . . . . . . . . . . . . . . . . 0.5 Launch Application . . . . . . . . . . . . . . . 0.6 Code . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3
3 3 3 4 4 6 7 8 9 10 10 10

2 | 11

Android OpenCV :Face Localization using Haar Cascade Classifier

Android OpenCV :Face Localization using Haar Cascade Classifier

0.1 Introduction
In this article will look at simple application of Face Location on android platform using OpenCV libraries using Haar Cascade Classifiers.

0.2 Processing Raw Camera Data

The android application provides the raw camera image in YUV format as byte image in the registered preview Callback function.Refer to earlier artile about details of decoding raw camera data into format suitable for processing by OpenCV libraries.

0.3 Haar Cascade Classifiers

The Haar Cascade Classifiers was proposed by Viola and Jones, 2001 . It describes a machine learning approach for visual object detection which is capable of processing images extremely rapidly and achieving high detection rates.The features used are similar to harr wavelet functions are are called haar-like features.For a image path of size 24x24 total number of features extracted are about 180,000.Input to the Machine learning algorithms is a large set of positive and negative training images which contain face and non face images. The machine learning algorithm used in the present implementation is Adaboost which is used to learn a classification function. Since the object detection in a generic environment is a rate event detection problem Cascade of classifier is used to increases detection performances and to achieve reduced computation time.

3 | 11

Android OpenCV :Face Localization using Haar Cascade Classifier During the detection stage the feature extraction is performed at multiple scales which enables in detection of object at different scales. From the implementation perspecitive OpenCV provides routines for haar cascade classifer and training. The output of the training stage is a xml files which contains the parameters of learning function. The File for the frontal face profile is available with OpenCV package and this will be used directly instead of training a new classifier.

0.4 Haar Cascade Classifier : Implementation Details

0.4.1 Native Code
The first step of compilation using the opencv,javacv method is to write the header files. The below code provides a high level interface to opencv haar casacade detection class and also draws ROI on the image before returning. To achieve real time performance on the mobile phone the image is reduced to 160x120. The haar detection is performed on this downscaled image and the location of face region are scaled by suitable ammount to plot the ROI properly. If errors occur in the opencv native routine it will not be caught by the java interface.Hence a retun code is passed to indicate status of the opencv native routines.
1 2 3 4 5 6 7 8 9 10 11 12 13

class haarcascade { private : String face_cascade_name ; String eyes_cascade_name ; CascadeClassifier face_cascade ; // constructor for class which loads the cascade files public : haarcascade () { }

4 | 11

Android OpenCV :Face Localization using Haar Cascade Classifier

14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49

haarcascade ( char * name ) { face_cascade_name = name ; // load the cascade file if ( ! face_cascade . load ( face_cascade_name ) ) { printf ( " - -(!) Error loading \ n " ) ; }; } // method to detect faces in the image fram int detect ( Mat & frame ) { int flag =0; Mat f1 (240/2 ,320/2 , frame . type () ) ; cv :: flip ( frame , frame ,1) ; cv :: resize ( frame , f1 , f1 . size () ,0 ,0 , INTER_CUBIC ) ; // ROI of face locations std :: vector < Rect > faces ; Mat frame_gray ; // converting to grayscale cvtColor ( f1 , frame_gray , CV_BGR2GRAY ) ; // pre processing frame using histogram equalization equalizeHist ( frame_gray , frame_gray ) ; // multiscale detection of faces face_cascade . detectMultiScale ( frame_gray , faces , 1.1 , 2 , 0| CV_HAAR_SCALE_IMAGE , Size (30 , 30) ) ; for ( int i = 0; i < faces . size () ; i ++ ) { Point center ( faces [ i ]. x + faces [ i ]. width *0.5 , faces [ i ]. y + faces [ i ]. height *0.5 ) ; int scalex = frame . cols / f1 . cols ; int scaley = frame . rows / f1 . rows ; Point ncenter ; ncenter . x = center . x * scalex ; ncenter . y = center . y * scaley ; ellipse ( frame , ncenter , Size ( faces [ i ]. width *0.5* scalex , faces [ i ]. height *0.5* scaley ) , 0 , 0 , 360 , Scalar ( 255 , 0 , 255 ) , -1) ; } flag =1; return flag ; } }; // class which provides interface to java code class OpenCVProcess { private : haarcascade face ; public : OpenCVProcess () {} int run ( int width , int height , signed char * _yuv , int * _bgra ) {

50 51 52 53 54 55 56 57 58 59 60 61 62 63

5 | 11

Android OpenCV :Face Localization using Haar Cascade Classifier

64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79

};

} // method to initialize haar cascad detector void initDetector ( const char * name ) { face = haarcascade (( char *) name ) ; }

// decoding the camera data to BGRA Mat image Mat myuv ( height + height /2 , width , CV_8UC1 , ( unsigned char *) _yuv ) ; Mat mbgra ( height , width , CV_8UC4 , ( unsigned char *) _bgra ) ; cvtColor ( myuv , mbgra , CV_YUV420sp2BGR , 4) ; Mat image ; cvtColor ( mbgra , image , CV_BGRA2BGR ) ; int flag = face . detect ( image ) ; cvtColor ( image , mbgra , CV_BGR2BGRA ) ; return flag ;

0.4.2 Java Code

The next step is to write the Java Files to native interface with the header files
 file AndOCVFaceDetectionEx.java contains main android routine which launches the application.  file ProcessImage.java contains methods to interface with the native code.  file Preview.java contains methods to initialize the camera,set the camera parameter,get the raw camera  file DrawOnTop.java contains methods to draw the processed image on top of camera preview.
1 2 3 4 5 6 7

@Namespace ( " OpenCV " ) public class ProcessImage { // specifying the path where the header and library found public static final String androidIncludepath = /"; public static final String androidLinkpath = armeabi " ; public static final String genericIncludepath = / include / " ; public static final String genericLinkpath = / lib / "

is to be " ../ include " ../ libs / " / opt / local " / usr / local

6 | 11

Android OpenCV :Face Localization using Haar Cascade Classifier

8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37

public static final CV_MAJOR_VERSION CV_MINOR_VERSION CV_SUBMINOR_VERSION

int = 2, = 4, = 2;

public static final String CV_VERSION = CV_MAJOR_VERSION + " . " + CV_MINOR_VERSION + " . " + CV_SUBMINOR_VERSION ; // loading the opencv libraries at instantiation static { if ( load () != null ) { String platformName = getPlatformName () ; } } // declaring the interface of native OpenCV class public static class OpenCVProcess extends Pointer { static { Loader . load () ; } // initialize the haar cascade detector public OpenCVProcess () { allocate () ; initDetector ( AndOCVFaceDetectionEx . APPLICATION_DIR + " / haarcascade_frontalface_alt . xml " ) ; } private native void allocate () ; // main method to be called for processing image public native @Byref int run ( int width , int height , byte yuv [] , int [] rgba ) ; } }

0.4.3 Project Setup

The next step is to copy the libraries and set up project dependencies. The the javacv and javacpp files found in the javacv package to the libs directory of the android project and also add the jar files to the project. Copy all the opencv libraries to libs/armeabi directory of the android project The javacv package also provides javacv-android-arm.jar file.Unzip

7 | 11

Android OpenCV :Face Localization using Haar Cascade Classifier contents of the files which are .so library files and copy them to the libs/armeabi directory of the android project.

0.4.4 Compilation Setup

The next step is code compilation.First step is to compile the native code. Go to the libs/armeabi directory in the file system.Open the javacpp.jar file and navigate to the path /com/googlecode/javacpp/ properties and open the file android-arm.properties. make changes in the files as per the local android ndk path and set the appropriate flags as per the arm architecture of the desired target devise. below is the sample of the file that is used in the present project
1 2 3 4 5 6 7 8

9 10

11 12

13 14 15 16 17 18 19 20

platform . name = android - arm platform . root =/ opt / android - ndk - r7 / path . separator =: compiler . path = toolchains / arm - linux - androideabi -4.4.3/ prebuilt / linux - x86 / bin / arm - linux - androideabi - g ++ compiler . sysroot . prefix = - - sysroot = compiler . sysroot = platforms / android -14/ arch - arm / compiler . includepath . prefix = - I compiler . includepath = sources / cxx - stl / gnu - libstdc ++/ include /: sources / cxx - stl / gnu - libstdc ++/ libs / armeabi - v7a / include /::/ usr / local / include :/ usr / local / include / opencv :/ usr / local / include / opencv2 compiler . options = - march = armv7 - a - mfloat - abi = softfp - mfpu = vfp ffast - math -Wl , - - fix - cortex - a8 compiler . output . prefix = - Wl , - rpath , lib / - DANDROID - ffunction sections - funwind - tables - fstack - protector - funswitch - loops finline - limit =300 - Wall - O3 - nodefaultlibs - fPIC - shared -Wl , - no - allow - shlib - undefined -s -o \ u0020 compiler . options . fastfpu = - march = armv7 - a - mfloat - abi = softfp - mfpu = vfp - ffast - math -Wl , - - fix - cortex - a8 compiler . output . prefix = - Wl , - rpath , lib / - DANDROID - ffunction sections - funwind - tables - fstack - protector - funswitch - loops finline - limit =300 - Wall - O3 - nodefaultlibs - fPIC - shared -Wl , - no - allow - shlib - undefined -s -o \ u0020 compiler . linkpath . prefix = - L compiler . linkpath = sources / cxx - stl / gnu - libstdc ++/ libs / armeabi - v7a /: libs / armeabi2 compiler . link . prefix = - l compiler . link . suffix = compiler . link = log : gnustl_static : gcc : dl : z : m : c compiler . framework . prefix = - F compiler . framework . suffix = compiler . framework =

8 | 11

Android OpenCV :Face Localization using Haar Cascade Classifier

21 22 23

source . suffix =. cpp library . prefix = lib library . suffix =. so

0.4.5 Compilation
Place the commands required for compilation in the script file and execute from the workspace/main directory of the android project. First step is to copy the native header file AndroidOpenCVProcessImage.hpp to the ndk package include directory.The base directory is as per compiler.sysroot path configured in the above file(/opt/android-ndk-r7/platforms/android-14/arch-arm/). Next set the java home and other variable required by Java.This Java SDK should be same as one used by the Java IDE used to compilation of android application. Next step is to compile the Java file which provides the native interface passing the javacv and Java jar files as class path arguments. The final step is to run the Java command passing the jar files argument as javacpp.jar ,properties file name as android-arm and platform.root argument as the local android ndk path and compiled native Java class files as input and output is the shared library.

2 3 4 5 6 7 8 9 10

cp / home / pi19404 / AndroidOpenCVProcessImage . hpp / opt / android - ndk - r7 / platforms / android -14/ arch - arm / usr / include / opencv_support_class .h export home1 = ` pwd ` cd $home1 export JAVA_HOME =/ opt / softwares - linux / jdk1 .7.0 _04 / JAR = $home1 / libs / javacpp . jar JAR1 = $home1 / libs / javacv . jar SRC = $home1 / src / com / android / opencv / ProcessImage . java LIBS = $home1 / libs / armeabi $JAVA_HOME / bin / javac - cp $JAR - classpath bin / classes : build / classes : $JAR1 : $JAR $SRC $JAVA_HOME / bin / java - Dplatform . root =/ opt / android - ndk - r7 - jar $JAR - properties android - arm - classpath $home1 / src : $JAR1 : $JAR com . android . opencv . ProcessImage -d libs / armeabi

9 | 11

Android OpenCV :Face Localization using Haar Cascade Classifier This will generate the libjniProcessImage.so file in the libs/armeabi directory. Next compile the android java application .

0.5 Launch Application

Transfer the apk file generated to device and test the application.

(a) Screenshot Image

Image

0.6 Code
The code can be found in code repository https://github.com/ pi19404/m19404/tree/master/Android/AndroidOpenCV1.1 or https://code. google.com/p/m19404/source/browse/Android/AndroidOpenCV1.1. The header files is located in jni directory.The library files are not placed in the repository download them from appropriate packages on send a mail separately for download link.

10 | 11

Bibliography

Bibliography
[1] Paul A. Viola and Michael J. Jones.  Rapid Object Detection using a Boosted Cascade of Simple Features. In: CVPR (1). 2001, pp. 511518.

11 | 11