Advanced Multimedia Processing Lab -- Projects -- Face Tracking

            About AMP Lab        Projects        Downloads        Publications        People        Links

Project - Face Tracking

Contents

 

Team Member

Fu Jie Huang

jhuangfu@cmu.edu

Top of this page

                                  

Goal

Face tracking can serve as a front end to further analysis modules, such as face recognition, face expression analysis, gaze tracking and lip-reading, to name a few. Face tracking is also a core component to enable the computer to "see" the computer user in a Human-Computer Interface system.

We want to build a robust and accurate face tracking system running in real-time, as shown in the Quicktime demo video (Realvideo encoded):

 

 

 

Also we want to extend current single face tracking system to track multiple faces, even when there's occlusion between faces. The image below shows some frames extracted from a real-time video sequence (Realvideo encoded):

 

Top of this page

                                         

System Description

The face tracking system is based on the stochastic color model and the deformable template.

Top of this page

 

Download

We build a face tracking toolkit based the algorithm described above. The toolkit is pre-compiled as a dynamic link library (.dll) file on the Windows platform, and a static library on the Linux platform.

Here's some description about the library:

This Face Tracking SDK is a dynamic link library (dll) written in C++. You can link the library to your own program and call the functions provided by the library to perform face tracking in a color image sequence. The core function of the library is FtTrackNextFrame (RECT &FaceRect, unsigned char *ImgData), in which you need to tell the function the location of the face in the previous image frame and provide the image data buffer. The function will update the face location parameters using the information in the current image frame. Note that the face location in the previous frame is stored in the variable FaceRect of type RECT, and the function will update the content of this variable. The pointer ImgData points to a memory buffer containing one image frame in the format of RGBRGB…, with each pixel taking up three bytes.

To use the library, you first need to include the header file named FaceTrack.h into your program. Then you can call the functions provided by the library in your own program. When your program runs, it will dynamically link to the library which is stored at the same directory or in a directory in the system path. 

In your program, you first need to create an object of the type CfaceTrack. It has three main member functions for you to use:

Here’s an example program segment:

{
    CfaceTrack   FTrack;            
/*define a face tracking object*/
    FTrack.FtInitialization (640, 480);
         /* set the image size to 640x480 */
    unsigned char ImgData = ReadAFrame();         /* read a frame from image sequence
                                     this ReadAFrame function is not defined by the library and should be implementd by the user */
    RECT FaceRect = GetFacePosition ();         /* get face position, only needed at the very first frame 
                                    this GetFacePosition function is not defined by the library and should be implementd by the user */
    FTrack.FtTraining (FaceRect, ImgData);
  /* train classifier using current frame */
    BOOL bSuccess =  TRUE;

    while (bSuccess && !IsSequenceEmpty()){
        
ImgData = ReadAFrame();  /* read a new frame */
        
bSuccess = FTrack.FtTrackNextFrame (FaceRect, ImgData);/* update the parameters */
 
       
if (bSuccess){
                /* TODO: user can add the any other functions here with the given face location*/
         }
    }
}

Download the library.

Top of this page

 

Our work is used by...

Top of this page

 

Publications

Top of this page

 

Contact

Any suggestions or comments are welcome. Please send them to Fu Jie Huang

Top of this page