在KTH做的第一个比较有意思的实验，虽然个人觉得有点无聊，不过老师们貌似都挺感兴趣，和大家分享下，欢迎点评。

Keywords:	Face Recognition,Masquerading, Filch, Counterfeit
Date:	2010-10-11
Status:	Complete
Prerequisites:	Computers with windows system, Ability of Image editing, Face recognition system, Camera, Printer
Person hours:	To complete: 48 person-hours To repeat: 30 person-hours
Educational value:	*****_____
Booked by:	Huan Meng ( huanm@kth.se ) & Jing Ba ( jingb@kth.se )

  Copyright: Huan Meng ( huanm@kth.se ) & Jing Ba ( jingb@kth.se )

Autumn Term 2010 Proposal

 

FaceRecognitionAttack

 

Summary

Now the face identification has been used in many security areas. We can get the pictures of victims from video chat, a photo and so on, then use these information to have a unauthorized access to a system with face identification mechanism.

Goals

Try to Invade a system with ASUS Smartlogon face identification by using a counterfeit of face recognition and analyze some feasible solutions.

Method

1.analyze the principle of operation about the face identification and ASUS Smartlogon system

2.filch the pictures of a familiar person’s(the victim) face(from video chat, photos or some samples from his computer).

3. process the images and make some facial image samples of victim.

4.access the victim’s system with a counterfeit face identification.

5.analyze the attack and some solutions to deal with the attack.

Introduction and Smartlogon analysis

Facial recognition is a digital application used to identify and verify the identity through digital images or videos.

In our this assignment,the target is the software called Smartlogon.In this software,it has used the feature-based recognition algorithms and appearance-based recognition algorithms.So the method of facial recognition it uses is mainly the method with geometrical character.Because of this method,the identification application has its deficiency,we can just use the image replacement to cheat it when we need to log on.According to this point,we can attack the identification application through stealing photos in victim’s PC and printing screen[1].

After checking the documents about the software,we can find a database in this software to reserve the users’ images.The database saved hundreds of the user’s images.And we find that it takes dozens of photos from user everytime,then the algorithm synthesizes these photos and makes a integrative image to be the identification standard.So we can intromit some our own images in order to diluting the standard.But there is one thing very important,it’s that we have to intromit enough images.

Traditional techniques

 

 The basic method:

1.Face recognition using geometrical character

This method uses the geometrical relationship between the facial organs such as the interval.

2.Method based on eigenface(PCA) [2]

This method is based on KL transformation which is the optimal orthogonal transformation about image compression.

3.Neural network method

This method uses the study capacity and classification ability of neural network.

4.Elastic graph matching method [2]

This method uses zeniths in topological diagram to express facial eigenvector,then records the information around the zeniths.

5.LHD method

This method uses distance of the line sets extracted from the facial grayscale images.It’s a better choise in some different light conditions.

The algorithms used in the facial recognition:

1.Feature-based recognition algorithms

2.Appearance-based recognition algorithms

3.Template-based recognition algorithms

4.Recognition algorithms using neural network

5.Illumination pretreatment based on Gammar gray-level registration[3]

3D-method:

This method uses not only the distance of the specific facial organs,but also the individual features of one organ such as the height and depth.So it could avoid the affection from the lighting.And it could finish the identification within different angles. [9]

ASUS Smartlogon Setup:

Click the link below to download ASUS Smartlogon:

http://asus-smartlogon.software.informer.com/

ASUS Smartlogon analysis:

 

Principles of Face recognition

1. Select a user name and enter a password. Your password is used when for some reason your face fail to logon to Windows.

2. You can change the security level using the slider in order to better protect your Notebook PC You can change the security level using the slider in order to better protect your Notebook PC from others logging on but may also block yourself if you change your appearance. (Your typed password can still be used to logon.)

3. Before a face is registered, question marks will appear above the head. The number 1 or 2 in the top right of the camera screen is the number of faces detected. This face recognition can capture several people’s faces simultaneously and identify the user.

4. After you registered a face, the user’s name (here is Darren Meng) will appear above the head.

Principles of Logon Process

 1. Users can view this log to check when someone tried to login with face detection on your computer.

2. There are three results “Success”, “Fail”, and “Learning”. Learning is gained when a user inputs a password to login and face captures are imported (red circle above).

Options

Camera Devices: The built-in camera is selected as default. Change if necessary.

Desktop Control: Check box and set timer to enable. See previous “Menu Items” for descriptions.

ASUS Smartlogon database

C:\Program Files (x86)\ASUS\SmartLogon\log\image

This file records the every image which is successful to logon Windows.

C:\Program Files (x86)\ASUS\SmartCore\gallery\d98cd74f-2f8a-44c1-b8ee-e5dde17ea6d2\imageThis file is used to store the captured images.

It is the Smartlogon database of all the user’s face images which are combined and analyzed with an algorism in order to provide the logon standard for a legitimate user.

Attack Details

 

Used Tools:

1. Photopaper and Camera FinePix HS11

We choose to use digital camera because of its excellent imaging and high resolution, which makes it possible to obtain photos with good quality and also makes it easier to extract clear face pictures from a photo. [5]

2. Laser Printer

Altough a laser printer has the advantages in speed and economy, we choose to use it mainly because of its precision. A printed picture is very different from developing the photos and the former one has low glossiness and definition. But if we use a high resolution picture with photopaper, the result of using a laser printer will change a lot. Make sure that your samples are enough to obtain a clear picture. [6]

Step1: Try to obtain a familiar victim’s face photos

Because the face recognition of Smart logon is based on the database of gallery and the rate of success logon depends on the light and PCA (Principle Components Analysis), we should use different ways to get victim’s different face samples to increase the rate matching the database and pass the logon detection. Here we adopt three methods to get picture samples in order to get proper counterfeit pictures, these methods are as below:

1. Obtain pictures from video chat with victim

We use an IM with video chat function to try to convince the victim using video chat. Then we capture several face pictures of victim during video chat. The image definition is based on the victim’s camera. The advantage of this method is that we can get the victim’s face picture directly, because we often use our head portrait during the process of video chat. Although the image definition may be low most of the time, we can edit and make them look better.

2. Obtain pictures from some victim’s life photos.

It is relatively easy to get a familiar victim’s life photos. When we go to travel or take some photos in daily life, we can get those pictures from camera. A digital camera and DSLR (Digital Single Lens Reflex) can make high quality photos, but they are usually not single face pictures, so we must extract the face pictures from those photos and the editing is also important.

3. Obtain some samples from the familiar victim’s computer

When we have the opportunity to use a familiar victim’s computer, we may find some samples of his or her face pictures (especially the picture database of SmartLogon). But the fact is that these pictures are always in small sizes and not clear enough to make a counterfeit photo. However we can refer to contrast level, brightness level and tones of pictures when we edit those face pictures.

Step2: Choose some good pictures and edit those victim’s face pictures

We will choose some face pictures from the three methods, combine them and edit them. By using Photoshop CS4, we can adjust the size, contrast level, brightness level, tones of pictures and so on, these are the most important elements for a picture, hence they also determine the rate of success logon. These are the most important elements for a picture, hence they also determine the rate of success logon.

So we give an editing scheme to those obtained pictures in order to cover more aspects of face pictures. By using this way, we can use the least samples to reach our goal. In consideration of the definition of the third method, we can not use these pictures to make a sample, but we can know some information about the database of the gallery which will help us make better samples. These are the editing methods we choose:

Editing for life photos

We cut the extra part of the photo first, what we need is just the head portrait. Notice that you should leave more space around the face rather than cut them all.

Because the picture from a digital camera has a good exposure and color, we mainly focus on changing the tone of the face picture.

As we know, the environment and the tone are very important for this kind of face recognition technique, so we should try to match the general situation considering of the different tones.

 

Editing for video chat photos

These photos are obtained from screenshots during video chat, so the quality is relatively low compared with a digital photo, but they have a more effective performance in the outline of face because of the direct captures on face. In this way, we may gain a higher rate of success logon when the face recognition system identifies the picture.

For these pictures, we should change the contrast, brightness and color saturation to make the captured pictures clearer and easier to be identified.

Step3: make some samples with these edited face pictures

After editing the obtained pictures, we need to print them by using a laser printer.

Note that the quality of a photo paper is very important to get verisimilar samples of face pictures, if you use a common paper to print those samples, the rate of success logon will be reduced much. One side of a photo paper is smooth and using a good photo paper will increase the brightness and color saturation of a picture, and it can gain the sense of layers, which makes those pictures more vivid. [4]

In the meantime, choose an advanced laser printer is also crucial.

Step4: invade the victim’s computer with a counterfeit face identification

Try to access the victim’s computer with those printed samples. This process is to cheat the Smartlogon and let it recognize our counterfeit face portrait. Use your samples carefully, when you pick up your samples, leave some space around the face pictures. If the computer can not identify the pictures quickly, don’t worry and just try to change the angles with patience. You should try to use every sample one by one to check which one can match most.

We have already uploaded the whole process to Youtube:

http://www.youtube.com/watch?v=Y6eKNV9foTo [8]

Conclusion

In this experiment, we attack the facial recognition software with counterfeit face pictures successfully. Finally, the optimum picture is the one edited from video chat. It shows that the video chat picture is more proper because the user adopts this way to set the logon. The key of the process is the image samples. If the attackers can easily get the images of a victim, they can use the photos to make counterfeit pictures to access the system unauthorized. And also, with a good design or editing ability, an attacker may create your samples more like yourself. ASUS Smartlogon uses the compound information of these photos set by a user in database and it also has an algorithm to identify the face when we try to access the system. Because of the limitation in the algorithm in plane identification and the technique Smartlogon adopts, it’s possible to invade the system by this way.Through this experiment, we can find that there are following keys in this kind of facial recognition.

1. Similarity: The faces and the facial organs of different people have similar structure, so software sometimes is hard to distinguishing the faces in details, which makes it possible to forge a face pictures.

2. Mutability: The lighting, expression and angles could affect the images of faces, so it can’t ensure that every time we log on the system, there are the totally same images appearing in front of the camera. The environment around a face can make a big difference.

Now with the fast development of the science, there have been some effectual methods approved. For example, the 3D method uses the depth and height of the facial organs to increase the characters for identification. So it can increase the security level of the facial recognition. Some more advanced methods include the elastic graph matching method which provides the treatment about the elastic deformation caused by expression change and the multiple light source method based on infrared ray which could deal with the affection from lighting and angles. [8]

Difficulties we face:

1. The lighting and angles

In the experiment, we can’t access the system protected by facial recognition just by the original photos because of the differences on lighting and angles between these photos and the source reserved in database. So we have to edit the photos to match the images, but how we edit is very crucial to determine the result.

2. Matching probability

In these attacks, although we invade the system successfully, we just used several photos. Because of the affection from the lightness, angles and expressions, the success rate is a little low. In fact, we are against our low matching probability by making more efforts on pictures editing in order to get a good result. However, if possible, we should consider of comprehensive factors.

Shortages in our attack

1. High level mechanism

In this experiment, we just attacked some lower security level recognition. If the system is protected by higher levels, the face identification will be strict. Generally, a user will not choose to use a high level security, because with that level, the user himself may hard to logon with the different logon environment. But if it does, we may need to use some techniques such as the three-dimensions and thermal imagery.

2. Sample space

Theoretically, we’d better use victim’s photos as many as possible in this method. Compared with a handful of photos we use a large number of samples will get a good result. Few pictures can’ not match the characters of the face enough because the application uses the compound results to identify. We think a large number samples should be made if possible.

Other possible improvement

Our attack is based on the face recognition detection when an attacker login. So our method is to make a counterfeit photo. But indeed, the logon identification is just one aspect of face recognition. There is an assumption and possible improvement we give to make further attack, which we think is supposed to be more exact. We can attack the database and find the short of algorithm in identification process. E.g. we can add an unauthorized people’s pictures into the Smartlogon’s gallery database so that the unauthorized people may access the system with Smartlgon by using his own face. If we do so, we should intromit the photos to the database of the identification application and this method needs enough images of attackers[9]. In the meantime, if we can find the algorithm of Smartlogon, we can also analyze the algorithm about how the Smartlogon identify these photos. However, this method has a shortage that victims can check the picture’s information in the database of the software, so the attacks can be found.

JPEGsnoop can identify a picture’s information, if we change or edit the picture, these elements of the picture will change so that we can use it to know if this picture is an original one. For a defender, this tool will help him know more about his pictures whether they are counterfeited, edited, modified or not.

Reference

[1]http://forums.whirlpool.net.au/archive/1127492

[2]http://en.wikipedia.org/wiki/Facial_recognition_system

[3]http://baike.baidu.com/view/246859.htm

[4]http://www.enet.com.cn/eschool/zhuanti/photo/

[5]http://electronics.howstuffworks.com/cameras-photography/digital/digital-camera.htm

[6]http://computer.howstuffworks.com/laser-printer.htm

[7]http://www.webopedia.com/TERM/L/laser_printer.html

Video

[8]http://www.youtube.com/watch?gl=AU&hl=en-GB&v=W_Qdag5FdE8

Paper

[9]http://www.biometrics.gov/Documents/FaceRec.pdf