Counter‐terrorism and crime prevention often depend on our ability to match images of unfamiliar faces. For example, when issuing passports, staff must establish an applicant's identity by comparing the submitted photograph witht those in the database of current passports to ensure that multiple documents are not issued to the same person under different names. Previous research has shown that this is a difficult and error prone task. We suggest that this ‘passport problem’ may be due to an over‐reliance on the appearance of external facial features that can be unreliable cues to identity. Compatible with this explanation, we demonstrate that in difficult trials involving a change of appearance or attempted fraud involving a similar looking foil, participants are better able to determine whether two images are of the same person when shown only the internal features of the faces rather than whole images. This discovery has significant practical implications and could form the basis of a procedure to improve the detection of identity fraud.Copyright © 2016 John Wiley & Sons, Ltd.