Most fear-of-crime research uses resident’s neighborhood as a key reference location to measure fear, yet the location effects of one’s own dwelling unit on crime-specific fear has not been explicitly studied theoretically in the literature. Drawing upon routine activities theory, this study undertakes an investigation into the levels and determinants of residence-based fear of crime across three racial/ethnic groups—Whites, African Americans, and non-White Hispanics. Data used in the analyses were collected from a random-sample telephone survey of 1,239 respondents in Houston, Texas. The results derived from factor analyses revealed that residents do distinguish between fear in the neighborhood and fear at home. Proximity to motivated offenders measured by perception of crime was found to be the most salient predictor of fear, followed by the measures of target vulnerability and capable guardianship. In addition, residence-based fear varies significantly across racial/ethnic groups. The significance of these findings and the policy implications are highlighted.