High incarceration rates have led to more interest in the effects of incarceration on families, especially children. Most research has focused on the relationship between incarcerated mothers and their children. This study centered on father–child relationships and potential barriers to maintaining and developing them. A sample of 185 incarcerated fathers was interviewed within a maximum security state prison in the southern United States. Multiple regression analyses indicated two family barriers (children’s mother’s attitude and children’s attitude) made a statistically significant contribution to explaining inmates’ self-reports concerning relationships with their children. Measures of institutional barriers were not found to statistically contribute to the models. Fathers’ responses to an open-ended question about relationships with their children support the quantitative findings. The research suggests that strategies for improving parent–child relationships for the male incarcerated population should focus not only on institutional barriers but also on family barriers.