According to racial invariance positions and mainstream sociological perspectives on race and crime, race differences in structural conditions should account for most if not all of the racial composition (or percent black) effect on aggregate‐level violence rates. However, prior research (mostly conducted prior to 1990) generally provides mixed or contrary evidence for this position, showing instead that greater concentrations of blacks are linked to increased violence even after accounting for racial differences in socioeconomic conditions. The current study uses recent data and a novel unit of analysis to go beyond extant research in two ways. First, we include percent Latino in our examination of the extent to which both racial and ethnic composition effects on violent crime rates are mediated by racial/ethnic disparities in socioeconomic disadvantage. Second, we test whether racial/ethnic composition effects are conditioned by size of place, through the use of census places as a uniquely varying unit of analysis. We find that both black and Latino composition effects are partly explained by controlling for structural conditions (especially structural disadvantage), but this characterizes smaller places much more than the largest, most urbanized places.