85% of China's GHG emissions are attributed to urban economic activities, and this share is expected to rise given China's fast urbanization process. This paper provides estimates of city‐level industrial CO2 emissions and their growth rates for all 287 Chinese prefecture‐level and above cities during the years 1998–2009. We decompose the CO2 emission changes into scale, composition and technique effects. The decomposition results show that these three effects differ significantly across the three tiers of cities in China. The scale effect contributes to rising CO2 emissions, while the technique effect leads to declining CO2 emissions in all cities. The composition effect leads to increasing CO2 emissions in the third‐tier cities, while it reduces CO2 emissions in the first and second‐tier cities, due to the relocation of energy‐intensive industries from the latter to the former type of cities. Based on these decomposition results, we identify the separate channels through which the inflow of FDI and the environmental regulations affect city‐level CO2 emissions. The decomposition framework in our paper can help policy makers and scholars to better understand Chinese cities’ trade‐offs between economic growth and environmental goals.