Aim Mild cognitive impairment (MCI) is conceptualized as prodromal of Alzheimer's disease (AD). Declined physical function is one of the risk factors for progression to AD, whereas genetic factors, especially the apolipoprotein E ε4 allele (APOE4), also contribute to the risk of AD and affect the risk factors. However, the association between APOE4 and physical function in MCI is still unclear. To elucidate the association, we carried out the present study to compare physical function between APOE4 carriers and non‐carriers in MCI. Methods The study included 386 MCI patients (mean age 71.6 years) who were classified into APOE4 carrier and non‐carrier groups. The participants' characteristics were assessed by face‐to‐face interviews. Grip strength, five chair stand test, walking speed and 6‐min walking test were measured as physical functions. Results In univariate analysis, the APOE4 carrier group had decreased walking speed compared with the non‐carrier group (P < 0.05), but other measures of physical function were not significantly different between the groups. The linear regression model showed a significant association between walking speed and APOE4 after adjusting for covariates (P < 0.05). Conclusions The present study showed that APOE4 carrier participants with MCI had decreased walking speed compared with non‐carrier participants. Thus, APOE4 genotypes should be considered as a potential mediator when the measure of walking speed is used as a risk factor of dementia in older adults with MCI. Further studies using prospective data are required to clarify the relationship between physical function and genetic factors in MCI. Geriatr Gerontol Int 2014; ●●: ●●–●●.