Aim Although lower glycated hemoglobin (HbA1c) has been believed to be an important marker of improvement of glycemic control in order to maintain better quality of life for patients with diabetes mellitus, a too low HbA1c might be harmful in older adults. We investigated whether this was the case with respect to risk of support/care‐need certification in community‐dwelling older patients with type 2 diabetes mellitus. Methods We analyzed 184 diabetes patients aged 65–94 years receiving glucose‐lowering medication/insulin. The end‐points were first support/care‐need certification and/or death. The relationships between four classes of HbA1c and risk of support/care‐need certification and/or death were determined using the Cox proportional hazards regression model. Results During 5 years, 42 first support/care‐need certifications and 13 deaths occurred. The association of HbA1c with risk of support/care‐need certification after adjustment for age, sex and confounding variables was J‐shaped, with the nadir at an HbA1c level of 6.5 to <7.0%, and with an increased risk of support/care‐need certification (HR 3.45, 95% CI 1.02–11.6, P = 0.046) at an HbA1c level of <6.0% compared with the nadir. When compared with patients with HbA1c ≥6.0%, those with HbA1c <6.0% showed a higher risk of support/care‐need certification as a result of dementia (HR 12.5, 95% CI 3.00–52.2, P = 0.001), but not as a result of arthralgia/fracture, stroke or other disorders. Conclusions These observations show that a too low HbA1c might be associated with a later risk of incident disability as a result of dementia in community‐dwelling older diabetes patients. Geriatr Gerontol Int 2017; •••: ••–••.