Aim The use of potentially inappropriate medications (PIMs) for older patients is a major public health problem. However, there is little information regarding PIMs in Japanese primary care settings, and the association between psychological problems and PIMs is unknown. The present study was carried out to explore the prevalence of PIMs among older patients in a primary care setting in Tokyo, and to assess the association between PIMs and depression and anxiety. Methods A cross‐sectional study in a Japanese outpatient clinic providing primary care was carried out. Data were collected from January 2016 to March 2016. A total of 740 patients aged ≥65 years with chronic diseases were enrolled. Information regarding sociodemographic status, comorbidities, and prescription and psychological status was collected using a questionnaire that patients were required to complete, and by reviewing the patients’ medical records. Results The Screening Tool of Older Person's Prescriptions criteria version 2 revealed PIM prescription for 32.3% of patients. Benzodiazepines, hypnotic Z‐drugs and proton pump inhibitors accounted for a majority of PIMs. After adjusting for age, sex, comorbidities, estimated glomerular filtration rate and the number of medications, anxiety was identified as a predictor for PIMs. Conclusions PIMs among older patients are common in Japanese primary care settings, and prescription of benzodiazepines or hypnotic Z‐drugs and proton pump inhibitors was frequent as PIMs. PIMs might be associated with anxiety; therefore, their association should be focused on and addressed to reduce PIMs. Geriatr Gerontol Int 2017; ••: ••–••.