We compared 121 native and 114 non-native speakers of Dutch (with 35 different first languages) on four digit-span tasks, varying modality (visual/auditory) and direction (forward/backward). An interaction was observed between nativeness and modality, such that, while natives performed better than the non-natives on the auditory tasks (which were performed in the non-natives’ second language), performance on the visual tasks (which was performed in participants’ dominant language) did not significantly differ between natives and non-natives. The interaction between nativeness and modality disappeared when the data were corrected for Dutch proficiency. Correction for Dutch proficiency elevated non-native speakers’ scores on the auditory tasks, without altering the non-natives’ digit-span rank order. Despite considerable differences in mean length of the digit names zero to nine in the non-natives’ first languages, these differences were not significantly correlated with their visual digit-span scores. While further research is needed on the sources of variation in digit-span performance, we recommend the use of the visual digit-span task (forward or backward) for cross-linguistic research and advise researchers to be aware of the association between language proficiency and verbal working-memory performance.