Critically discussing the causal social ontologies presented by Dave Elder-Vass and John Searle, the article argues that these views implausibly identify the causal ontological source of human sociality in collectively known, recognized and accepted statuses, criteria, norms and the like. This is implausible, for it ignores human sociality as occurring in temporally and spatially dispersed on-going processes of human interaction of differently placed, often unequal, and thus epistemically differently equipped actors in division of labour. Human scientific concepts are best seen as picking out (aspects of) such complex and heterogeneous processes, not what the participating actors allegedly collectively know, accept or recognize about them. The article observes that similar appeals to collective recognition are also common in Wittgensteinian literature that, however, have claimed to reject the causal ontological view. It is argued that the ultimate value in rejecting the causal ontological view, and the accompanying idea of collective recognition, resides in that we thereby avoid epistemically homogenizing and over-intellectualizing the diverse mass of dispersed, often unequal, but interacting actors.