The advent of the Internet has opened wholly new possibilities for data collecting. At the same time, several unanswered methodological pitfalls are present, possibly questioning the usability of data from web surveys. This article contributes to the scientific debate regarding the usability and reliability of web surveys in the social sciences by discussing the reliability of the data collected in passive sampling web surveys. The article analyzes two surveys containing the same questions, of which the first is based on a probabilistic offline sample, the other on a passive sampling online sample. Both surveys were aimed at measuring the respondents’ knowledge of politics. The findings related to the demographic characteristics were in-line with previous research: men, young, better educated, and politically interested persons dominated the online sample. Regarding the reliability of the results of the knowledge questions, the findings show that online surveys are capable of providing valuable and reliable information about the differences (and similarities) between different groups of respondents, that is, about general trends. However, the findings strongly suggest that passive sampling web surveys should not be used for drawing conclusions about the absolute levels of political knowledge in a population. The big question which remains is whether the Internet can be seen as just another sampling environment or does the "unstructured anonymity" of the Internet require tapping into a wholly new sampling methodology.