Videos are often used in web surveys to assess attitudes. While including videos may allow researchers to test immediate reactions, there may be issues associated with displaying videos that are overlooked. In this article, we examine the effects of using video stimuli on responses in a probability-based web survey. Specifically, we evaluate the association between demographics, mobile device usage, and the ability to view videos; differences in ad recall based on whether respondents saw a video or still images of the video; whether respondents’ complete viewing of videos is related to presentation order; and the data quality of follow-up questions to the videos as a function of presentation order and complete viewing. Overall, we found that respondents using mobile browsers were less likely to be able to view videos in the survey. Those who could view videos were more likely to indicate recall compared to those who viewed images, and videos that were shown later in the survey were viewed in their entirety less frequently than those shown earlier. These results directly pertain to the legitimacy of using videos in web surveys to gather data about attitudes.