Realistic evaluation emphasizes the importance of exploring the mechanisms through which organizational interventions are effected. A well-known mechanism in organizational interventions is the screening process. Standardized questionnaires, in popular use, neither consider individuals’ appraisals of working conditions nor the specific context of the workplace. Screening with items tailored to intervention contexts may overcome the limitations of standardized questionnaires. In the present study, we evaluate an approach to develop a tailored questionnaire to measure employees’ appraisals of their specific working conditions. First, we interviewed 56 employees and 17 managers and, later, developed tailored items focused on the working conditions in a postal service. In follow-up interviews, we explore participants’ experiences with the tailored questionnaire, including the development of initiatives, compared with their previous experiences with the company’s annual attitude survey that used standardized scales. Results indicated that participants felt the tailored questionnaire highlighted issues that had previously been ignored, that initiatives were easier to develop owing to its specificity and that the feedback strategy was useful in prioritizing questionnaires. Overall, it can be concluded that tailored questionnaires may be appropriate for use in organizational intervention research and more broadly that evaluations of organizational interventions need to be contextually grounded.