Behind the numbers: Questioning questionnaires

Is complex, ambiguous, and fluctuating social reality measurable? Sometimes yes, perhaps, but often not. At least not in the fairly straightforward way assumed by many researchers. This study is an ethnographic inquiry into data collection during a survey research project. Based on our observations of participants’ spontaneous thoughts and confusions as they filled in questionnaires on “leadership” and “teamwork”, we draw attention to hidden problems in much organizational research. Many respondents found measures ambiguous, irrelevant, or misleading. We (a) underline the inherently interpretative nature of research into complex organizational phenomena, (b) warn against lack of reflexivity and overreliance on existing survey instruments when we study complex social aspects of organizations, (c) identify five categories of possible problems, and (d) suggest paths towards better informed research that take context seriously.