Designing Experiments in the Social Sciences
Lecturer: Bernhard Kittel
Modality: In presence
Week 2: 17-21 August 2026
Workshop contents and objectives
An experiment is a method of data collection under controlled conditions. By manipulating the explanatory variable, controlling potentially confounding factors, and randomizing the assignment of participants to different treatments and control conditions, the method is well-suited to generate data for rigorously testing hypotheses derived from theories. Given the opportunity to create a wide variety of social contexts in the laboratory, it also enables researchers to explore counterfactual situations. Likewise, it allows us to systematically study the effects of interventions in everyday situations and study attitudes, behavioral intentions, and behavior that would be susceptible to social desirability bias in ordinary surveys.
The course offers a broad introduction to the design of experiments in sociology, political science and economics. Its objective is to enable participants to develop and implement their own experiments. While going through the full circle of developing a research question and a theory-based hypothesis, designing an experiment that captures the essential elements to test the hypothesis, implementing the experiment and analyzing the data, it discusses the considerations relevant in each stage of this process. For this test, the participants design a laboratory, a field and a survey experiment. In doing this, they are introduced to the particularities of each type and the differences between them. The course further elaborates on issues such as validity and the role of incentives as well as ethical questions. Software programming of experiments is not part of the course, but participants will be briefly introduced to some available options.
Workshop design
The course consists of a mixture of lectures, practical tasks and discussions. Typically, the mornings focus on lectures and discussions, whereas the afternoons are reserved for practical work in groups of two or three participants and the discussion of that work in plenum. On day 4, each group implements one of the experimental designs prepared in the previous days and participants serve as subjects for each other’s experiments, perhaps (and hopefully) complemented by other summer school participants who are recruited by advertisement. Or they may even unwittingly participate in a field experiment administered on campus. Groups analyze their data and present the results in the afternoon of the final day.
Detailed lecture plan (daily schedule)
| Day 1 | Morning | |
| 45 min | Introduction | |
| 90 min | Practical session (plenum): Example of laboratory experiment | |
| Afternoon | ||
| 45 min | Theory in experiments | |
| 90 min | Practical session (group): Theory development | |
| 90 min | Practical session (group): Derivation of testable hypothesis | |
| Day 2 | Morning | |
| 90 min | Causality and experimental design | |
| 90 min | Laboratory experiments | |
| Afternoon | ||
| 90 min | Field experiments | |
| 90 min | Practical session (group): Design of own laboratory or field experiment (paper and pencil) | |
| 90 min | Practical session (plenum): Group discussion of experimental designs | |
| Day 3 | Morning | |
| 90 min | Validity | |
| 90 min | Survey experiments: Principles | |
| Afternoon | ||
| 90 min | Survey experiments: Design issues | |
| 90 min | Practical session (group): Design of own experiment (survey) | |
| 90 min | Practical session (plenum): Group discussion of experimental designs | |
| Day 4 | Morning | |
| 90 min | Statistical analysis of experimental data | |
| 90 min | Practical session (individual): Models for analyzing experimental designs | |
| Afternoon | ||
| 90 min | Practical session (group): Implementation of one of the own experiments | |
| 90 min | Practical session (group): Implementation of one of the own experiments | |
| 90 min | Practical session (group): Data analysis of the implemented experiment | |
| Day 5 | Morning | |
| 60 min | Incentives | |
| 60 min | Ethics and deception | |
| 30 min | Natural experiments | |
| 30 min | Software | |
| Afternoon | ||
| 120 min | Practical session (plenum): Presentation of results |
Class materials
All materials will be provided online.
Prerequisites
Basic understanding of statistics: tests for categorical, ordinal and interval data, regression.
Recommended reading or preliminary material
- Barrera, D., Gërxhani, K., Kittel, B., Miller, L. and Wolbring, T. (2024): Experimental Sociology. Outline of a Scientific Field. Cambridge: Cambridge University Press.
- Kittel, B., Luhan, W. J. and Morton, R. B. (ed.) (2012): Experimental Political Science: Principles and Practices. Basingstoke: Palgrave-Macmillan.
- Morton, R. B. and Williams, K. (2010): Experimental Political Science and the Study of Causality. From Nature to the Lab. Cambridge: Cambridge University Press.
- Weimann, J. and Brosig-Koch, J. (2019): Methods in Experimental Economics. Berlin: Springer.
- Willer, D. and Walker, H. A. (2007): Building Experiments. Testing Social Theory. Standford: Stanford University Press.
What our participants appreciated most
"Bernhard is extremely knowledgeable and able to illustrate the experimental designs from examples, during the week | was confidently able to develop my quasi experimental design and understand how to address complexities in my research design and incorporate them in a transparent way. ‘Outstanding :-)"
"The workshop was great! | really enjoyed gaining a lot of theoretical knowledge and having the opportunity to apply it directly to my own research. | also appreciated that the professor gave us the chance to work individually."
Bernhard Kittel
University of Vienna, Austria
Bernhard Kittel is professor of economic sociology at the University of Vienna. Previous appointments include the Max Planck Institute for the Study of Societies in Cologne, the University of Bremen, the Universiteit van Amsterdam and the Carl von Ossietzky University Oldenburg. His work focuses on justice attitudes, group decision making and marginal groups in the labor market, using experimental methods, population surveys and interviews. More recently, he has also been involved in work on deliberation, and he has been co-organizer of a citizens convention on the Austrian security strategy. He is co-author, with Davide Barrera, Klarita Gërxhani, Luis Miller and Tobias Wolbring, of Experimental Sociology. Outline of a Scientific Field (Cambridge University Press, 2025). With Stefan Traub he has co-edited the volume Priority of Needs? An Informed Theory of Need-based Justice (Springer, 2024). 2020-2023 he has directed the Austrian Corona Panel Project which collected data on the societal implications of the Covid-19 pandemic. In 2017-18, he has been a member of the questionnaire design team for the module on Fairness and Justice in Europe in round 9 of the European Social Survey. He has been academic convenor of the ECPR Methods School in Ljubljana (2006-2015) and he is currently chairperson of the scientific advisory board of GESIS.