Università della Svizzera italiana
Summer School in Social Sciences Methods
Università della Svizzera italiana
Summer School in Social Sciences Methods
6-21 August 2026

Navigating the AI Frontier in Online Survey Research: A Methodological Workshop

Instructor: Mario Callegaro & Ana Villar-Casas

Modality: In presence

Week 1: 10-14th august 2026

 

Workshop content and objectives

Course Overview

The field of survey methodology is currently undergoing its most significant paradigm shift since the transition from telephone to web-based data collection. The integration of Generative AI (GenAI) and Large Language Models (LLMs) offers unprecedented opportunities for efficiency, but it also introduces novel risks to data integrity and representativeness.

This intensive five-day workshop provides a rigorous, evidence-based roadmap for designing and executing online surveys in the AI-augmented era. We critically evaluate where AI provides genuine methodological value, and where it poses threats to data quality.

 

The AI-Integrated Curriculum

This workshop critically examines the role of AI across all stages of the research lifecycle. Participants will explore:

  • Agentic Survey Design: Moving from static forms to AI-assisted instrument development.
  • The "Bot" Threat: Identifying and mitigating the rise of agentic browsers and sophisticated automated respondents.
  • Governance & Ethics: Deep-diving into the 2026 AAPOR Guidelines on Responsible AI Integration.

 

Learning Objectives

By the conclusion of this workshop, participants will be equipped to:

  1. Master the End-to-End Workflow: Execute a professional online survey from initial sampling to final archiving.
  2. Evaluate AI Efficacy: Critically assess which AI tools enhance research quality versus those that introduce algorithmic bias.
  3. Optimize for Efficiency: Implement AI-driven strategies to reduce measurement error and streamline questionnaire programming (including Doc2Survey workflows).
  4. Safeguard Data Quality: Deploy advanced techniques to detect survey fraud in an era of human-mimicking bots.
  5. Innovate with WAS: Understand the intersection of mobile surveys with Wearables, Apps, and Sensors (WAS).

 

Workshop Structure & Methodology

This is an applied workshop. We depart from passive lecturing in favor of a laboratory environment. Participants are encouraged to bring their own research projects or work with our high-fidelity datasets. Using Qualtrics as our primary laboratory environment, we will bridge the gap between theoretical methodology and technical implementation.

 

Daily schedule

Day 1

The Theoretical Foundation & Ethical Governance

  • The LLM Landscape: Survey-specific prompt engineering (Custom Instructions, Gems, and MyGPTs).
  • Ethics & Transparency: Addressing "Black Box" algorithms in social science research.
  • Policy: Reviewing the newly released AAPOR Guidelines (May 2026) on AI integration.
Day 2

Sampling and Representation in a Digital World

  • Probability vs. Non-Probability: The evolving role of online panels.
  • Quality Metrics: Computing Absolute Average Error (AAE) and Root Mean Squared Error (RMSE) to validate sample health.
  • AI in Platforms: Assessing the native AI features emerging within modern survey software.
Day 3

Instrumented Design & Multilingual Logic

  • Automated Programming: Hands-on with Doc2Survey (Word-to-QSF conversion).
  • Complex Logic: Deploying vignettes and branching with AI-assisted validation.
  • Linguistic Precision: Handling multi-language translations and cultural adaptation via LLMs.
Day 4

Data Quality and the "Agentic" Threat

  • The New Adversaries: Identifying AI bots and agentic browsers in your data.
  • Fraud Detection: Evaluating industry-leading mechanisms for maintaining panel integrity.
  • Mobile & Beyond: Optimizing for the "Mobile-First" respondent and the role of Sensors (WAS).
Day 5

Synthesis, Analysis, and Reporting

  • Structural Understanding: Using the QASU benchmark for questionnaire processing.
  • Monitoring: Calculating AAPOR-standard response rates in the digital age.
  • The Future of Reporting: AI-assisted data preparation and archiving for Reproducible Science.

 

Target Audience

This course is designed for Master’s and PhD students, Postdoctoral researchers, and senior academics within the Social Sciences. It is equally relevant for User Experience (UX) Researchers and Market Research professionals seeking to modernize their data collection protocols with methodological rigor.

 

Prerequisites

Participants should possess a foundational understanding of the Total Survey Error (TSE) framework and basic survey design principles. A laptop is required for daily laboratory sessions.

Mario Callegaro

Mario Callegaro is an independent consultant with over 35 years of experience on survey research methods. Mario worked for 15 years at Google as survey research scientist in the marketing organization first, and then as user experience researcher in the Cloud organization. Mario obtained a Master and Ph.D. in Survey Research and Methodology from the University of Nebraska, Lincoln. His first job after the Ph.D. was to work as survey research scientist for the probability-based online panel Knowledge Panel (now Ipsos-Knowledge Panel). He is also the lead editor of the volume: Online panel research: A data quality perspective (Wiley). Mario published Web Survey Methodology with Sage, a handbook on online surveys, also available as open access, and cited over 900 times. Mario publications are all available at callegaroresearch.com.

Ana Villar Casas

Ana Villar Casas, PhD, is a survey methodologist and senior researcher with over 25 years of experience spanning high-level academia and the global technology sector. Most recently, she spent eight years as a Senior UX Researcher at Meta, where she led strategic survey measurement for integrity and safety initiatives across Facebook, Instagram, and WhatsApp. Dr. Villar Casas’s academic career includes research tenures at Stanford University and City St George’s, University of London. At City, she directed the CRONOS project, the first cross-national, probability-based online survey panel. A specialist in cross-cultural methodology and questionnaire design, her work has been featured in peer-reviewed journals and books covering survey response styles, translation-related measurement effects, and public opinion on climate change. She holds a Ph.D. in Survey Research and Methodology from the University of Nebraska-Lincoln.

  • Share

Print