Public Administration and Policy Review ›› 2024, Vol. 13 ›› Issue (4): 88-.

Previous Articles     Next Articles

Improving the Adaptability of Experimental Replication in Public Administration: A Bayesian Experimental Design Framework

  

  • Online:2024-07-17 Published:2024-06-27

提高公共管理实验复制的适应性:一种贝叶斯实验设计框架

  

Abstract:

In recent years, public administration experiments and their replication have become an important way to improve the generalizability of theories. However, experimental replication still faces a series of methodological issues that need to be addressed: how to effectively establish the relevance and comparability of replication with the original study? How to design and analyze replication experiments reasonably and allow for flexible adjustments? How to optimize the design to reduce sample size and cost, and to improve efficiency and effectiveness? This paper proposes a design framework based on Bayesian Experiments to provide a more adaptive path for replication of public administration experiments. Unlike Randomized Controlled Trials (RCTs) based on Frequency Statistics, Bayesian Experiments can, first, take background knowledge such as original research as priors and perform Bayesian updating based on data and likelihood functions. Second, the use of posterior probabilities instead of using P-values to test research hypotheses avoids problems such as P-hacking. In a sequential setting, subsequent designs can be quickly adjusted based on previous results, while ensuring comparability of results across experimental arms. It is also possible to determine the location, context, and sample for further experimental replication based on structural speculation. Finally, Bayesian experiments can be used to transform the intervention maximization problem into a "multi-armed bandit problem" in reinforcement learning by using algorithms such as as Thompson sampling to deterministically assign subjects, it could significantly reduce the sample size and experimental cost, and has a wide range of application prospects.

Key words: Experimental Replication, Generalizability, Randomized , Controlled Trials, Bayesian Experiment, Adaptive Design

摘要:

近年来,公共管理实验及其复制成为提高理论可推广性的一个重要途径。然而,实验复制仍然面临一系列方法论问题有待解决:如何有效建立复制与原始研究的相关性与可比性? 如何合理地设计并分析复制实验,并且允许进行灵活调整? 如何优化设计来降低样本量与成本,并提高效率与效果? 本文提出了一种基于贝叶斯实验的设计框架,为公共管理实验复制提供更具适应性的路径。与基于频率统计学的随机对照试验 (RCT)不同,首先,贝叶斯实验可以将原始研究等背景知识作为先验概率,基于数据与似然函数进行贝叶斯更新。其次,采用后验概率而不是使用P 值来检验研究假设,避免了根据显著性来报告结果等问题。在序贯情景下,可以基于先前结果快速调整后续设计,同时保证各实验臂结果的可比性。还可以基于结构性推测来确定进一步实验复制的地点、背景与样本。最后,贝叶斯实验通过将干预效果最大化问题转换为强化学习中的 “多臂老虎机问题”,使用汤普森采样等算法来确定性地分配样本,能显著降低样本量和实验成本,具有广泛的应用前景。

关键词: 实验复制, 可推广性, 随机对照试验, 贝叶斯实验, 适应性设计