International Seminar on Selective Inference

A weekly online seminar on selective inference, multiple testing, and post-selection inference.

Gratefully inspired by the Online Causal Inference Seminar

Mailing List

For announcements and Zoom invitations please subscribe to our mailing list.

Upcoming Seminar Presentations

All seminars take place Tuesdays at 8:30 am PT / 11:30 am ET / 4:30 pm London / 6:30 pm Tel Aviv. Past seminar presentations are posted here.

• Tuesday, May 20, 2025 [Link]

  • Speaker: Aytijhya Saha (Indian Statistical Institute)

  • Title: Post-detection inference for sequential changepoint localization

  • Abstract: This talk will focus on a fundamental but largely unexplored challenge in sequential changepoint analysis: conducting inference following a detected change. We study the problem of localizing the changepoint using only the data observed up to a data-dependent stopping time at which a sequential detection algorithm $\mathcal A$ declares a change. We first construct confidence sets for the unknown changepoint when pre- and post-change distributions are assumed to be known. We then extend our framework to composite pre- and post-change scenarios. We impose no conditions on the observation space or on $\mathcal A$ --- we only need to be able to run $\mathcal A$ on simulated data sequences. In summary, this work offers both theoretically sound and practically effective tools for sequential changepoint localization.

  • Discussant: Yao Xie (Georgia Institute of Technology)

  • Links:  [Relevant papers: paper #1]

• Tuesday, May 27, 2025 [Link]

  • Speaker: Neil Xu (Carnegie Mellon University)

  • Title: Bringing Closure to FDR Control With a Uniform Improvement of the e-Benjamini-Hochberg Procedure

  • Abstract: We present a novel necessary and sufficient principle for multiple testing methods. This principle asserts that every multiple testing method is a special case of a general closed testing procedure based on e-values. It generalizes the standard closure principle, known to underlie all methods controlling familywise error and tail probabilities of false discovery proportions, to a large class of error rates --- in particular, this generalized closure principle applies to methods controlling the false discovery rate (FDR). By writing existing methods as special cases of this procedure, we can achieve uniform improvements of these methods, and we show this in particular for the eBH and the BY procedures, as well as the self-consistent method of Su (2018). We also show that methods derived using the closure principle have several valuable properties. They generally control their error rate not just for one rejected set, but simultaneously over many, allowing post hoc flexibility for the researcher. Moreover, we show that because all multiple testing methods for all error rates are special cases of the same procedure, researchers may even choose the target error rate post hoc. Under certain conditions, this flexibility even extends to post hoc choice of the nominal error rate. In addition, the closure principle allows methods to exploit logical relationships between hypotheses to gain power.

This is joint work with Aldo Solari, Lasse Fischer, Rianne de Heide, Aaditya Ramdas, and Jelle Goeman.

• • Discussant: 

  • Links:  [Relevant papers: paper #1, paper #2]

• Tuesday, June 3, 2025 [Link]

  • Speaker: Ziang Niu (University of Pennsylvania)

  • Title: Assumption-lean weak limits and tests for two-stage adaptive experiments

  • Abstract: Adaptive experiments are becoming increasingly popular in real-world applications for effectively maximizing in-sample welfare and efficiency by data-driven sampling. Despite their growing prevalence, however, the statistical foundations for valid inference in such settings remain underdeveloped. Focusing on two-stage adaptive experimental designs, we address this gap by deriving new weak convergence results for mean outcomes and their differences. In particular, our results apply to a broad class of estimators, weighted inverse probability weighting (WIPW) estimators. In contrast to prior work, our results require significantly weaker assumptions and sharply characterize phase transitions in limiting behavior across different signal regimes. Through this common lens, our general results unify previously fragmented results under the two-stage setup. To address potential non-normal limiting behavior, we propose a computationally efficient and provably valid plug-in bootstrap method for hypothesis testing. Our results and approaches are sufficiently general to accommodate various adaptive experimental designs, including batched bandit and subgroup enrichment experiments. Simulations and semi-synthetic studies demonstrate the practical value of our approach, revealing statistical phenomena unique to adaptive experiments.

  • Discussant: 

  • Links:  [Relevant papers: ]

What is selective inference?

Broadly construed, selective inference means searching for interesting patterns in data, usually with inferential guarantees that account for the search process. It encompasses:

• Multiple testing: testing many hypotheses at once (and paying disproportionate attention to rejections)

• Post-selection inference: examining the data to decide what question to ask, or what model to use, then carrying out one or more appropriate inferences

• Adaptive / interactive inference: sequentially asking one question after another of the same data set, where each question is informed by the answers to preceding questions

• Cheating: cherry-picking, double dipping, data snooping, data dredging, p-hacking, HARKing, and other low-down dirty rotten tricks; basically any of the above, but done wrong!