03: Sampling and Experiments

IMS, Ch. 2

Benjamin S. Baumer

Smith College

Feb 2, 2026

Sampling

Statistical inference

https://python.datasciencebook.ca/inference.html

Null distribution

Simple random sampling

http://en.wikipedia.org/wiki/Simple_random_sampling

Stratified sampling

http://en.wikipedia.org/wiki/Stratified_sampling

Cluster sampling

http://en.wikipedia.org/wiki/Cluster_sampling

Group problem set, page 1

Experiments

Confounding

  • Confounding
    • variable is correlated with both the response and explanatory variable
  • Common response
    • variable causes change in both the response and explanatory variables

Causality

  • correlation does not imply causation!
  • Be especially wary of time as a confounder
  • Causality generally cannot be inferred from observational data – only from randomized, controlled experiments.
  • Causal inference is one of the hottest fields in statistics!

https://twitter.com/causalkathy

Causal diagram (generic)

  • How does the treatment affect the outcome?

Causal diagram (example)

https://christophm.github.io/modeling-mindsets/causal-inference.html

  • How does the number of streets affect air quality?

Key terms in experimental design

  • Controlling
    • evening out differences (e.g. using a control group to simulate the action of the treatment group)
  • Randomization
  • Blocking
    • dividing groups proactively to compare like with like
  • Blinding and double-blinding
    • prevent bias on the part of researchers and patients

Reproducibility vs. replicability

  • Replication
    • a study performed multiple times should get the same results!
  • How are these different?
    • replication: different people get the same results with different data
    • reproducibility: different people get the same results with the same data

Group problem set, page 2