Case Study Guidelines

Author

Kaitlyn Cook, edited by Ben Baumer

There will be three case studies assigned throughout SDS 291. These case studies give you a chance to work in pairs, grapple with an open-ended problem, select and complete an appropriate analysis, and interpret your results. The focus of these assignments is equally on the correctness of your statistical analysis and on the clarity with which you communicate your procedure and findings. The final product for each case study is a short summary report.

Why Case Studies?

Case studies look a lot like how statistics works in practice! Someone will come to you with a question—and data they would like to use to answer that question—but they will leave it up to you to decide what analysis techniques and tools are most appropriate. Case studies are also an increasingly common component of data science job interviews: interviewers will provide you with an open-ended data-based problem, and will then evaluate your ability to problem solve, select appropriate analyses, and clearly communicate your thinking.

Case studies in SDS 291 are thus designed to be a fun way to sharpen your statistical modeling and problem-solving skills and to prepare you for these “real world” situations!

Content

Report

Your written report is a big-picture summary of your analysis and conclusions. It should be 500 to 1,000 words in length, with the following components:

  • An introduction to the problem. Your introduction should provide sufficient background information to motivate and explain your analysis. In short, what question(s) are you trying to answer and why?
  • A description of the data you have on hand to answer your question. Your description should include a discussion of relevant summary statistics (EDA) in context, with at least one accompanying graph.
  • A (brief) description of your modeling process, the final model(s) explored and the relevant results. This description must include:
    • A mathematical description of your final population model(s) for the mean, with all relevant variables clearly defined. For example, \[ \mathbb{E}[\log(income) | education] = \beta_0 + \beta_1 \cdot education, \] where \(\log(income)\) is the natural logarithm of an individual’s net income (measured in US dollars in 2023) and \(education\) is the number of years of schooling the individual has completed.
    • The estimates of the model parameters (e.g., \(\beta_0\) and \(\beta_1\) in the above population model) and their accompanying standard errors.
    • Interpretations of the relevant results in context and with accompanying confidence interval(s) or prediction interval(s), as necessary to answer the main question(s) posed by the case study
  • A final summary of your findings and conclusions, as well as a discussion of any limitations of your analysis and conclusions. When you think about possible limitations of your analysis, some questions to consider include: to what population(s) do your results generalize (and is this the population that you are actually most interested in)? Can the associations you describe be interpreted as cause-and-effect relationships? Are there any lingering assumption violations that you have concerns about?
  • Your written report should include the R code you used to conduct your analysis (including regression models, assumption checks, and code that you used to make plots and tables). Your code should be commented throughout, telling me what each section of code is doing. Please edit your code so that there are no typos or errors, and—while I do want to see the associated R output or plots from your code—please avoid printing entire (or partial!) datasets to the screen.

Format & Case Study Submission

Your report should be written entirely in Quarto.

Your final submission to Moodle should be a single Quarto document (*.qmd).

Time Investment

Case studies are intended to take no more than 5 to 7 hours. In general:

  • Aim to spend about 3 hours exploring the data, deciding on an analysis approach, and building and assessing your models.
  • The remaining time should be spent on interpreting your results, writing up your findings, and polishing your final report.

As you work, keep in mind that “done is better than perfect.” You should start by completing a rough draft of simpler analyses that sufficiently address the case study objectives; then, as time permits, you can refine and extend your statistical approach.

Case Study Rubric

Case studies will be evaluated holistically on the basis of the formal components of the report (introduction, data description, methods/results, and discussion), the statistical methods selected, and the writing style. Each of these components will be graded out of either 5 or 10 points, for a total of 45 points:

  • Score of 5 (or 10): work that is exemplary
  • Score of 4 (or 8): work that is very good
  • Score of 3 (or 6): work that is satisfactory
  • Score of 2 (or 4): work that is below expectations
  • Score of 1 (or 2): a reasonable attempt.

The bulleted items below represent guidelines for each of the six components. An exemplary report should generally be able to answer YES to most of these items. Please note that case studies are only intended to be 500 to 1,000 words in length, so each item below may reasonably be addressed in only a sentence or two!

NoteIntroduction (5 points)
Achieved Description Points
[YES/NO] The topic of the case study is clearly explained and some justification/motivation for the analysis is provided. 0–2
[YES/NO] The research question and goals of the analysis are clearly explained. 0–2
[YES/NO] The response and explanatory variables of interest are clearly identified. 0–1
NoteData Description & Exploratory Data Analysis (5 points)
Achieved Description Points
[YES/NO] The original source of the data is clearly explained. 0–1
[YES/NO] Any important data wrangling steps (e.g., removing observations with missing values, constructing new variables) are clearly described. 0–1
[YES/NO] The number of observations in the dataset (and the number of observations removed from analysis) is clearly identified. 0–1
[YES/NO] The distributions of the response variable and relevant explanatory variables (as well as the relationships between them) are clearly and accurately described. This description should draw on appropriate summary statistics (e.g., the range, the mean/median) and involve at least one data visualization; these statistics and data visualization should be unpacked and interpreted for the reader. 0–2
NoteStatistical Methods (10 points)
Achieved Description Points
[YES/NO] The statistical methods and modeling approach that were chosen are logical. They are appropriately motivated by the research question and exploratory data analysis. 0–4
[YES/NO] The methods and approach are described succinctly. The level of detail should be such that someone knowledgeable about regression could replicate your analysis (perhaps using a programming language other than R) and evaluate the analysis decisions that you made. 0–3
[YES/NO] A mathematical representation of the final population regression model for the mean is provided without errors. 0–2
[YES/NO] The notation used for the response and explanatory variable(s) in this model is clearly and succinctly defined. 0–1
NoteResults (10 points)
Achieved Description Points
[YES/NO] The estimates of the model parameters and their accompanying standard errors are provided, perhaps in a table. 0–4
[YES/NO] Relevant features of the model (as determined by the stated research question and goals) are fully interpreted in context for the reader. Note that this is not a requirement that all components of the model are interpreted; instead, you should interpret only those estimated slopes (or hypothesis tests, population means, or predictions) that are important to the narrative of your case study and findings. 0–5
[YES/NO] Appropriate measures of uncertainty are provided (perhaps parenthetically) for the interpreted quantities: for slopes or population means, a confidence interval; for hypothesis tests, the test statistic and \(p\)-value; for predictions, a prediction interval. 0–2
[YES/NO] The final results of the analysis are logical and explained in a clear and straightforward manner. These results are clearly tied back to the research question and goals. 0–2
[YES/NO] The regression conditions and model goodness-of-fit are investigated and appropriately assessed. These items may be included in the R appendix rather than the case study write-up; if included in the R appendix, this code should be clearly marked with comments. 0–1
NoteDiscussion (5 points)
Achieved Description Points
[YES/NO] The case study objectives, results, and conclusion are clearly restated. 0–1
[YES/NO] The results are discussed in the broader context of the case study (i.e., the results and conclusion are connected back to the greater motivation for the analysis). 0–1
[YES/NO] Potential limitations of the data and/or the analysis are acknowledged and briefly discussed. 0–3
NoteWriting Style (10 points)
Achieved Description Points
[YES/NO] The report has a descriptive title and the names of all authors are provided. 0–1
[YES/NO] Throughout the report, ideas are expressed precisely and are written in a manner that is almost entirely free from spelling and/or grammatical error. 0–1
[YES/NO] Throughout the report, the writing and exposition are geared towards someone with a similar level of statistical training and domain knowledge. 0–2
[YES/NO] The figures and tables included in the case study complement the writing and support the themes/arguments of the case study. 0–2
[YES/NO] These figures and tables are fully unpacked and interpreted for the reader. 0–2
[YES/NO] Figures and tables are appropriately formatted, styled, and polished. Figures should include descriptive axis labels and titles. 0–1
[YES/NO] The R code is interwoven sensibly in the case study. The report is free from messages, warning messages, and error messages. 0–1
[YES/NO] The report contains all the R code used for the case study, with clear documentation. 0–1