Ethics Demonstration

Over the course of the semester, we have engaged in structured thinking about data science ethics. You have had readings, class discussions, and written assignments that inform your thinking about data science ethics and challenge you to analyze and articulate the ethical implications of your work.

Much of the work we have done in data science ethics to date has focused on raising awareness, building comprehension of fundamental issues, application of acquired knowledge, and analysis. The highest level in Bloom’s taxonomy is evaluation.

In this assignment, you will evaluate actors in real-world situations where data science ethics are in play.

You have the option of demonstrating your understanding via a written exam, or an in-class mock trial.

Learning goals

  • Assess the ethical implications to society of data-based research, analyses, and technology in an informed manner.
  • Use resources, such as professional guidelines, institutional review boards, and published research, to inform ethical responsibilities.

Content

Major readings

  • O’Neil (2016)
  • D’Ignazio and Klein (2020)
  • Bender et al. (2021)
  • Elliott et al. (2018)
  • Washington and Kuo (2020)
  • Lum and Isaac (2016)
  • Angwin et al. (2016)

Modules

Option 1: Written exam

You will write an essay on data science ethics, technology, and society (in Quarto), via the procedures for written exams

The essay prompt will likely be something like this:

  • Choose one episode from D’Ignazio and Klein (2020) and analyze it in the context of the Data Values and Principles manifesto. Did the actors in this episode behave ethically? Explain why or why not by linking specific actions to specific ethical principles.

In your essay, you should critically evaluate the actors in a real-world data science episode in which ethical considerations are salient. The chapter on data science ethics in Baumer et al. (2021) includes some short examples of this type of analysis.

Rubric

Ethics essay rubric
Criteria Twelve Fourteen Sixteen
Overall Quality No evaluation is made. Content is mostly description of the ethical dilemma. Ideas are not clearly connected. No outside references. Many grammatical and/or formatting errors. Ethical dilemma is described, but no evaluation is made. Opinions or claims are unsubstantiated. Actors are clearly evaluated in reference to specific ethical principles. Structure of essay is clear. Ideas are clearly connected. Outside research is relevant, authoritative, and appropriately sourced. Formatting makes essay more readable.

Option 2: Mock Trial

The goals are similar to those of the essay: We aim to put a real person on trial for violations of some set of data science ethical principles.

Roles will need to be defined by students. I will be the judge. Students who choose the essay option will serve as jurors. Remaining students will be assigned to plaintiff or defense teams as needed. Legal procedures require at least:

  • a defendant: the person accused of violating the code of ethics
  • a plaintiff: the person allegedly harmed by the defendant’s actions
  • a defense team: lawyers arguing that the defendant is not liable
  • a plantiff’s team: lawyers arguing that the defendant is liable
  • various witnesses, including expert witnesses, who need to be prepped before the trial

Steps will include:

  • Sprint 1: Choose roles
  • Sprint 2: Preliminary hearing (bring charges)
  • Sprint 3: Trial (part 1)
  • Sprint 4: Trial (part 2), jury deliberation, and verdict

References

Angwin, Julia, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2016. Machine Bias. ProPublica; ProPublica. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing.
Baumer, Benjamin S., Daniel T. Kaplan, and Nicholas J. Horton. 2021. Modern Data Science with R. 2nd ed. Chapman; Hall/CRC Press: Boca Raton. https://www.routledge.com/Modern-Data-Science-with-R/Baumer-Kaplan-Horton/p/book/9780367191498.
Bender, Emily M, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?🦜.” Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 610–23. https://doi.org/10.1145/3442188.3445922.
D’Ignazio, Catherine, and Lauren F Klein. 2020. Data Feminism. MIT Press. https://mitpress.mit.edu/books/data-feminism.
Elliott, Alan C, S Lynne Stokes, and Jing Cao. 2018. “Teaching Ethics in a Statistics Curriculum with a Cross-Cultural Emphasis.” The American Statistician 72 (4): 359–67. https://doi.org/10.1080/00031305.2017.1307140.
Lum, Kristian, and William Isaac. 2016. “To Predict and Serve?” Significance 13 (5): 14–19. https://doi.org/10.1111/j.1740-9713.2016.00960.x.
O’Neil, Cathy. 2016. Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. Crown. https://www.penguinrandomhouse.com/books/241363/weapons-of-math-destruction-by-cathy-oneil/.
Washington, Anne L, and Rachel Kuo. 2020. “Whose Side Are Ethics Codes on? Power, Responsibility and the Social Good.” Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 230–40. https://doi.org/10.1145/3351095.3372844.