Abstract
So much is known about SCOTUS oral hearings, so little is known about SCC oral hearings. Why? The answer at least partly turns on the availability of evidence: Americans have ready access to transcripts of Supreme Court hearings and Canadians do not. This project addresses that challenge by introducing an artificial intelligence-based approach to transcribe the SCC’s 2021-2022 oral arguments, enabling detailed empirical analyses of some of the speaking patterns of justices. To demonstrate potential research avenues, it explores two questions: which judges speak the most and which of the two offıcial languages do they speak? The research here shows that there are major qualitative gendered differences: judges who are men speak much more in Court than judges who are women. Similarly, there is a significant language difference: judges not from Quebec rarely spoke French in oral hearings.
Citation Information
Wallace, Simon.
"Speaking Like a Judge: Using Artificial Intelligence to Empirically Assess JudicialSpeech in Supreme Court of Canada Hearings by Language Spoken and Gender of the Speaker."
The Supreme Court Law Review: Osgoode’s Annual Constitutional Cases Conference
115.
(2024).
DOI: https://doi.org/10.60082/2563-8505.1456
https://digitalcommons.osgoode.yorku.ca/sclr/vol115/iss1/14
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 License.
References
1 Adam Liptak, "Clarence Thomas Breaks 10 Years of Silence at Supreme Court" The New York Times (February 29, 2016), online: https://www.nytimes.com/2016/03/01/us/politics/supreme-court-clarence-thomas.html.
2 Noah Bergam, Emily Allaway & Kathleen McKeown, "Legal and Political Stance Detection of SCOTUS Language", arXiv (2022), online: https://arxiv.org/pdf/2211.11724.pdf [preprint]. https://doi.org/10.18653/v1/2022.nllp-1.25
3 Adam Feldman & Rebecca D. Gill, "Power Dynamics in Supreme Court Oral Arguments: The Relationship between Gender and Justice-to-Justice Interruptions" (2019) 40:3 Justice System Journal 173. https://doi.org/10.1080/0098261X.2019.1637309
4 Tonja Jacobi & Matthew Sag, "Taking Laughter Seriously at the Supreme Court" (2019) 72 V. and L. Rev. 1423 at 1423.
5 Jack Metzler, "Most Tweetable Justice: An Empirical Study" (2015) 18 Green Bag's Micro-Symposium on Supreme Court Rankings. https://doi.org/10.2139/ssrn.2638558
6 Eve M. Ringsmuth, Matthew Sag, Timothy R. Johnson & Tonja Jacobi, "SCOTUS in the time of COVID: The evolution of justice dynamics during Oral arguments" (2023) 45:1 Law & Policy 66. https://doi.org/10.1111/lapo.12204
7 Ryan C. Black, Timothy R. Johnson & Justin Wedeking, Oral Arguments and Coalition Formation on the U.S. Supreme Court: A Deliberate Dialogue (Ann Arbor, Mich.: University of Michigan Press, 2012).
8 Lee Epstein, William M. Landes & Richard A. Posner, "Inferring the Winning Party in the Supreme Court from the Pattern of Questioning at Oral Argument" (2010) 39:2 J. Leg. Stud. 433. https://doi.org/10.1086/651511
9 Of course, there are other factors at play: the different media landscapes between the two countries, public interest in apex court hearings, and so on.
10 Victor Kuperman et al., "A lingering question addressed: Reading rate and most efficient listening rate are highly similar" (2021) 47:8 Journal of Experimental Psychology: Human Perception and Performance 1103. https://doi.org/10.1037/xhp0000932
11 Justin Grimmer, Margaret E. Roberts & Brandon M. Stewart, Text as Data: A New Framework for Machine Learning and the Social Sciences (Princeton: Princeton University Press, 2022).
12 Elizabeth A. Sheehy, Defending Battered Women on Trial: Lessons from the Transcripts (Vancouver: University of British Columbia Press, 2014) at 14.
13 Marie-Ève Hudon, Bilingualism in the Federal Courts (Library of Parliament, 2011).
14 Alexandrea Nasager, "The Supreme Court, Functional Bilingualism, and the Indigenous Candidate: Reconciling the Bench" (2019) 57 Alta. L. Rev. 797. https://doi.org/10.29173/alr2595
15 Jean-Christophe Bédard-Rubin & Tiago Rubin, "The Elusive Quest for French on the Bench: Bilingualism Scores for Canadian Supreme Court Justices, 1985-2013" (2022) 37:2 CJLS/LRCDS 249 and Jean-Christophe Bedard-Rubin & Tiago Rubin, "Assessing the Impact of Unilingualism at the Supreme Court of Canada: Panel Composition, Assertiveness, Caseload, and Deference" (2018) 55 Osgoode Hall L.J. 713.
16 Matthew Shoemaker, "Bilingualism and Bijuralism at the Supreme Court of Canada" (2012) 35:2 Canadian Parliamentary Review 30 and Sébastien Grammond & Mark Power, "Should Supreme Court Judges be Bilingual?" in Nadia Verrelli ed., Democratic Dilemma: Reforming Canada's Supreme Court (Montreal & Kingston: McGill-Queen's University Press, 2013).
17 Fees for Court Transcripts, O. Reg. 94/14.
18 See, for example, Phelim Kine, "My journey down the rabbit hole of every journalist's favorite app" POLITICO (February 16, 2022), online: https://www.politico.com/news/2022/02/16/my-journey-down-the-rabbit-hole-of-every-journalists-favorite-app-00009216.
19 Ashish Vaswani et al., "Attention Is All You Need" arXiv (2017), online: https://arxiv.org/pdf/1706.03762.pdf.
20 The main public repository for artificial intelligence and machine learning models is, at the time of writing, hosted by the private company HuggingFace. At the time of writing, HuggingFace hosted over 250,000 models available for download: "Hugging Face - The AI community building the future", online: https://huggingface.co/.
21 Emily M. Bender et al., "On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?", Association for Computing Machinery (New York, 2021). https://doi.org/10.1145/3442188.3445922
22 Avi Goldfarb & Catherine Tucker, "Digital Economics" (2019) 57:1 Journal of Economic Literature 3 at 13. https://doi.org/10.1257/jel.20171452
23 Alec Radford et al., "Robust Speech Recognition via Large-Scale Weak Supervision" (2022) [preprint].
24 Hervé Bredin et al., "pyannote.audio: neural building blocks for speaker diarization" arXiv (2019), online: https://arxiv.org/pdf/1911.01255.pdf.
25 Mirco Ravanelli et al., SpeechBrain: A General-Purpose Speech Toolkit arXiv (2021), online: https://arxiv.org/pdf/2106.04624.pdf.
26 Quarto, online: https://quarto.org/.
27 All the data were scraped from Supreme Court of Canada, "Supreme Court of Canada-Archived Webcasts", online: https://www.scc-csc.ca/case-dossier/info/webcasts-webdiffusionseng.aspx Last Modified: 2022-12-14.
28 Ziwei Ji et al., "Survey of Hallucination in Natural Language Generation" (2023) 55:12 ACM Comput Surv 248:1-248:38. https://doi.org/10.1145/3571730
29 Elizabeth A. Sheehy, Defending battered women on trial: lessons from the transcripts (Vancouver: University of British Columbia Press, 2014) at 14.
30 Jeremy Howard & Sylvain Gugger, "Fastai: A Layered API for Deep Learning" (2020) 11:2 Information 108. https://doi.org/10.3390/info11020108
31 Rules of the Supreme Court of Canada, SOR/2002-156, s. 11.
32 Supreme Court Act, R.S.C. 1985, c. S-26.
33 Emmett Macfarlane, Governing from the Bench: The Supreme Court of Canada and the Judicial Role (Vancouver: University of British Columbia Press, 2013) at 78. https://doi.org/10.59962/9780774823524
34 Emmett Macfarlane, Governing from the Bench: The Supreme Court of Canada and the Judicial Role (Vancouver: University of British Columbia Press, 2013) at 99. https://doi.org/10.59962/9780774823524
35 Within the American literature, this is a recurring question; see, as examples: Robert J. Martineau, "The Value of Appellate Oral Argument: A Challenge to the Conventional Wisdom" (1986) 72 Iowa L. Rev. 1; Warren D. Wolfson, "Oral Argument: Does It Matter?" (2001) 35 Ind. L. Rev. 451; John G. Roberts Jr., "Oral Advocacy and the Re-emergence of a Supreme Court Bar" (2005) 30:1 Journal of Supreme Court History 68; Timothy R. Johnson, Paul J. Wahlbeck & James F. Spriggs, "The Influence of Oral Arguments on the U.S. Supreme Court" (2006) 100:1 American Political Science Review 99; Michael Duvall, "When Is Oral Argument Important? A Judicial Clerk's View of the Debate" (2007) 9 J. App. Prac. & Process 121; Lawrence S. Wrightsman, Oral Arguments Before the Supreme Court: An Empirical Approach (New York: Oxford University Press, 2008).
36 Emmett Macfarlane, Governing from the Bench: The Supreme Court of Canada and the Judicial Role (Vancouver: University of British Columbia Press, 2013) at 98. https://doi.org/10.59962/9780774823524
37 Emmett Macfarlane, Governing from the Bench: The Supreme Court of Canada and the Judicial Role (Vancouver: University of British Columbia Press, 2013) at 94-95. https://doi.org/10.59962/9780774823524
38 Timothy R. Johnson, Oral Arguments and Decision Making on the United States Supreme Court (New York: State University of New York Press, 2004) at 126.
39 Timothy R. Johnson, Oral Arguments and Decision Making on the United States Supreme Court (New York: State University of New York Press, 2004) at 126.
40 Timothy R. Johnson, Oral Arguments and Decision Making on the United States Supreme Court (New York: State University of New York Press, 2004) at 97.
41 Arthur Peltomaa, "Gender and Racial Diversity of Counsel at the Supreme Court of Canada: An Empirical Study" (2021), online: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3930191. https://doi.org/10.2139/ssrn.3930191
42 Gema Lax-Martinez et al., "Expanding the World Gender-Name Dictionary: WGND 2.0" (2023) World Intellectual Property Organization (WIPO) Economic Research Working Paper Series No. 64. https://doi.org/10.2139/ssrn.4422333
43 Jean-Christophe Bedard-Rubin & Tiago Rubin, "Assessing the Impact of Unilingualism at the Supreme Court of Canada: Panel Composition, Assertiveness, Caseload, and Deference" (2018) 55 Osgoode Hall L.J. 713. https://doi.org/10.60082/2817-5069.3351
44 Office of the Commissioner for Federal Judicial Affairs Canada, "Qualifications and Assessment Criteria", online: https://www.fja-cmf.gc.ca/scc-csc/2023/qualifications-eng.html Last Modified: 2023-06-20.
45 It is worth recalling that there are methodological limitations involved in counting words. For example, some people use more filler words ("umm", "hmm", "ah", "well") than others. Do these count as words or not? And for transcription purposes, is it possible to either include or exclude all of them? I do not think so. This reminds us that transcription remains an art and, despite the real numbers produced by quantification, there is a margin of ambiguity at play here. This said, my examination of the machine transcriber's outputs shows that it rarely included filler words.
46 Supreme Court of Canada, "Supreme Court of Canada - Judges of the Court", online: https://www.scc-csc.ca/judges-juges/index-eng.aspx.
47 An intervention is defined as a chunk of speech that the machine transcriber attributes to a single judicial speaker.
48 This ratio may be somewhat misleading because different languages and grammatical structures use words at different rates. Put differently, some languages, French compared to English for instance, are "wordier" than others. To test this, I took all the official judgments of the Supreme Court of Canada issued between 2010 and 2020, which are published in both official languages, and counted the number of words. With reference to this one decade alone, all of the French judgments were 1.1 times longer than all of the equivalent English judgments.
49 Criminal Code, R.S.C. 1985, c. C-46.
50 Franco Moretti, Graphs, Maps, Trees: Abstract Models for Literary History (London: Verso, 2007) at 4.
51 Franco Moretti, "Literature, Measured" Stanford Literary Lab (April 2016), online: https://litlab.stanford.edu/LiteraryLabPamphlet12.pdf at 7.
