13.2.1 Michigan Legal Help

In 2012 the Court partnered with the State Bar to launch “Michigan Legal Help,” a website designed to provide practical, easy-to-follow information for people with civil legal problems who lack the resources to obtain an attorney (or who simply prefer to self-represent).Footnote ²² The foundational idea for Michigan Legal Help is that promoting access to justice means helping those with legal problems get the information they need to successfully navigate their problem, even if they don’t need or cannot obtain an attorney. So many legal processes require a litigant to have a specific document, filed at the right time and in the proper place. Making sure people know what they need to do and giving them the tools to do it can go a long way.

By design, Michigan Legal Help does not provide legal “advice.” Instead, it provides topical information, forms, and dozens of “do-it-yourself” tool kits for specific legal problems. For example, there are tool kits for obtaining a divorce if you have minor children, another tool kit if there are no children, and a tool kit on how to respond to a complaint for divorce. The materials are constantly updated to reflect changes in the law, court rules, and real-world trends. There is an article on “Zoom hearings” that includes information on what to expect and tips on how to connect. A legal triage system – the “Guide to Legal Help” – is built into the website and directs users to specific resources based on their responses to a series of short, easy-to-understand questions. On weekdays, a “live help” chat feature staffed by volunteer law students provides personal assistance to help people find what they need; on weekends and after hours, users can leave messages that an agent will respond to. Michigan Legal Help staff members train the student volunteers and review the chat conversations to ensure quality control for the hundred-plus live chat sessions and a similar number of overnight emails that the system sees each day. Michigan Legal Help is also affiliated with (physical) legal self-help centers located around the state, where users can get assistance navigating the site and completing and filing documents.Footnote ²³

Michigan Legal Help has enormous, unique benefits as one tool in a civil justice response system. In addition to the obvious, Michigan Legal Help (and similar web-based programs) generate data that provides a window into current trends and needs, which can then be used to inform other access-to-justice efforts. It is agile and iterative; the tool kits, forms, and processes are updated and improved based on user experience. And, of course, it is infinitely scalable. Indeed, Michigan Legal Help is maintained by a staff of only eight people, including four attorneys. In 2020, it had nearly 3.5 million sessions, 2.7 million users, and 6.6 million page views. Nearly 153,000 people used its resources to complete legal forms. Even accounting for the “pandemic spike” of users in spring of 2020, most of whom accessed information on unemployment benefits, the website has seen year-to-year increases since its rollout in 2012.

13.2.2 ADR/ODR

Michigan has promoted court-supported alternative dispute resolution as another solution for some legal problems and leveraged technology to support it. Beginning in 2014, a first-in-the-nation pilot program introduced online ticket resolution in two district courts. Using a system pioneered by University of Michigan Professor (and fellow volume contributor) J.J. Prescott, the program empowers residents to contest traffic tickets online. It is now offered in more than thirty district courts around the state.Footnote ²⁴ Traditionally, ticketed drivers wanting to seek a dismissal or negotiate a reduction in penalties had to show up to court in-person; the only “remote” option was to accept responsibility and pay the ticket as issued. Through the online system, users can request a review of their citation, submit a statement, and upload photos or other information. According to a study of outcomes in six Michigan courts, this online ticket-review system reduces court staff time by 80 percent, and cases are resolved in an average of two weeks instead of seven. Most importantly, 39 percent of users reported that they would not have been able to come to the court in person.Footnote ²⁵ Courts nationwide are now using this platform, and the service has been a critical tool in helping courts respond to the pandemic.Footnote ²⁶

In 2019, the Michigan Supreme Court used this same platform to launch “MI-Resolve” in seventeen Michigan counties.Footnote ²⁷ MI-Resolve is a fully online, asynchronous, dispute resolution program designed to resolve minor disputes, such as small claims, debt collection, noneviction landlord-tenant matters (like the return of a security deposit), and neighborhood disputes. Unlike traditional court-referred or party-selected mediation, which is usually done in-person and involves a mediator whose services are paid for by the parties, MI-Resolve is free for the participants and can be done with or without a mediator. The program is confidential and can be used regardless of whether there is active litigation. If the participants agree to a particular resolution, the system will produce a binding agreement and also generate any necessary forms for filing in court.

Michigan has led the way in online dispute resolution (ODR) programs like MI-Resolve, but experiences in other jurisdictions also show that ODR can provide an efficient and affordable way of resolving cases that do not involve complicated legal questions and where the amount at issue is modest. In 2020, the Joint Technology Committee of the Conference of State Court Administrators (COSCA), the National Association for Court Management (NACM), and the National Center for State Courts (NCSC) published case studies from ODR programs across the nation.Footnote ²⁸ In Franklin County, Ohio, use of ODR for city income tax cases led to fewer default judgments and a more even distribution of positive case dispositions regardless of socioeconomic factors and race.Footnote ²⁹ Utah used ODR as part of a total restructuring of its court system’s process for resolving small claims. The program has led to fewer hearings and faster dispositions in cases that do end up “in court.”Footnote ³⁰ In Connecticut, ODR has been used to create an online ticket review program for traffic court. Most ticket recipients chose to “opt into” the program, which has shortened the number of days from citation to adjudication.Footnote ³¹

13.2.3 SCAO’s IT Solutions for Local Courts and Filers Depend on Agile Culture

There have also been significant efforts in Michigan to transform the user experience of “traditional” court with technology solutions. Even without the benefit of a unified court system, the SCAO’s technology division has partnered with private sector firms to speed up development and implementation of IT solutions that connect Michigan’s courts with each other and with users, while improving the experience for both. If we could successfully provide excellent IT products and service to trial courts, we could make progress toward voluntary technological unification – carrots. But it means a fundamental transformation of our approach to IT. We are all familiar with government IT projects that promise to revolutionize process but fall short of expectations. Instead, we consulted private sector firms to bring agile culture to our public sector operation.

SCAO adopted Menlo Innovations’ “high-tech anthropology.”Footnote ³² Just as smartphone apps undergo continuous incremental updates that don’t involve any inconvenience or loss of service, so too our court technology solutions improve as users provide feedback. We visit local courts and watch clerks, judges, and court users to see how they used our products and solicit their ideas to improve efficiency and service to the public. Agility is unusual in government IT and contrasts with the previous approach of taking down systems over a holiday weekend and launching a new platform on Monday morning. It limits risk, engages users, and steadily provides value-add, allowing for constant updates and changes with each iteration.

The agile process has been critical as we expand our case management product (MiCourt) to judges and court administrators who can be resistant to change. By engaging users for their input throughout the process, our developers benefit from on-the-ground knowledge and experience, and buy-in eases change. As a result, we have provided new features users asked for, including automatic calendaring, electronic docket boards, public dockets, case search, and text-message reminders of upcoming court dates and payments due. We are also using the agile approach to launch a new public website this spring that emphasizes mobile device use, which provides better and equal access to information for all users, as well as security, speed, and improved information architecture.Footnote ³³

The success of MiCOURT can be measured both by the satisfaction of court users, but also by the expansion of the case management system to courts throughout the state as they learn about the unique, value-added features not provided by their current systems. And, as the list of MiCOURT courts grows, system-wide change becomes easier and more efficient.Footnote ³⁴ A single case management system – our ultimate goal – creates enormous potential for statewide improvement. It will allow for real-time access to comprehensive court data, statewide process updates, and a simpler system for court users. That’s the holy grail.

To be sure, the picture is not all rosy. Michigan does not yet have statewide e-filing, and it will be another few years before that project is finished. Locally funded courts and locally selected IT systems mean that a statewide, online, case-filing system that aims to provide a consistent user experience for all filers in the state, regardless of jurisdiction and case type, is a heavy lift. Several counties had homegrown e-filing systems and electronic document management systems, while others continue to be limited to paper files. In other words, we are getting there, but it’s a slog.

13.2.4 Justice for All Taskforce

The State Bar of Michigan (MSBF), the Bar’s access-to-justice arm, and other access-to-justice oriented entities (like legal aid and law school clinics) traditionally collaborated through informal networks, not a formal structure. As a result, there had been no comprehensive assessment of the civil justice gaps and needs across the state, nor a measure of how the various programs and innovations were meeting the need.

In May 2019, the Michigan Supreme Court formed a task force to change that by conducting a full assessment of the current state of our civil justice system. Partnering with the MSBF and Michigan Legal Help, and with funding from the National Center for State Courts (NCSC), the task force set out to do a comprehensive statewide inventory of resources for civil legal problems, as well as identify the gaps in those resources. The task force was charged with developing a strategic plan to address the legal needs of every Michigander.

***

Heading into 2020, the task force felt that access to justice was a priority in Michigan and that the community of lawyers, judges, administrators, and technologists working on it were looking for innovative solutions to civil justice problems. Despite structural barriers, progress was being made. We would not have declared the problem solved, but we were optimistic about what we might accomplish. We were working diligently on pockets of innovation, as well as an overall strategy. And yet, transformational change was not really on our radar.Footnote ³⁵

14.1.1 PACER Basics

PACERFootnote ¹⁰ began as a dial-up-based experiment in 1988 by the Judicial Conference,Footnote ¹¹ at a time when the public had to go to physical courthouses for case information.Footnote ¹² Because PACER isn’t free to operate, and Congress doesn’t appropriate funds for it,Footnote ¹³ the Judicial Conference has long charged fees for PACER use. Since 1998, PACER has been accessible through World Wide Web connections; the per-page fee for downloaded information is now $0.10.Footnote ¹⁴

Such fees make large case-info downloads costly. By policy choice, the Judicial Conference limits fees in some ways; today, charges are capped at $3.00 per document.Footnote ¹⁵ In addition, PACER waives fees for any quarter in which a user incurs under $30 in charges.Footnote ¹⁶ Parties and their attorneys receive a free copy of any electronically filed document “if receipt is required by law or directed by the filer.”Footnote ¹⁷ Judicial opinions are supposedly free for anyone’s download,Footnote ¹⁸ although that may not always be true in practice.Footnote ¹⁹

Unfortunately, these policies do little for scholars who must download many documents for their work. The Judicial Conference does allow courts to exempt some persons from fees, including “individual researchers associated with educational institutions,”Footnote ²⁰ and exemptions have facilitated important research, but much less than broader scholarly access would allow.

14.1.2 PACER Serves Up Case Records in a Clunky Way

What PACER allows users to access remains far too limited. Critically, as the Free Law Project notes: “There is no document-level search.”Footnote ²¹ And although PACER users can search for litigants’ names or retrieve a case using a known docket number, it is impossible to search, say, for some version of “all cases that have at least one docket entry with the string ‘FAILURE TO STATE A CLAIM’ or ‘QUALIFIED IMMUNITY.’” The Free Law Project observes: “This makes the system utterly unusable for general purpose research.”Footnote ²² Anyone seeking to obtain data from many cases of a type – those with a Rule 12(b)(6) motion, or involving qualified immunity – must pay for enormous numbers of documents. This drains research budgets for no good reason.

14.1.3 Financial Facts for PACER

The Judiciary receives no funding from Congress for PACER, and the CM/ECF backbone is supported entirely by PACER fees. Each year these fees bring in around $150 million.Footnote ²³ To put it mildly, that greatly exceeds the costs to the Judiciary of operating PACER, which one source recently put at $3 million.Footnote ²⁴

Who uses PACER? The Judiciary emphasizes PACER’s enormous user base – more than 3 million user accounts, by 2020.Footnote ²⁵ As of 2015, 75 percent of PACER usage was unpaid due to fee waivers (perhaps principally the $30/quarter free-usage allotment), even as 75 percent of PACER revenues came from fees paid by just 1 percent of account holders.Footnote ²⁶ Although I am aware of no precise data, it’s easy to guess that the biggest paying users include the big three electronic legal data vendors – Westlaw, LexisNexis, and BLAW – and bankruptcy data providers that service claimants’ data needs. As of 2015, 63 percent of PACER usage was undertaken by law firms, 12 percent by litigants themselves, 10 percent by commercial users, and only 3 percent by academics.Footnote ²⁷ That academics use PACER so little is likely a testament to the fact that the large data pulls we need to do serious quantitative research would cost so much that most scholars either never start or rapidly abandon plans to study federal litigation using PACER data; see discussion in Sections 14.1.4 and 14.2.2 for more.

These facts paint a simple picture. PACER’s paywall brings some money to the Judiciary, allowing Congress to fund it a bit less. The wall doesn’t stop the biggest legal data vendors from downloading a lot of data, which they make available to subscribers who pay for platform access. The wall likely doesn’t stop litigants in most individual cases, at least not with respect to their own cases. Nor does it likely do much to keep interested small-hobbyists out.Footnote ²⁸ But the PACER paywall sure does screen out scholars.

14.1.4 PACER Is Prohibitively Expensive for Scholars

To study litigation, the judicial system, and how they interact with social phenomena, scholars need large amounts of data. Because of PACER’s high per-page download fees, even projects involving relatively modest data can be very expensive.Footnote ²⁹ To wit, a recent PACER fee waiver request estimates that to download only the docket sheets for just two years’ worth of FOIA cases would cost nearly $4,000.Footnote ³⁰ And consider a commentary recently published in Science,Footnote ³¹ which presents information on thousands of cases in which a party sought in forma pauperis status and shows considerable variation within districts in the likelihood such a waiver would be granted. The authors downloaded every docket report for all cases filed in 2016, at a cost they describe as “more than $100,000.”Footnote ³² That won’t become a common research strategy.

Although some research is conducted with second-best data sources (see section14.2.2, infra), I’ve had numerous conversations over the years with scholars who abandoned research ideas they thought promising because PACER’s prohibitive costs made it too costly to push forward.

14.2.1 On Principle, Court Operations Should Be Presumptively Open to the Public

It’s a basic principle of a democratic society that government’s operations should be open to public scrutiny. Otherwise, the citizenry cannot tell whether government is operating justly, speedily, or inexpensively.Footnote ³⁴ Even appropriate scrutiny of one governmental branch by another can be limited without sufficient openness.

Constitutional and federal common law bases for public rights to observe judicial proceedings are most developed in the context of criminal trials,Footnote ³⁵ but these are of course only one spot in the landscape of judicial proceedings. Of the nearly 80,000 federal criminal cases filed in 2018, only about 2 percent went to trial,Footnote ³⁶ a number that isn’t much different for civil cases. With so few trials, pretrial proceedings delineate the fairness and future incentive effects of federal litigation. Accordingly, whatever interests of publicness inhere in open trial proceedings also extend to pretrial ones, too.

Professor Peter W. Martin has described five purposes for public access to litigation-related documents: (1) assuring fairness – both actual and perceived – of proceedings, (2) ensuring that the public’s ability to assess and criticize judicial performance can be done on an informed basis, (3) educating the public about how litigation works, and – hopefully – inspiring confidence as a result, (4) enabling the “check[ing]” of the judicial system by the public, and (5) “[p]roviding an outlet for community ‘concern, hostility, and emotion’ in cases that are in the public eye”.Footnote ³⁷ At least the first four of these are implicated in the Judiciary’s refusal to allow affordable public access to PACER for scholarly or journalistic research.Footnote ³⁸

In sum, as House Judiciary Committee Chairman Jerrold Nadler recently stated: “It is a disservice that in today’s digital age, the public’s access to public records in public proceedings is so resource-intensive and burdensome …. It is indefensible that the public must pay fees – and unjustifiably high fees, at that – to know what is happening in their own courts.”Footnote ³⁹

14.2.2 Important Research Could Be Done with Freer Access, and for Free to the Judicial System

This section provides a brief, ad hoc review of quantitative empirical work that has been done using federal court data.Footnote ⁴⁰

The Limits of Non-PACER Data Sources. Even before PACER, there was much quantitative empirical research published using federal court litigation data. A chief alternate data source is the Federal Judicial Center’s Integrated Database (IDB), “by far the most prominent” of data sources used for statistical research by legal scholars.Footnote ⁴¹ Scholarly work using the IDB has spanned the gamut of topic areas for years.Footnote ⁴² But the IDB has its flaws.Footnote ⁴³

So, when FJC researchers set out to study summary judgment practice at the request of the Advisory Committee on Civil Rules in the wake of the Supreme Court’s 1986 summary judgment trilogy, these researchers analyzed docket sheets randomly selected in three district courts.Footnote ⁴⁴ Subsequent FJC studies of summary judgment practice also used docket information.Footnote ⁴⁵ This is particularly notable because summary judgment dispositions are the sort of issue of interest not just to scholars or elected officials, but also to the Judiciary. In fact, the sort of data the FJC, and Professor Burbank,Footnote ⁴⁶ has used to study summary judgment is the primary source of information for judges doing rule-making. Yet as intensive as this work was, it typically involved studying few districts, or even just one at a time.

The CM/ECF system now contains all data necessary to enable nationwide studies, and by 2008 (if not sooner), the FJC was doing so in response to Advisory Committee requests.Footnote ⁴⁷ Due to PACER’s paywall, though, outside researchers cannot generally do likewise. That means the Judiciary, and society more widely, are partly subject to the limits of FJC’s resources, however impressive its ingenuity, in addressing matters of interest to rule-making.

Another area of great interest in which the IDB is limited is the pleading standard, whose alteration in Bell Atlantic Corp. v. TwomblyFootnote ⁴⁸ and Ashcroft v. IqbalFootnote ⁴⁹ was arguably the top focus of procedure scholars for a period of several years. As Professor Burbank wrote even before those cases, the IDB’s limitations make it difficult, at best, to determine anything specific about Rule 12(b)(6) and 12(c) motions decided under the Rule 8(a)(2) pleading standard, including even “the number of such motions filed or decided.”Footnote ⁵⁰

There are still other data sources for scholars studying the federal legal system, all with key limitations. For example, the US Sentencing Commission posts defendant-level data on sentences in federal criminal cases.Footnote ⁵¹ Interesting research has been conducted using these data, for example, on racial disparities and the federal sentencing guidelines.Footnote ⁵² But they obviously concern only criminal sentences. Judicial opinion databases are another source of data; in some cases these can be useful,Footnote ⁵³ though it is well known that for many others this approach often brings bias given that only some opinions are published.Footnote ⁵⁴ Another alternative is the Court Listener RECAP archive,Footnote ⁵⁵ hosted by the Free Law Project, which contains “millions of PACER documents and dockets”Footnote ⁵⁶ and grows over time as more users use RECAP, even as users are able to avoid paying for documents that RECAP already has.Footnote ⁵⁷ As great a resource as RECAP is for documents that have been stored, there is no general way to measure how representative RECAP’s sample of documents and docket reports is.Footnote ⁵⁸

PACER’s Unrealized Potential. This brings me to work using all of PACER, or representative samples. Bankruptcy has been a prolific substantive area for empirical study, due in part to the enterprising efforts of Professors Teresa Sullivan, Elizabeth Warren, and Jay Westbrook. The challenges surmounted in this work are described engagingly by Professor Lynn LoPucki. In brief, the professors flew themselves and some photocopiers around the country, copied court files in the clerks’ offices, and sent the copies to Austin, Texas, where they coded the information they would use – even though copy machines in the clerks’ offices were unused after hours.Footnote ⁵⁹ In 2001, after PACER’s advent, the same researchers were able to secure judicial permission to hire “moonlighting assistant clerks to photocopy the files and ship the copies,” because that allowed them to avoid PACER’s per-page download fees.Footnote ⁶⁰ As Professor LoPucki writes, “With the political constraints removed by judges, the 1981 technology – photocopying in the clerks’ offices – was cheaper than the 2001 technology – internet PACER – with its one-page-at-a-time political restriction.”Footnote ⁶¹ In sum, at enormous and duplicative cost, these researchers were able to obtain substantial data on bankruptcy cases. They used these data to publish numerous studies about an issue of great public policy concern.Footnote ⁶² Subsequent research on consumer bankruptcy reform was able to use both PACER fee waiversFootnote ⁶³ – obviously a welcome development – and help from AACER, the Automated Access to Court Electronic Records, a service that collects bankruptcy court data on a daily basis and repackages it commercially.Footnote ⁶⁴ AACER must itself pay the Judiciary to download data that are already in the PACER system, but the facts of the bankruptcy system evidently make it possible for market forces to sustain AACER’s more rational approach to data distribution. There is little prospect for such a market solution to PACER’s paywall for nonbankruptcy cases (and would-be users must still pay AACER).

Other scholarly work has been published using the fruits of PACER fee waivers. Examples include work by Professor David Engstrom,Footnote ⁶⁵ Professor Gillian Hadfield,Footnote ⁶⁶ and Professor Alexander Reinert.Footnote ⁶⁷ But PACER fee waivers are at most a partial solution to the PACER paywall, and a highly imperfect one at that. Even though there is now a centralized process for requesting a fee waiver,Footnote ⁶⁸ the decision over whether to grant requested waivers is up to individual district courts.Footnote ⁶⁹ This can extend the time and uncertainty related to project feasibility. It results from the fact that the CM/ECF system underlying PACER is really an amalgamation of individual systems, each run by an individual court as it sees fit. It is hard to think of good reasons why one court would deny fee waivers for the same scholarly research another approves, yet that happens.Footnote ⁷⁰ Further, my conversations with other scholars indicate the possibility of inconsistent treatment. For example, although Professor Lawless and his coauthors received fee waivers from all but the Southern District of Texas,Footnote ⁷¹ one scholar at a workshop on federal court data access that I organized in 2015Footnote ⁷² reported being unable to receive fee waivers from more than thirty-five of the ninety-four district courts.Footnote ⁷³ Study of the Supreme Court’s alteration of the pleading standard in Twombly and Iqbal also shows PACER’s potential, and the limitations of its alternatives.Footnote ⁷⁴

Another anecdote involves a July 2020 request by Plymouth State University Professors Chantalle Forgues and Daniel Lee for PACER fee waivers so they could download docket reports, complaints, and petitions for all New England district courts.Footnote ⁷⁵ The scholars planned to use computer programs to target a subset of cases as classified using Nature of Suit codes.Footnote ⁷⁶ All district courts petitioned granted the request except for the District of Massachusetts.Footnote ⁷⁷ Events over several months culminated in a court order denying the professors’ request,Footnote ⁷⁸ because of the Chief Judge’s belief that the researchers planned to use software that, “according to our IT security personnel, would present an unacceptable degree of risk to our system.”Footnote ⁷⁹ Professors Forgues and Lee explained that “the script that petitioners propose is used everyday, including on weekends, to download files from PACER. The script at issue is used by big law firms and larger universities on a regular basis, and has been used on the Court’s system consistently for over seven (7) years.”Footnote ⁸⁰ A motion seeking reconsideration remains pending.Footnote ⁸¹

The upshot of all of this is two-fold. First, bulk downloads are available, without limit, to users who can afford to pay PACER fees, even as federal courts create situations in which researchers who cannot afford PACER fees are out of luck. More generally, the PACER system, and judicial policies and practices around it, stymie what would otherwise be PACER’s staggering research potential. PACER contains over a billion documents,Footnote ⁸² and that number grows daily. Many other federal agencies have made their data available, with privacy restrictions, allowing the public to benefit from enormous amounts of professional-quality research at no cost to the federal government.Footnote ⁸³ The many examples of research noted above – from enterprising bankruptcy scholarship to illuminating work on summary judgment and pleading to work on trials – hint at the possibilities presented by piercing PACER’s paywall. This research, much of it using inferior or limited sources of data, is a drop in the bucket compared to what we could have with free PACER.

14.2.3 What about Privacy?

A natural basis for concern about free PACER involves privacy of both litigants and witnesses. This essay is not the place to fully develop either the bases for such concerns, which are real, or the responses to them. But engaging these topics can allay certain aspects of these concerns while informing further inquiry as to others.

A first concern involves risks in the criminal justice system. Stephen Schultze chronicles one example:

What DOJ wanted to preserve is what is sometimes called “practical obscurity” – “print-era barriers to access to sensitive criminal information.”Footnote ⁸⁵ After describing how federal judges engaged these questions, including at the level of the Judicial Conference’s Court Administration and Case Management Committee, Schultze observes that “by the time whosarat.com appeared in newspaper headlines, the Judicial Conference had codified its approach to sensitive information in electronic court records,” with the result that the Justice Department “could protect sensitive plea agreements by sealing or redacting the documents, but it could not rely on the Judiciary to recreate anachronistic approximations of practical obscurity.”Footnote ⁸⁶

If this balance between privacy concerns and public access is good enough for those who might be at risk of physical violence, it is difficult to see why it should not also be good enough for others in the justice system. This is especially true in the civil context, where concerns related to personal embarrassment or trade secret interests can be handled on a case-by-case basis. Just as the Freedom of Information Act contains exceptions for trade secrets and certain other delicate information,Footnote ⁸⁷ so, too, may federal courts allow litigants to file documents under seal in appropriate circumstances.

The privacy case for limiting access to PACER records en masse seems especially weak given that the very same records are already available electronically. To the extent that practical obscurity exists now, it is thus limited only by PACER fees. The paywall does not protect anyone from PACER’s large commercial buyers, including, for example, bankruptcy-related services such as AACER. But these fees are high enough to keep out scholars, most of whom have little to no interest in individual information – the point of much scholarly research being to generalize rather than investigate details of unknown persons. It also includes journalists, which places PACER in tension with the First Amendment values that animate the constitutional basis for open court records that the Supreme Court found in Richmond Newspapers, Inc. v. Virginia.Footnote ⁸⁸

14.3.1 Free PACER and Replace Its Fees with General Revenue

Encouragingly, in December 2021 the Open Courts Act of 2021 was reported out of the Senate Judiciary Committee with bipartisan support.Footnote ⁹³ This bill would require all court documents to be presumptively freely accessible to the public within three years after enactment.Footnote ⁹⁴ The bill also calls for modernization of PACER’s feature set,Footnote ⁹⁵ and a similar bill is pending in the US House as of this writing.Footnote ⁹⁶ Although the costs of modernization might be offset partly by congressional appropriations,Footnote ⁹⁷ future funding would need to come from fees paid by federal agencies for their PACER use, based on inflation-adjusted fees agencies paid in 2018,Footnote ⁹⁸ and possibly also increases in filing fees on parties.Footnote ⁹⁹

I have doubts about this approach to funding. First, I have not been able to find a breakdown of PACER fees by source, but logic indicates that fees paid by federal agencies are only part of the revenue from PACER. Thus, without increasing filing fees, the Judiciary would absorb at least some revenue loss. My goal is to free PACER, not cut courts. Second, if the Judiciary exercised the authority the Open Courts Act provides to raise filing fees, that burden would fall on litigants. My goal is to free PACER, not raise additional financial barriers to courts.

The better course is for Congress to simply (1) eliminate the Judiciary’s discretion to charge PACER fees and (2) increase general revenue funds for the Judiciary. Congress could hold the Judiciary harmless, replacing all PACER paywall revenues, for just $150 million.Footnote ¹⁰⁰

Although $150 million would be a lot of money to have for yourself, it’s a teeny-tiny little droplet in congressional terms. By way of comparison, Americans spend roughly twenty-five times as much on wild bird food as PACER fees brought in.Footnote ¹⁰¹ And overall congressional funding to the Judiciary in 2020 was fifty times what the PACER paywall brought in.Footnote ¹⁰²

A situation in which we have hamstrung public inquiry into the functioning of the federal litigation system over a literal rounding error may not be funny, but it really is silly. Let’s stop it already. Congress should just cut an annual check to the Judiciary, allowing the public to better understand – and better inform Congress – how one of the three branches of our government functions.Footnote ¹⁰³ It wouldn’t even make a dent in the monthly auction of Treasury debt, and it would allow the Judiciary to free PACER and lose nothing.Footnote ¹⁰⁴

14.3.2 Infrastructure Improvements

As I discuss above, one of the real problems with PACER is that it provides very limited search capacity. The House bill introduced in the last Congress had modernization features directed at this problem,Footnote ¹⁰⁵ and the Senate bill spoke generally of improvements in “data accessibility … and performance.”Footnote ¹⁰⁶ Not knowing anything about PACER’s back-end design and capabilities, I don’t know whether its lack of a docket-entry or document text search function is a bug or is deliberate. But there are surely many nonprofit organizations that would and could host the data. As noted, the Internet Archive has already offered to do so for free, and to do so with considerably better functionality than PACER currently has.

There are other infrastructure improvements that the Judiciary could make. Some require more burden than others. Less demanding would be more refined docketing, for example, allowing docket coding for motions to dismiss according to the basis for the motion. On the high end would be redesigned forms such as the civil cover sheet, requiring more and/or better information from litigants.

A group of professors recently discussed many of the pathologies I have already described, in Science magazine.Footnote ¹⁰⁷ They advocate a “collaborative research agenda to empower the public to access and analyze court records.”Footnote ¹⁰⁸ This agenda includes better organization of the data because of the fact that court records tend to include unstructured text. The professors advocate developing

So there are surely a range of other infrastructure improvements worth considering. My purpose here is only to suggest that more about PACER than just the paywall can be improved.

14.3.3 The Judiciary Should Adopt an Attitude of Openness

Certainly there are many judges who embrace openness. Judges have approved many, perhaps even most, PACER fee waivers that scholars have actually submitted to the courts. An amicus brief by retired judges in the ongoing PACER class-action litigation indicates laudable support for public access to court data.Footnote ¹¹⁰ Over the years, I have had numerous conversations with professionals at the FJC and AO that indicate a range of support between partial and complete for providing more data access.

At the same time, it is hard not to read a message of opposition from some of the Judiciary’s choices, including setting PACER fees at an unnecessarily high level, opposition to legislation that would increase PACER access (including by circulating boilerplate “talking points” for federal judges to use in lobbying members of Congress),Footnote ¹¹¹ and anecdotal evidence that some judges may reject openness as a categorical matter.Footnote ¹¹²

My own experience is at least partially consistent with that picture, and it suggests that this rejection exists not only among particular judges, but also as an institutional fact. In 2015 I attended two workshops about access to federal court data. The first was one I organized at Penn Law, which was funded by the National Science Foundation,Footnote ¹¹³ and the second was the Federal Courts Civil Data Project Roundtable, which was held in Washington, DC, and hosted by the ABA Standing Committee on the American Judicial System and the Duke Law Center for Judicial Studies. Participants at one or both of these workshops included federal judges; academics who work with federal data; staff researchers and/or attorneys from federal agencies including the Federal Judicial Center, the AO, the Administrative Conference of the United States; a representative from the National Center for State Courts; and Michael Lissner, executive director of the Free Law Project. I have limited space to describe my impressions of the views of the Judiciary’s representatives at these events. There were a variety of views, but it seems clear that AO representatives harbor skepticism about the economic, bureaucratic, and technological feasibility of opening up PACER (though the multi-court fee waiver exemption process that now exists occurred not too long after these events).Footnote ¹¹⁴

In addition, at the ABA event, one Article III judge spoke forcefully against additional PACER access. This judge, whom I know from prior communications to be a devoted public servant, pointed to the example of a news story about a colleague that the judge thought was both unfair and professionally embarrassing. The judge at the conference expressed sincere and profound concern that the judiciary would be unduly pressured in this and other ways by the public availability of additional data that could be searched and filtered easily.

There is some evidence that the Civil Justice Reform Act distorts how judges approach their dockets, perhaps to avoid embarrassment.Footnote ¹¹⁵ I’m an economist who believes that people generally respond to incentives, so I’m not surprised. Even so, Article III judges have substantial constitutional protections precisely to allow them to buck public pressure and embarrassment in the service of judicial independence. Most obviously, they have life tenure, and their salaries cannot be reduced.Footnote ¹¹⁶ Their jobs carry considerable prestige, and no small amount of power; Chief Justice Stone once remarked, “the only protection against unwise decisions, and even judicial usurpation, is careful scrutiny of their action and fearless comment upon it.”Footnote ¹¹⁷ I oppose unfair treatment of judges as much as anyone else, but it should go without saying that unfair treatment is not the same as informed public scrutiny. Citizens should have confidence that the federal judiciary will not be deterred from doing its job with integrity by the availability of such scrutiny, and for this reason the Judiciary should welcome rather than avoid it.Footnote ¹¹⁸

Judges play an important and powerful role in our democracy, and citizens have a right to the fruits of studying how the courts do their, and thus our, business. Any defensible vision for the future of public access to court records should include a decision by the Judiciary to switch gears and foster an institutional attitude of openness toward the public.

15.1.1 The Stubbornness of High Legal Fees

The effort required to represent clients explains in large part why legal fees are expensive. Understanding their clients’ circumstances requires effort and time; so too does understanding the relevant legal authority. And the economies of scale are limited, as each client is unique. As lawyers gain experience, they are able to work more efficiently, accomplishing more over the same unit of time than less-experienced lawyers. To reflect this experience, however, experienced lawyers often are more expensive.Footnote ⁵

In The Cost Disease, William Baumol and William Bowen explain how certain jobs rise in compensation, even in the absence of an increase in labor productivity.Footnote ⁶ The authors characterize two types of jobs: (1) workers who become more productive over time, in large part due to technology and innovation and (2) workers largely immune to these advances. Auto workers provide an example of the former: because technology allows each worker to contribute more efficiently to the production of a car, her hourly wages have plateaued, even declined, as machines have come to perform more of the assembly process. For the latter, a string quartet: musicians devote the same amount of time to perform Beethoven today as did their predecessors. Compensation for workers in these technology-resistant fields rises if demand for these services remains constant and employers have to compete for these workers.

Until recently, lawyers were more akin to classical musicians than auto workers. The effort required to represent clients over time has remained stubbornly consistent. While digital libraries have enabled lawyers to identify relevant legal materials more efficiently, lawyers still have to process them.Footnote ⁷ Lawyers historically used leverage to contain legal fees: partners delegating the straightforward and labor-intensive work to less-expensive associates.Footnote ⁸ But the rate-limiting factor remains human effort, which – when unaffected by gains in productivity – becomes increasingly expensive.

The legal profession licensure requirements also create barriers to entry. The practice of law requires attending law school and passing the bar, reducing the supply of lawyers.Footnote ⁹ Over time, non-lawyers have engaged in certain legal work, under the authority of lawyers, and in some instances entirely on their own.Footnote ¹⁰ But even as non-lawyers reduce legal fees, human labor remains the rate-limiting factor.

15.1.2 Three Stages of Lawyering

For all these reasons, the legal profession has enjoyed an extended period operating as a well-paying guild.Footnote ¹¹ Until recently, the fundamental nature of legal practice remained unchanged, even as the profession adopted new technologies. During that time, the profession has evolved across three stages: from analog to digital to computational.Footnote ¹² It is only the third and latest transition that seems poised to force the legal profession to fundamentally change its approach, both conceptual and financial.Footnote ¹³ I briefly summarize these stages.

Analog. This stage represents the era where lawyers relied on hard copies of legal references – for example, case reporters, treatises, regulations, statutes, restatements – in their course of their legal work. This era’s defining feature is that the lawyers themselves were the sole mechanism to process these references and to apply them to their clients’ situation. Humans were the rate-limiting factor in the speed and efficacy of legal representation. Although varying in ability, lawyers performed tasks primarily as a linear function of the hours worked. Accordingly, larger law firms were able to take on more complex work because they could deploy larger numbers of lawyers onto a legal problem.

Digital. This stage coincided with the emergence of two technological developments during the 1980s: the personal computer and the creation of electronic legal libraries. The personal computer enabled greater efficiency. Lawyers who previously used typewriters found that an electronic format allowed them to write, edit, and share documents more easily. The advent of electronic legal libraries – notably Westlaw and LexisNexis – enabled lawyers to conduct their legal research through a dedicated terminal and, soon thereafter, on their personal computer. At a minimum, lawyers could access any decision, statute, or regulation by looking up its citation. Even more helpfully, lawyers could use keywords to identify the universe of relevant materials. The advent of electronic legal libraries untethered lawyers from physical reference materials, while simultaneously allowing them to digitally memorialize their work. These advances improved the rate of production, but, as with the analog stage, lawyer productivity remained a function of hours worked.

Computational. This latest stage coincided with the emergence of AI and its application to law. Early developments during this stage are really extensions of the analog era, made possible because of the increased digitization of legal records. Discovery is one such example. During this pre-trial process, lawyers ask opposing counsel for materials – usually in the form of documents – that they believe could be legally relevant. The process historically required lawyers to read each document. Digitization expedites the process by allowing lawyers to electronically search for relevant words or phrases. A plethora of e-discovery start-ups exists to assist in this process.Footnote ¹⁴ What once required a small army of lawyers now requires a single lawyer and a few lines of code.

As its name suggests, the computational stage refers to the ability of machines (computers) to go beyond identifying relevant legal references, drawing upon these references, in conjunction with specific information about a client case, to generate legal analysis. It uses two methods of AI: machine learning (ML) and natural language processing (NLP), described briefly below.

ML statistically analyzes structured data to predict outcomes.Footnote ¹⁵ This process draws upon regression analysis, commonly used in both STEM and social sciences, as a means to test hypotheses by looking at data generated by observable phenomena. The difference lies in the objective: Rather than use regressions to explain outcomes, ML seeks to predict them. ML in legal technology involves digitizing legal information. Information from legal references – notably cases – can be expressed as pieces of data. These variables can be binomial in nature (is something X or Y); multinomial (is something A, B, C, D, etc.); or continuous (e.g., earnings, damages).

To illustrate: Whether a worker is an employee or independent contractorFootnote ¹⁶ frequently arises in tax and employment law, determining the rights and obligations of the worker. In most jurisdictions, courts weigh several factors – for example, the amount of training, who sets the hours, the right to delegate work – in making this determination. It is possible to code published decisions into a dataset that captures how courts weigh these factors in reaching their decisions. ML can predict, given a specific set of facts, how a court would decide this matter, given existing precedent. Transforming written text to bytes of data can be labor-intensive, particularly when done manually. Once transformed, however, scaling this data is trivially small: A million people can draw upon this data just as easily as a single person.

NLP, which sits at the intersection of computer science and linguistics, is designed to understand and interpret human language, whether written or spoken. Whereas ML analyzes structured data – that is, that which could be captured in a spreadsheet – NLP is capable of working with unstructured data,Footnote ¹⁷ such as the text of a memo or an opinion. Both NLP and ML can identify patterns or relationships in data faster and more comprehensively than humans. The relationship can involve individual words, phrases, or even the proximity of words in relation to other words. NLP allows for interpretation beyond words qua words, extending to words in relation to other words, even when those words are not juxtaposed.

NLP already has applications in both the consumer and legal context. For example, firms can use NLP to more effectively evaluate customer feedback across multiple formats – for example, email, surveys, conversations from call centers – to identify recurring or common problems, even as customers express themselves in a myriad of ways.Footnote ¹⁸ Within law, firms can use NLP to evaluate and identify relevant provisions of a contract and their meaning.Footnote ¹⁹ This technology is helpful, given the human time, expense, and error that often results when done manually. Even with the promise of comprehending unstructured data, NLP performs better the more structured the text.

15.1.3 Legal Ethics

As the number of applications grows, the rate-limiting principle will not be technological, but ethical. It is helpful to distinguish between the ethical and the legal ethical considerations of AI. The two overlap but remain distinct. The general ethical debate speaks to the ethical dilemmas that technology poses, which can inform whether the technology should even be built, and if built, how to do it in accordance with ethical considerations. Legal ethics is arguably more restrictive, looking to how the profession uses technology in accordance with its own fiduciary and ethical obligations.

Legal technology raises several ethical concerns.Footnote ²⁰ One is privacy, the blurring of boundaries of attorney-client privilege. Many of these technologies rest in the cloud, and therefore potentially allow third parties illegal access to privileged information. The second is unlawful discrimination when algorithmic tools are used in criminalFootnote ²¹ and employment contexts. The third is the unauthorized practice of law, particularly as technologies advance and can perform tasks that draw closer to legal advice.Footnote ²² Finally, the evolution of legal technology raises issues of access to justice, because much of the technology is geared toward the elite private bar, not lawyers who serve regular people.Footnote ²³ Most practicing lawyers lack these resources.Footnote ²⁴

15.2.1 Caseloads

In a sense, courts perform a reactive role. They adjudicate disputes coming before them, but – save select courts, such as the US Supreme Court – do not control their docket. Nevertheless, the demand on courts continues to grow. Among US district courts, the caseload has grown over the years by 165 percent (from 127,280 cases in 1970 to 337,537 cases in 2017). The caseload demands are arguably even greater than the numbers suggest: Much of this growth involved civil disputes between private parties, which on average take more time to resolve than criminal cases (10.8 months versus 7.0 months in September 2019 in US District Courts, and there has been a relatively consistent discrepancy of increased length for civil cases since at least 2000).Footnote ²⁵

The growth in caseload necessarily increases demands on the judiciary. The first is on the judges themselves. Among US district courts, the number of authorized judgeships from 1970 to 2017 grew by only 68 percent (from 394 judges to 663 judges).Footnote ²⁶ Accordingly, judges are tasked with greater responsibilities, particularly after IqbalFootnote ²⁷ and Twombly,Footnote ²⁸ whose heightened pleading requirements require judges to adjudicate matters earlier in the litigation process. Another is the litigants. Longer periods for a trial date, combined with costs of litigation, increase the likelihood that parties resolve their disputes through trial diminishes. It is a well-known and often repeated maxim that the vast majority of cases resolve prior to trial.Footnote ²⁹

15.2.2 Funding

As a branch of government, the judiciary is subject to the financial constraints that all public institutions face. In nominal dollars, judiciaries are often spared the budgetary cuts that befall other public services.Footnote ³⁰ For example, Article III of the US Constitution authorizes Congress to create federal judgeships, but not diminish their salaries. Congress wields the authority to reduce the number of federal judges; it has yet to do so.

In most jurisdictions, salaries constitute the single largest expense of the judiciary budget. Drawing again from the US federal example, salaries (and benefits) from judges, clerks, and staff totalled 64 percent of the FY 2021 total budget request.Footnote ³¹ Given a total appropriation of nearly $6 billion for this year, the federal commitment to salaries is sizable.

At the same time, it is important to consider judicial salaries in a broader context. US district court judges, the most common form of Article III judge, in 2020 earn $216,400,Footnote ³² much higher than the US median income,Footnote ³³ but much less than the elite private bar.Footnote ³⁴ These pay differentials may deter high-skilled lawyers from joining the bench or, in some instances, encourage departure.Footnote ³⁵ Moreover, this gap has only grown over time as judicial salaries lag behind the rate of inflation.Footnote ³⁶ Concurrently, legislatures resist expanding the number of judgeships, resulting in the number of judgeships failing to keep pace with caseload increases.

15.2.3 Judicial Reluctance toward Technology

It is an understatement to say that courts and technology are not the best of friends. Judges, on average, constitute the higher end of the age distribution. Judges typically have distinguished themselves earlier in their career, whether it be in private practice, government, or academia. This process takes time, so it is perhaps no surprise that judges are often in their fifties by the time they join the bench. For example, the average age of an Article III judge at the time of their commission is fifty-one years old. However, given lifetime tenure, most judges remain on the bench well past the traditional retirement age of sixty-five. In 2020, the average age of a district judge was sixty-seven years old and sixty-nine years old for a court of appeals judge.Footnote ³⁷

Judges, like many older adults,Footnote ³⁸ may be reluctant to adopt new technologies. Surveys show that among the general population, over 60 percent of Americans aged fifty to sixty-four and nearly 75 percent of those above age sixty-five identify as needing someone else to set up their new electronic devices.Footnote ³⁹ Other academic studies show that older adults often possess a phobia about technology, particularly when it involves a technology that they did not develop familiarity with during their younger years.Footnote ⁴⁰

Judges likely share these experiences of the general public. Many judges are years removed from the type of legal work now reliant upon this technology. Either the nature of their work at the time did not demand it or they had junior lawyers who could perform these tasks. On the bench, technology has changed little since the 1980s inception of digital libraries such as Westlaw or LexisNexis. Newer technologies, however, differ categorically from traditional digital search technology by going beyond search to providing answers.

These technologies are in potential tension with an idealization of courts, and judges in particular. Judges, by institutional design, are deemed impartial arbiters of disputes. They are equal to the legislative and executive branches, and are protected against removal without cause. Judges confer with their clerks, or even other judges, but ultimately issue their own decisions, free from any outside influence. Newer technologies challenge the independent role of judges, allowing actors outside the judiciary to influence how judges make decisions. Under this view, judges may cede autonomy. It is one thing for judges to draw upon resources – from clerks, judges, the common law – to inform their view. It is something altogether different for judges to draw upon resources advising them what to do.

15.2.4 Security of Judicial Documents

Another concern for the courts is security of judicial documents. Increasingly, legal technology utilizes cloud computing, hosting applications on remote servers on the web rather than personal computers or local servers. These servers are vulnerable to access by unauthorized third parties. This risk is particularly concerning for the legal profession, where lawyers are ethically bound to maintain strict client confidentiality.Footnote ⁴¹ These ethics also apply to judges, who similarly must take affirmative steps to avoid disclosing case information outside of the judiciary.Footnote ⁴²

Client security is a long-standing issue for the legal profession, at the same time co-existent with technology. Before email, lawyers corresponded with clients through traditional mail or fax. Both had the virtue of being secure, given their person-to-person formats. Email offered the lawyers the promise of instantaneous communication with their clients, while they risked the perils of breaches of client confidentiality, which in many instances could neither be easily remedied nor even traced. The legal profession initially responded by discouraging its use for client communication, but eventually recognized its advantages for facilitating communication between lawyers and clients.Footnote ⁴³ Today, lawyers widely use email, cognizant of the inherent risks of electronic communication.Footnote ⁴⁴ Contracts are now routinely finalized on the cloud.Footnote ⁴⁵

15.2.5 Judicial Aversion to Public Scrutiny

Judges, in contrast to legislators, are public servants who operate predominantly outside of public view.Footnote ⁴⁶ Whereas trials and certain pre-trial proceedings are open to the public, much of what judges do occurs in private. In contrast to a legislature, judges do not reveal their deliberations, only their final decision in the form of a motion or an opinion. Adding to this opacity, while the disposition judges’ decisions are of public record, they are not necessarily readily available. Moreover, judges elect whether to categorize their opinions as published or unpublished, which further limits their availability.Footnote ⁴⁷

The public’s ability to access federal judicial opinions has historically required lawyers to have access to a physical volume of the federal reporter, or, alternatively, a subscription to an online legal database. In recent years, new media platforms (e.g., Justia) have made federal decisions and statutes publicly available and free of charge. Other public judicial documents, however, remain less accessible.Footnote ⁴⁸

In addition, courts have been loath to link the identities of their judges to their decisions. For example, the Federal Judicial Center makes publicly available data on individual civil and criminal defendant filings and terminations filed in the federal district and appellate courts.Footnote ⁴⁹ This data, however, omits the judges’ identities for each record.Footnote ⁵⁰ This omission prevents those analyzing the data to observe how differences in case outcomes vary by individual judge.Footnote ⁵¹

From an academic and policy perspective, knowing which judges decide cases is essential for rigorous statistical analyses. For example, in a recent influential study on the bail decisions, researchers found that state court judges make two types of errors: (1) release defendants on bail who subsequently commit a crime while released and (2) deny bail to defendants who, based on observable characteristics, would have complied with bail conditions if released.Footnote ⁵² Given the random assignment of cases, researchers could identify when different judges were treating similarly situated defendants differently, and more broadly, the extent to which judges vary from one another.

Other countries worry about public scrutiny of judges. France has outlawed the publication of judge analytics, a crime punishable by up to five years of prison.Footnote ⁵³ In accordance with the Act, any publication that is likely to undermine the security of any parties to a judicial decision, including judges, must be obscured.Footnote ⁵⁴ In my own research, I sought to evaluate bail decisions of courts within a Canadian province. The province’s judicial council – comprising judges – was willing to release individual-level data on bail proceedings but declined to reveal the identity of judges for each case, on the grounds that it could draw undue criticism of the provincial judges.

Arguments in support of judge anonymity range from distortion of the litigation process to the safety of judges. In 2019, the Constitutional Court of France upheld the constitutionality of the prohibition on revealing judges’ identities, determining that court analytics based on individual judges could promote forum shopping, to the detriment of the administration of justice.Footnote ⁵⁵ In 2020, an individual injured and killed family members of a US district judge, after having appeared before the judge in prior litigation and criticizing her handling of the case on social media.Footnote ⁵⁶ This followed earlier instances – in 2005,Footnote ⁵⁷ 1989,Footnote ⁵⁸ 1988,Footnote ⁵⁹ and 1979Footnote ⁶⁰ – where federal judges were killed, all connected to litigants appearing before these judges. The Secretary of the Judicial Conference of the United States testified that identified threats and inappropriate communications directed at federal judges grew from under 1,000 in 2015 to nearly 4,500 in 2019.Footnote ⁶¹

15.2.6 Litigation as Arms Race

Litigation has long favored those with resources. Litigants with the most resources tend to attract the highest-ability lawyers. At the same time, even sophisticated litigants prefer to trim unnecessary legal expenses. Many legal technology tools allow lawyers to perform legal tasks at a fraction of the time and cost that it took even a decade or two earlier. Thus, there is a segment of the practicing bar that embraces legal technology to improve both the efficiency and quality of their practice. On the other end of the spectrum, many lawyers represent litigants of more modest means. Given the nature of their practice and their own limited resources, these lawyers are slower to incorporate these technologies.

Judges are not exempt from this competition. This asymmetry of legal sophistication threatens our civil and criminal justice system. This imbalance might be more palatable if litigation involved only parties of comparable means: that is, affluent litigants filing suit against affluent litigants; poor litigants filing suit against poor litigants. While this phenomenon occurs in certain situations – for example, intellectual property and corporate disputes typically involve well-financed litigantsFootnote ⁶² – many instances reflect a more David-versus-Goliath disparity, such as personal injury, immigration, or civil rights. This imbalance may distort legal outcomes at trial, if lawyers financed by well-resourced litigants can generate outcomes that maximize chances of prevailing or, if that is not possible, ameliorating the terms of an unfavorable outcome. These decisions, in turn, affect both the manner and terms under which subsequent litigants resolve their disputes, irrespective of whether they proceed to trial.Footnote ⁶³

The courts’ lack of technological sophistication also poses risks to our civil and criminal justice system. In terms of resources, courts fall somewhere between well-financed and modestly financed litigants. They certainly possess more resources – technological as well as labor – than the typical lawyer, but are increasingly falling behind the elite bar, typically those practicing at the largest law firms. Legal technology is transforming how the elite bar practices. In contrast, legal technology has had a modest effect on the judiciary. While existing developments are welcome – for example, judges increasingly allow litigants to electronically file their court documents – the courts’ earlier adaptation to technology suggests a slow transition.

15.3.1 Greater Technological Competence in the Judiciary

At first blush, this competence may appear unnecessary. Very few cases result in a trial. Among civil cases filed in district court in the period 1970 through 2019, fewer than 3 percent of filed civil cases ended in a verdict.Footnote ⁶⁶ Given this relatively low percentage, one can argue that judges have ample time to properly adjudicate the cases before them, without additional technology. Looking only at trial outcomes, however, provides an incomplete picture of the integral role that judges have in helping parties resolve their disputes. Based on civil filings in federal district court, parties resolved disputes on their own in roughly half the cases, without court adjudication. But in nearly a quarter of all cases, judges adjudicated the dispute, prior to trial, on procedural – for example, lack of jurisdiction – or substantive – for example, summary judgment, affirming findings of an arbitrator or magistrate judge – grounds.Footnote ⁶⁷ This data provides evidence that, at a minimum, judges play an active role in helping many parties resolve their disputes. Even when parties do not proceed to trial, they take guidance from judges during the pre-trial phase.

The judiciary’s response to the COVID-19 pandemic provides optimism that judges may be more open to technology than originally perceived. In the United States, every state has initiated online hearings in response to the pandemic.Footnote ⁶⁸ In addition, a majority of states allow litigants to electronically file court documents.Footnote ⁶⁹ In many respects, these transformations reflect the judiciary’s adopting well-established processes within the practicing bar.

These changes also provide second-order changes that are just as beneficial, namely democratizing the litigation process by increasing its access. Online hearings and electronic filing allow litigants to advance their claims without having to miss time off from work or find childcare. Even as the pandemic brought about curfews and shelter-in-place ordinances, courts reported a dramatic drop in failure-to-appear rates for civil and criminal proceedings.Footnote ⁷⁰

It will be interesting to see how courts respond when governments – at all levels – lift restrictions as COVID-19 moves from a pandemic to an endemic state. Will courts revert to their traditional practices that rely heavily on in-person interaction, or will they make a permanent transition to using technology to reduce the transaction costs of litigating? While judicial adaption of technological has its bumps,Footnote ⁷¹ its benefits, on balance, outweigh the growing pains.

Judges can and should make greater use of legal technology. While some obstacles to adoption may be budgetary, judicial norms likely play a larger role. As with most matters, judges exercise considerable discretion as to how they use technology in their courtroom. As more judges become comfortable with technology, and advocate for its use, judicial norms will evolve toward using legal technology as yet another resource at their disposal. A New York Supreme Court justice in support of using technology recently commented, “Judges should have some knowledge of the technology …. If [judges] have the knowledge … it makes the case more efficient.”Footnote ⁷²

15.3.2 Wide Availability of Judicial Data

Using technology to increase access to justice is an important step, but only a partial solution. As discussed throughout the chapter, the available legal technologies go far beyond access and search. The next frontier of legal technology will be tools that assist lawyers in their tasks that involve discretion and judgment. Imagine a system where lawyers can, early on, have a comprehensive understanding of the client’s legal issues, the factors most likely to determine success, and a probabilistic assessment of the litigation outcome. Of course, a good lawyer is capable of doing all of this, but it involves a fair amount of time. The emergence of legal technology allows the lawyer to achieve these objectives more efficiently, more accurately, and at less expense to their clients.

Development of this legal technology requires greater access to judicial materials. Most judicial documents – for example, opinions, orders, motions - are deemed public documents but are often difficult and costly to access. Courts can make these materials more readily available, most easily through their own webpages. Just as importantly, courts can encourage great use of these documents by the public, by making them available in electronic format (rather than as an image or pdf). Doing so saves third parties the time and expense of digitizing the documents and focuses on developing legal analytic tools.

Litigants, in turn, could use these analytic tools to make more informed choices about their legal disputes. Many litigants already do this: Sophisticated litigants – often through their lawyers – effectively evaluate the merits of their dispute, and choose the appropriate path. Less sophisticated litigants, however, often lack the resources to make this same determination. Analytic tools can help litigants more accurately evaluate their claim: for example, whether they have a legal claim, whether their legal claim is supported by statute or precedent, and even how a court might rule.

Litigants, by better understanding the legal dimensions of their disputes, could collectively change the distribution of case filings. In this reformulation, parties could resolve disputes where precedent points to a clear resolution, allowing courts to focus on disputes where either

1. (1) parties agree on the facts and for which the existing precedent fails to produce a clear result or

2. (2) parties disagree on the facts, such that they lead to divergent outcomes as a matter of law.

Academia and private sector researchers analyzing judicial data can have a symbiotic effect on one another to better understand the litigation, the judiciary, the legal profession, and the common law itself. Some research may be of interest primarily to academics or to the private sector. In many instances, however, research will generate interest among both groups, and push each other to develop tools that satisfy the rigor of their respective markets.

In addition to making judicial opinions widely accessible, courts can also release information about other judicial documents, such as motions and orders. As with opinions, orders are public in nature, but often difficult for the public to access, depending on the jurisdiction. This data is useful as a means of evaluating the courts’ performance in both civil and criminal matters, both procedurally and potentially substantively. For example, judicial research has revealed that judges are most likely to deliver a favorable ruling to plaintiffs shortly after they had eaten.Footnote ⁷³ A large-scale study revealed that judges routinely committed both Type 1 and Type 2 errors with respect to pretrial bail determinations.Footnote ⁷⁴ These academic studies were made possible only because courts shared their data.Footnote ⁷⁵

While this chapter has focused on the promise of legal technology of judges, there are reasons to exercise caution. Judges, more than litigants, must be sensitive to constitutional considerations of technology as it relates to privacy, discrimination, and access to the courts. Not every technology will be applicable to the courts, and even those that are may prove problematic. In addition, logistical issues abound, in part because of the decentralized nature of courts, both across and within jurisdictions, which impedes data availability.Footnote ⁷⁶ Lastly, judicial governance of technology is highly complex, for which scholars provide conceptual frameworks.Footnote ⁷⁷ Ultimately, courts should neither delegate their authority to legal technology nor eschew it altogether. Rather, like much of the elite practicing bar today, courts can use this technology as a tool to inform their own independent decisions.Footnote ⁷⁸

16.1.1 Structural Racism and Bias

In the criminal justice context, abundant high-quality policing and court data have allowed researchers to document the scope and depth of structural racism and other biases. This research underscores what smaller-scale observational and quantitative studies had previously shown: Black Americans are systematically treated worse than white Americans at every juncture of the criminal justice system. Policing data have shown that Black people are more likely to be subjected to roadside stops and are more likely to be the victims of lethal force.Footnote ¹⁶ Court datasets have allowed researchers to show that Black drivers are likely to pay higher traffic fines than white driversFootnote ¹⁷ and that judicial mechanics and the peremptory challenge system adversely affect Black defendants.Footnote ¹⁸ If you are Black, you are more likely to receive a longer sentence than a white person in a plea deal.Footnote ¹⁹ These studies confirm the systemic pervasiveness of racism in criminal justice. Data access permits studies with sufficient granularity to pinpoint specific areas where race might have the strongest magnifying effect, which may suggest priorities for formulating and testing reforms.

Large-scale studies that investigate the impact of race, ethnic origin, and gender in the civil justice system are sparse but show promise.Footnote ²⁰ One major foray comes from the Princeton Eviction Lab, which published the first study documenting the demographics of evicted renters on a national scale. Using a dataset containing 4.1 million records from public, private, and commercial sources, the study showed Black renters experienced the highest rate of eviction filings and judgments, and Black and Latin female renters faced higher eviction rates than their male counterparts.Footnote ²¹ Data-driven studies like those by the Eviction Lab can demonstrate the magnitude of inequity in the civil justice system. Their study is a significant step in understanding which demographic groups in the United States are suffering the brunt of the eviction crisis and raises new questions about how the civil legal system plays into this crisis. Do differing eviction rates among racial and gender groups mirror structural biases in society at large? Or does the civil justice system magnify these problems? Accessible civil justice data are needed not only to understand the system’s role in replicating or worsening structural biases in eviction cases, but also its role in consumer debt and other areas that disproportionately affect poor people, people of color, and women. Population-level datasets contain large numbers of observations and variables to help pinpoint court contacts and processes that have the greatest impact.

16.1.2 When Do Lawyers Make a Difference (and Why?)

Civil justice data can also shed light on whether lawyers make a difference in outcomes and, if so, in what types of cases and why.Footnote ²² To date, studies comparing represented and non-represented parties have been site-specific and relied on observational data or randomized controlled trials. Consequently, their findings are not easily generalized to other subject matters and other settings. Large-scale court datasets will allow comparisons across courts, subject matter areas, types of representation, and jurisdictions, producing more nuanced knowledge with broader impact.

The few studies investigating whether being represented by a lawyer affects the outcome of a case tend to show that representation makes a difference, but how much difference it makes depends in large part on the type of service provided, case type, and procedural complexity of the case.Footnote ²³ Some research suggests that, in select settings, being represented by a non-lawyer advocate confers similar advantages to being represented by a lawyer. These studies raise important questions about how courts should deal with cases where one or both parties are not represented by a lawyer and how legal services providers should prioritize the provision of their resources.

Data-driven research confirming and extending these findings offers insights into how to “level the playing field” between represented and unrepresented parties. Some jurisdictions have changed their rules to allow judges a more active role in explaining the law or eliciting facts to a pro se party.Footnote ²⁴ Courts might also focus on simplifying procedures or prohibiting lawyers in certain matters or fora altogether.Footnote ²⁵

Research showing whether and when being represented affects case outcomes can also be important for legal services providers allocating resources. In legal aid settings, eligible applicants may receive representation depending on the availability of a lawyer specializing in their type of case. Research on the differential effects of representation might persuade legal services providers, which turn away almost half of eligible applicants,Footnote ²⁶ to shift their resource priorities. Some providers report that they have already begun targeting their resources to clients with income levels just below eligibility cut-offs after data-based research showed representation did not affect outcomes at lower income levels.Footnote ²⁷ Large-scale data studies about the impact of lawyers would also provide evidence for or against various approaches being promoted to increase access to the civil justice system. With the eviction crisis worsening, the movement to fund lawyers to represent tenants at risk of eviction is gaining ground.Footnote ²⁸

Such studies might also be helpful in answering the question of which types of services make a difference. In the last ten years, limited scope representation has been promoted as an important means of increasing access to justice by allowing fewer lawyers to represent more people, an approach that many legal services providers are pursuing on the assumption that they can help more clients.Footnote ²⁹ Despite the growth of brief advice services,Footnote ³⁰ there is little evidence that these services have a positive effect on outcomes. One study found that while brief advice assisted clients in surmounting procedural barriers, it did not have significant effects on substantive outcomes.Footnote ³¹ However, this study, which focused on the evictions caseload of a legal aid provider in San Mateo County during the summer of 2009, may not be generalizable to other settings.

Improving short-term outcomes – helping a client win a specific case, stay in their home, or avoid wage garnishment – is one measure of effective representation. Others include helping clients understand the law, supporting them through the process, and empowering them to advocate for themselves.Footnote ³² An understanding of whether and when representation makes a difference is important knowledge to have in allocating scarce resources among eligible applicants who live in poverty and seek legal assistance.

16.1.3 Civil Justice Involvement and Poverty

The questions we have identified so far focus on whether civil justice institutions treat people fairly, allocate resources equitably, and achieve just results. Other research avenues focus on the people involved, not the court encounters themselves.

Getting a Fuller Picture from Big Data. More accurate and consistent data provide opportunities for deeper understanding about individuals’ interactions with the civil justice system: who enters the system (and why), how they navigate the system, and what impacts this interaction has on them in the long term. Specifically, linked population-level datasets that consider the antecedents and consequences of court involvement can widen the lens of research, leading to inferences and interventions that can scale across sites. By linking information from before and after court involvement, it may be possible to understand mechanisms that help avoid entanglement in the justice system or to mitigate its negative effects. This research shifts the emphasis away from court processes and legal resources to understanding and addressing the conditions and events that lead up to court involvement to begin with. In doing so, it may help people avoid being evicted by landlords or sued by debt collectors altogether.Footnote ³³

This type of analysis is already being done at other intersections of poverty and policy. For example, researchers in Sonoma County, California, integrated four years of county and state records from health, mental health, substance abuse, housing, criminal justice, and human services systems.Footnote ³⁴ This cross-domain dataset showed that 1 percent of the population accounted for one-quarter of jail time and behavioral health services and half of overnight homeless housing.Footnote ³⁵ Another study linked administrative records from the Child Care and Development Fund, a federal subsidy program, with American Community Survey data to reveal that women who worked full-time but had less social support were more likely to receive the child care subsidy.Footnote ³⁶ The study considered characteristics of the mother, such as education level, attainment of labor skills, and community ties, and found that relative to other low-income women, the women who received the child care subsidy tended to have higher educational attainment (high school diploma or higher).Footnote ³⁷

Understanding the long-term consequences of a government intervention has also been made possible with linked datasets. One recent study created more than 150 million parent-child links to analyze the downstream intergenerational effects of the 1970 Clean Air Act Amendment and found that regulation-induced air quality improvements during pregnancy correlated with parental investment and higher college attendance in later generations.Footnote ³⁸ This analysis was achieved through linking decennial census data, administrative records, and survey data and utilizing Protected Identification Keys (PIKs) in order to anonymize personal identifiers.Footnote ³⁹ In another study, researchers linked national administrative data on the labor market and earnings with records on public housing to find that children who resided in public housing that was demolished later earned more than children who remained in non-demolished public housing, suggesting that job accessibility improved for children forced to relocate.Footnote ⁴⁰

The Interaction between Poverty and Entanglement with the Justice System. Just as these studies used linked data to understand the role of government policies in the lives of people living in poverty, research linking court datasets to datasets that are indicators of health, financial circumstances, and other measures of well-being can yield knowledge about the effect of court entanglement over the course of a person’s life. Research is already under way. The Institute for Research on Poverty at the University of Wisconsin–Madison found women with an incarcerated family member have a higher risk of poor health, and Black women are, in turn, disproportionately likely to have an incarcerated family member.Footnote ⁴¹ Researchers looking at Connecticut evictions found persistent, largely untreated, mental health problems and continuing housing instability.Footnote ⁴² Other research, using mixed methods and blending administrative data with court data, shows correlations between eviction and criminal justice involvement,Footnote ⁴³ how neighborhood disadvantage can exacerbate racial health disparities,Footnote ⁴⁴ and how eviction in early childhood is associated with neighborhood poverty, food insecurity, and obesity in later childhood and adolescence.Footnote ⁴⁵

Studies that investigate correlations between involvement with the civil justice system and other public institutions are beginning to emerge. For example, a study linking bankruptcy filings to administrative tax and foreclosure data found that filers who received Chapter 13 bankruptcy protections experienced increased annual earnings and decreased foreclosure rates.Footnote ⁴⁶ Several studies have also appeared that focus on the short- and long-term consequences of being evicted for poor families. Using New York City housing court records, researchers found that evictions correlated with an increased risk of homelessness, long-term residential instability, and emergency room use but did not significantly worsen longer-term financial circumstances.Footnote ⁴⁷ A longitudinal comparison of Cook County court records to credit bureau and payday loans data illustrated how the effects of successful evictions were small relative to the financial strain experienced by both evicted and non-evicted tenants in the time leading up to eviction actions.Footnote ⁴⁸ Taken together, these studies raise the possibility that the prevailing view (which assumes that preventing eviction will have long-term benefits on alleviating poverty) may be myopic and that the locus of policy should shift from court processes to poverty-reduction programs.

More and greatly accelerated research using large, linked datasets is needed to determine the consequences of court involvement for the life cycle of poverty across sites and population subgroups. Providing lawyers and legal assistance to poor litigants assumes that the best locus of intervention to alleviate the effects of poverty must be the civil justice system – and that more lawyers and more legal assistance is the best approach. More studies may tell us, however, that assistance and opportunities upstream may be the most effective way of addressing housing and financial insecurity to prevent eviction and consumer debt actions from being initiated in the first place. Studies may also show that the impact of representation varies across race, ethnic enclaves, age groups, and population density.

Easily accessible civil justice datasets are needed to undertake such studies. In Section 16.2, we describe current challenges to obtaining such data and explain how the development of a civil justice data commons may begin to overcome these barriers.

16.2.1 Data Disarray

Although courts are moving to electronic case management systems, many are still managing case files the old-fashioned way: paper. Others are using case management systems, but case pleadings are handwritten hard copies scanned into the digital system and are not machine-readable.

The absence of machine-readable pleadings, which contain valuable information about the case and the litigants, is a major technical impediment to granular research on the civil justice system. To overcome this barrier, some researchers have resorted to hand-coding pleadings, an expensive and time-consuming process.Footnote ⁵²

Over time, the adoption of e-filing that is integrated with case management systems may alleviate this problem (though many mandatory e-filing systems exempt unrepresented parties).Footnote ⁵³ For now, it is necessary to hand-transcribe the information or develop optical character-recognition tools that can translate pleadings that are marked with stamps and often contain handwriting into machine-readable form.

Administrative case level data scraped from online court case search websites – which pull from case management system data originally entered by hand by a court clerk – can be messy and contain incorrect or inconsistent values, making them difficult to interpret. Addressing these issues often requires a manual comparison with the corresponding docket by a specialist who can align concepts properly. That is tedious when the dockets are available online and onerous when it is necessary to contact the clerk of the court to understand which dispositions numeric codes refer to. Improving data capture is critical to improving analyses and the function of courts.

16.2.2 Barriers to a Common Taxonomy

Another data challenge is the absence of a common taxonomy among courts. The lack of shared substantive and procedural classification systems results from the diverse sites of authority among state, county, and lower-level courts. Different definitions across courts, even for concepts as fundamental as indigency, hamper coding and comparisons. Court procedures with different terms for various pleadings, requirements, and judgments also complicate matters. A court may lump evictions and contract cases together under “contracts,” or it may separate evictions and contracts but not distinguish among consumer debt cases (with some number of these classified separately under “small claims”). A rare few might use more granular categories that distinguish medical debt, consumer debt, and other debts, as an example. Coding these types of categories is necessary to study correlations between court involvement and life-course events. For instance, being able to identify medical debt is necessary to understand how much healthcare debt collection has grown since the pandemic.

In an attempt to tame some of this chaos, the National Center for State Courts (NCSC) developed the National Open Data System (NODS), intended to guide data collection in courts throughout the United States.Footnote ⁵⁴ NCSC’s exhaustive effort yielded standards for basic elements such as case type, representation by counsel, case filing, disposition, and case closure date. NODS is an ambitious effort to transform the collection and classification of court data. Its adoption will be dictated by the cost of implementing or reconfiguring court case management systems and the extent to which courts believe that its implementation is consistent with the court’s priorities.

While NODs is a forward-looking strategy, there may be other strategies to rationalize court records from different jurisdictions that have already been collected. One approach might be to use natural language processing to identify subcategories in court documents to generate classifications. These generated classifications could then be matched to NODS categories. For instance, an algorithm might be trained to recognize medical debt from other types of debt based on the language in pleadings. Whether such an approach could capture subcategories of case types or definitional and procedural variation – likely more challenging – remains to be seen.

16.2.3 The Special Challenge of Assigning Demographic Characteristics

A particular challenge involves obtaining demographic information for civil court participants – a prerequisite for understanding equity concerns within the civil justice system. Under NODS, civil courts are supposed to collect race, ethnicity, and other demographic data about litigants. Most courts do not currently capture these data, but even if they attempted to, they would face a fundamental difficulty: Many, if not most, defendants in certain types of civil actions do not appear in court. One recent study of debt-collection cases in state courts found that, for courts where there was available data, 70 percent of debt cases ended in default judgments.Footnote ⁵⁵ With such high rates of default, it is impossible to collect information about the race, ethnic origin, or gender of the tenant or debtor using only court data.

One promising strategy to append demographic characteristics to court data is through data linkage to census or commercial data. Linking at the per-person level requires the court records to have complete, accurate name data or other unique identifiers to make a match. Previous linkages to census data have had match rates between 40 and 90 percent, depending on the completeness of the names and addresses available, with the higher match rates possible when additional fields such as age or date of birth are available. The sparse identifiers present on civil court records have led other researchers to incorporate additional data sources. For example, Case Western Reserve University’s Center on Urban Poverty and Community Development matched Cleveland Housing Court data to Ohio Public Assistance records to obtain race information.Footnote ⁵⁶ While this linkage allowed researchers to add otherwise absent demographic characteristics to eviction files, they were able to do so only for defendants who applied for and obtained public assistance.

16.2.4 Legal Barriers

While individual case files are available either online or in the clerk’s office in every jurisdiction, legal rules may limit access to bulk electronic case records. Although all fifty states and the District of Columbia have enacted public records laws, more than a third exempt the judiciary from coverage.Footnote ⁵⁷ Even when public record laws apply to the judiciary, it is unclear whether they entitle people to obtain electronic case files in bulk form.

Approximately thirty-two states have rules governing the availability in bulk form of electronic public case records. Some states like Arizona make case file records generally available for bulk download.Footnote ⁵⁸ Even in states that do not bar access to case data, county courts may have rules that limit their accessibility.Footnote ⁵⁹ Some states prohibit the dissemination of bulk court records in electronic form, except where explicitly provided by a court rule or order.Footnote ⁶⁰ At least one state disseminates bulk court data only to commercial purchasers. Other states, in contrast, bar the bulk download of court files for commercial gain.Footnote ⁶¹ Finally, there are states like Hawaii, which will grant some requests for bulk data as long as the entity making the request is willing to pay for fees associated with the cost of providing the data.Footnote ⁶² Many states also charge a fee or have terms of service that prohibit the scraping of court websites or CAPTCHA intended to prevent it.Footnote ⁶³ On top of this confusion, states may have different privacy policies for court records with varying levels of restrictions.Footnote ⁶⁴

16.2.5 Institutional Barriers

Institutional barriers are arguably the most important to address to facilitate data sharing. In preliminary planning interviews for the CJDC, we found a wide spectrum of views among judges and court administrators about the value of collecting and sharing data. Some judges did not view the data as valuable for either research or operational purposes; others strongly believed that research on aspects of the civil justice system was important for determining how well the courts served their communities.

Courts also fear scope creep resulting in data misuse, where a project approved to study one topic veers to another and publishes a study that the data may not support. Safeguards and reviews of approved uses must keep pace with growing volumes of data. Courts may worry that errors in their data will come to light. The errors may result from hurried or flawed collection methods, or through poor data management, statistical analysis, logic, or communication. When errors arise, it is important to examine their origins and impacts.Footnote ⁶⁵

Even when court rules provide for public access to data, court personnel face administrative burdens to sharing them. They receive numerous requests for data and are pressed to develop internal custom-made approval processes on the fly. Most courts lack a data governance committee. Providing requested data is a time-consuming and mostly manual process fraught with risks of data misuse, errors in court data, and embarrassing findings.

16.3.1 What Is the CJDC?

The civil justice data commons (CJDC) is built according to principles that underlie data commons in the health sciences, created to accelerate research on medicine and disease using large databases.Footnote ⁶⁶ Like genomic and other health data commons, the CJDC is controlled by a governance regime negotiated between data sharers, data users, and the directors of the commons.Footnote ⁶⁷ The terms of service reflect the data security needs and project scoping requirements of data contributors with access demands of researchers.Footnote ⁶⁸ The governance model is “hardwired” into the commons’ permissions infrastructure, ensuring that approved uses and users gain access efficiently.Footnote ⁶⁹ Transparency is key: The commons makes clear what is the path to access, what requests have been granted, and what has been learned. Housed in an academic research center, the commons is able to act as an independent “trusted intermediary” for all parties.

The CJDC draws on the “Five Safes” framework to provide data access, balancing the terms and conditions of data providers and the needs of researchers.Footnote ⁷⁰ Under this framework, the CJDC approves only “safe projects” intended to increase knowledge of the civil justice system. Researchers cannot use data for commercial purposes or for administrative purposes that have direct implications for affected individuals in the data. CJDC users are “safe people,” who are affiliated with universities, courts, government agencies or non-profit institutions, such as policy institutes and community organizations. The CJDC is a “safe setting” for the data, which can be accessed only on the CJDC’s platform. The data are “safe data”: Data sharers certify that the data they provide are legally and ethically sourced, and records in the commons are documented so the chain of their provenance is clear. Last, the commons follows the principle of “safe outputs.” CJDC users must submit their analytic outputs to the CJDC team, who will ensure that the data were used for the study proposed and that statistical disclosure limitation methods have been applied to prevent re-identification of data subjects.

In essence, the data commons provides a regulated marketplace for data owners and data users. For data owners, the commons facilitates data provisioning and quality control and oversees researcher use of the data. For researchers, the commons provides an interface where researchers can conduct statistical analyses on requested data. The CJDC can enable data discovery across multiple sites and jurisdictions, allowing potential users to browse metadata documentation and propose projects. Project requests are reviewed and approved in accordance with terms of use dictated by the data contributors that govern such questions as who is allowed to use the data, the scope of the research, and the purpose for which the research is undertaken. When projects are approved, CJDC provisions data into a secure workspace with statistical applications that researchers need. By making this information more readily available, knowledge about the prevalence, incidence, and consequences of civil justice problems can be produced, reducing reliance on anecdotal, outdated, or qualitative data alone. CJDC provides a simple, responsive interface that permits researchers to determine what data are available and what research has already been done with various datasets. Users can receive updates about new or refreshed data sources, recommendations on new data topics, and findings from the community.

In its mature form, the CJDC will have a secure infrastructure to automate data provisioning and an interface for data contributors to facilitate data sharing. In addition, it will have separate interfaces for authorized researchers and civil justice institutions so that they have ready access to data for academic and policy research and operational insights. The CJDC will host only machine-readable and documented data, which will be encrypted and held according to industry standards for securely stored data. The CJDC will offer technical assistance or services to help with cleaning, harmonizing, data documentation, and linkages. Funding for the operations and expansion of the Commons might come from federal and state government and foundation grants. The prototype data commons integrates extracts of data from data contributors. A future version of the data commons may have a federated structure as well, enabling the controlled access of data from a data controller’s servers. This may alleviate concerns about storage and control.Footnote ⁷¹

16.3.2 Addressing the Barriers

Institutional Interests and Concerns. As noted above, the greatest barriers to making civil justice data more easily accessible are institutional. Courts are hesitant to share data out of concerns about data security, misuse, and errors, and findings that are embarrassing or ignite unwanted political controversy. The CJDC addresses these concerns by intermediating data sharing and access processes. Court administrators state their terms and conditions for data use, and CJDC screens requests and users, approving only requests that meet these conditions for data access. CJDC also takes over the monitoring role for approved data uses, ensuring scope and data security compliance. By using the well-established Five Safes framework for data access control, the CJDC ensures balance between the concerns of data providers and the needs of researchers. Institutions may also be sensitive to the optics of using these data. The CJDC provides greater privacy protection for data subjects than the open data portals already in place.Footnote ⁷² Finally, the CJDC helps courts avoid surprises by permitting output review. It is critical to have independent research and transparency but equally important for courts to be able to confirm that studies have interpreted local laws or policies soundly. Giving the courts this “heads up” can allow them to frame the findings as opportunities to address issues rather than being surprised by negative headlines.

The CJDC also provides valuable services for data sharers through the production of knowledge to increase their own transparency. We expect that CJDC documentation and data standardization will enable application developers to create tools that work across sites. CJDC data can inform institutions on where things are working well and where they are not, providing evidence to support resource reallocation.

Controlling the Data Disarray. Working with the commons is not an all-or-nothing proposal. Sites can contribute what they have and increase their data sharing over time. For example, a court may share their machine-readable docket information, knowing that key data elements are not yet digitized. Commons users may help the court address this gap, such as through character recognition software or clerical keying. This creates a win-win situation where both the court and researchers gain more usable data. Other courts may share data they know is discontinuous, perhaps because of a change in records management vendors, updates to laws or policies, or transitions from paper to electronic records. The CJDC staff will document the data, and CJDC users will enhance detail about the datasets and likely create code that bridges the discontinuity. By fostering a community of users, this knowledge will be shared back to the court and subsequent researchers.

A necessary step in building out the CJDC is the harmonization of court data across jurisdictions, with their disparate data collection policies and case management systems. The CJDC seeks alignment with and adoption of NODS. Important first steps are determining what court data are machine-readable across sites and documenting challenges to NODS standardization within and across datasets. CJDC also standardizes data elements across contributed files, for example, creating a common vocabulary for eviction and debt cases and geocoding location data to facilitate area-level analyses. This standardization can include both manual matching and algorithmic entity resolution, allowing an accurate picture of the actual cases despite typos or name changes in the data.

Much of this initial work will continue to be labor- and time-intensive until we understand paths to automate the process of aligning new sites and keeping information up-to-date from courts we have been working with. A key feature of the CJDC is user-friendly metadata (data about the data) noting the sources, time periods available, present or absent fields, and incompleteness rates. As this metadata conforms to NODS, both research and court users will benefit. This metadata goes hand-in-hand with tools to visualize the available data. This data cleaning and sorting is a significant benefit to data contributors, who likely do not have the resources or expertise to perform this work on their own, and provides a provides a clean foundation for research.

Appending Demographic Characteristics. Because court files cannot capture the demographic characteristics of litigants who do not appear in court, linkages to external data sets are necessary to produce this knowledge. The CJDC is pursuing linkages among civil justice, census, and administrative datasets to enable analyses of civil justice eviction and debt-collection cases to develop demographic information about court litigants. These data will be critically important for understanding systemic biases in the aftermath of the pandemic.