Do Androids Dream Of The Common Good?

Artificial Intelligence and Legal Scholarship

The sprawling, swirling debates over the common good as a political and legal concept show no signs of abating; indeed if anything they have intensified and broken into the mainstream among commentators with very large audiences. Part of the debate involves my book Common Good Constitutionalism (CGC), and it is rare that a week passes without someone, somewhere, posting some new paper or substack critiquing or defending or commenting upon CGC. Most of these I cannot read, time being a precious commodity, and having already spent more time than I would have wished reading commentary and writing responses and further responses. In an ideal world, one moves on to fresh positive work, rather than engaging in ever-more-detailed debates about past work.

Nonetheless, one recent paper did catch my eye, not so much for the content of its argument - I do not think that argument stands up in even a light breeze, as I will explain below - but instead for the author, or should I say authors. In that regard, if in no other, the paper raises some points of general interest about the use of artificial intelligence in legal scholarship, and about whether disclosure regimes are an adequate tool for addressing the resulting questions. (Hence this post, despite The New Digest being on August break).

Emeritus professor Thomas A. Smith of San Diego Law recently posted on SSRN a paper titled The Ruler’s Equity: Common-Good Constitutionalism and the Reoccupation of Jurisdiction. If Smith is the only listed author, why did I mention another? For that we must read the paper’s initial footnote, which discloses: “This article was developed and drafted in extended dialogue with Claude, an artificial-intelligence assistant made by Anthropic, which helped to structure the argument, to draft and revise the prose, and to locate and verify sources (my emphasis). The theses, the judgments, and the conclusions, together with responsibility for any error that remains, are the author’s own.” On August 20, Professor Lawrence Solum reposted the paper on his Legal Theory Blog, giving it a “Highly Recommended!” Solum does not mention the extended-dialogic contribution by Claude in his recommendation; more on that anon.

Several law reviews, including the Harvard Law Review, have recently adopted policies that allow submission of scholarship generated with the aid of artificial intelligence, so long as the use of AI to write text is disclosed. But the paper at issue here illustrates, I think, the inadequacy of such disclosure policies. The basic problem is that the terms of such disclosures allow a nearly infinite scope for vagueness, blurring of responsibility, and perhaps even gamesmanship in the use of the machine. (For similar reasons, indeed, some administrative agencies have historically had to move to extremely specific and tightly regulated requirements for the exact text of disclosures, something I assume law reviews are neither willing to do nor capable of doing).

Even before reading the body of the paper, the disclosure statement leaves me puzzled. I admit to being old-fashioned, but it seems to me that structuring an argument, drafting and revising the prose, and locating and verifying sources (!) is the very pith and marrow of the scholar’s own work, properly understood. Formulating some general concept and throwing it to the machine to execute is not itself scholarship. Any academic worth his salt knows that the writing is all, and that in some mysterious way an idea does not become incarnate until it is turned into prose. Even if Claude only “helped” to do the tasks that turn vague ideas into scholarly substance, one might well think that the machine should have been given express co-author credit, not merely relegated to a footnote. I would not think very well of a scholar who took sole author credit for a paper and then disclosed in a note, sotto voce, that a research assistant had helped to structure the argument, draft and revise the text, and find and verify the sources. One is also left puzzled by the vague talk of “extended dialogue.” I think it is at best conceptually controversial to think that one can really speak, in any real sense, to the machine; and more practically, the vagueness of such disclosures leaves it unclear who is really responsible for what, despite Smith’s later partial (but only partial) assumption of responsibility.

My puzzlement only deepened when I posted “Smith’s” draft into Pangram, a leading detector of machine-written content. The Pangram report informed me that 100% of the text of the paper was AI-generated (see the header image above). Now, I do not necessarily take this at face value. I have heard, vaguely, that Pangram sometimes yields false positives or overstatements, and for technical reasons I don’t really understand, a 100% score need not mean that literally every word was chosen by the machine. For all I know, some other detector might well yield a different result. Furthermore, a very literal interpretation of the disclosure footnote might be consistent with a 100% AI-generated text, if Smith is saying that he just proposed a general concept to the machine, pointed it in a certain direction, and let it write.

Still, it leaves one a bit queasy to find that a paper marketed under the name of a professor of law might consist, for all we know, of a text mostly or entirely composed by auto-completion. If indeed it is true that Claude “helped” to “draft and revise” approximately all of the text, “help” means something different than I thought it meant. The “extended dialogue” between Smith and Claude seems, at least as far as the actual writing is concerned, to have been a monologue by the machine, or close to it. Or maybe not; my point is not that we do know that Claude wrote the whole thing, but that we can’t really know who did what, and that is why disclosure, at least in the form given here, is entirely inadequate to inform us.

And how well did Claude do at its appointed tasks? I’m afraid that the answer is: quite poorly. Rather than weary the reader with yet another full-bore response, I will confine myself to one key point in the joint-dialogic argument developed by SmithClaude, or perhaps ClaudeSmith. (One cannot really know which moniker is more appropriate. “O brave new world, that has such [cyborgs] in’t!” Even “cyborg,” a hybrid of man and machine, may not be the right term, if the Pangram assessment is correct; perhaps “android,” a human-shaped machine, is closer to the truth, although my point is that the paper’s disclosure statement leaves these various possibilities unclear).

When lots of throat-clearing and generalities and verbiage are stripped away, ClaudeSmith argues, in essence, that CGC’s explanation of the classical concept of determinatio, the sometimes-unavoidable specification of background general principles of natural law into concrete legal rules, is a sinister repackaging of Carl Schmitt’s voluntarist and decisionist view of the specification of legal commands. Putting aside that this is a bad misreading of Schmitt, as I have explained elsewhere; and also putting aside the typically parochial horror of Schmitt that one encounters essentially only in Anglo-American legal discourse; the main point is that ClaudeSmith has somehow stumbled into a simple logical blunder. That Schmitt thought there was irreducible discretion in the specification of positive legal rules does not mean that any account of irreducible discretion in the specification of positive legal rules must count as “Schmittian.” The need for such specification, and relatedly the necessary indeterminacy of positive law in some set of hard cases, due to the limits of lawmakers’ foresight and the generality of language, is hardly an insight unique to Schmitt, as the late and very lamented Fred Schauer observed to me years ago. It is common to Aquinas, Blackstone, and many others, my favorite exposition being that given by Portalis.

As to determinatio in particular, there is a venerable tradition, perhaps most famously associated with Aquinas, holding that determinatio requires the lawmaker charged with care of the community to bring general principles of natural law down to the specific point of application. Crucially, this must sometimes require supplying legal content that, while reasoned in one sense, is in another sense a product of the lawmaker’s will-based choice. It may well be entirely reasonable to think that there ought to be some speed limit or other in a certain area, but the choice of 55 miles per hour, as opposed to (say) 60 miles per hour, cannot itself be entirely derived from the general reasonableness of speed limits. Reason itself requires a bounded choice within reasonable limits, a choice whose specific content reason cannot fully dictate. As CGC explained:

A corollary of the nature of determination is that the public authority - including the executive exercising delegated authority - may, without transgressing its boundaries, engage in what I have called “rationally arbitrary decisions.” Because determination involves specification within a range in which reason need not yield a unique answer, some element of irreducible judgment will be required. Should the statute of limitations for a given offense be ten years or fifteen? Or perhaps twelve? The law is not so sophomoric as to demand a first-order reason for the choice of one particular number over another, for it is impossible to give any such reason, at least within a reasonable range of choices. In this sense, reason itself allows a certain degree of arbitrary specification, which will thus not be coded as “arbitrary” in the legal sense for purposes of the common good framework.

If this is Schmittian, then so too is Professor Finnis’ view of determinatio, which indeed I cite and quote at length in CGC. As Finnis explains:

the kind of rational connection that holds even where the architect has wide freedom to choose amongst indefinitely many alternatives is called by Aquinas a determinatio of principle(s) - a kind of concretization of the general, particularization yoking the rational necessity of the principle with a freedom (of the lawmaker) to choose between alternative concretizations, a freedom which includes even elements of (in a benign sense) arbitrariness (my emphasis).

As Professor Finnis is not a Schmittian, something has gone badly wrong inside the gears and wheels of the ClaudeSmith android, or hybrid, or what have you.

All this of course poses larger questions about legal scholarship and the adequacy of disclosures and disclosure policies. So long as the human author discloses the “extended” dialogic contribution of the machine, while leaving it unclear what exactly the respective roles were, are we are to take the resulting argument on its own terms and give it whatever weight it naturally has? (Or rather: artificially has). Are law reviews now to publish scholarship generated in whole or in part by the machine, so long as a vague footnote acknowledges that one is reading the “scholarship” of a cyborg or android, one knows not which?

And what of Professor Solum? Does Professor Solum believe that disclosure of this sort is really sufficient? If so, ought not the obligation of disclosure carry over to those who publicly recommend the underlying work of “scholarship,” so that readers of the recommendation will know what exactly they are dealing with?

For my part, I do not think that disclosure of the machine’s participation is a cure-all, any more than disclosure would suffice if a scholar were to drop a bland footnote admitting that he had put a vast team of monkeys to typing away, sifted through their work product, and then selected a facially grammatical draft that the monkeys had stumbled upon by chance. Such an admission would merely aggravate the offense. Real scholarship, to be such and to deserve consideration as such, depends on the assumption that the scholar has not only published an output, but has generated that output through a particular exercise of the scholar’s craft: assessing evidence, generating and reflecting upon arguments, and turning them into writing.

Chief Justice Hughes once wrote for the Court that “the one who decides must hear,” meaning that the one who decides a controversy must be the one who hears and considers the evidence and arguments. He justified that principle by observing that “the weight ascribed by the law to the findings -- their conclusiveness when made within the sphere of the authority conferred -- rests upon the assumption that the officer who makes the findings has addressed himself to the evidence, and, upon that evidence, has conscientiously reached the conclusions which he deems it to justify.”¹ Analogously, in scholarship, the one who publishes must think, research and write, rather than outsourcing to the machine everything save some general idea - an idea, for all we know, as vague and general as “write a critique of Vermeule.”

All I can say is that, first, I will not be responding to machine-generated texts in the future (I felt it necessary to say something about this one in order to explain that general policy); and, second, that in my own scholarship, the reader will always and only receive the modern analogue of work composed by the gnarled hands of a monk plying a quill pen by candlelight. Readers of CGC will, I suppose, expect nothing more, or less.

1. In later cases, the Court made clear that, in virtue of the presumption of regularity, reviewing courts ought not look behind an officially stated rationale unless there has been a “strong preliminary showing of bad faith or improper behavior.” My view here is, by analogy, quite consistent with that further principle as well. I would not have even inquired into the real authorship of ClaudeSmith’s paper, or even, in all probability, focused on it, absent the confession of very substantial machine involvement. My preferred regime would be one in which authors commit not to using AI at all except for aid with background research, as opposed to writing; but in which, by a presumption of good faith, one assumes honorable compliance with that norm unless there is a strong preliminary showing to the contrary.