Matthew Sag (@matthewsag) header
Matthew Sag (@matthewsag) profile picture

Legal academic focused on AI & Copyright, and AI as a legal technology.

Atlanta
Joined May 2009
Photo by Matthew Sag (@matthewsag)Photo by Matthew Sag (@matthewsag)Photo by Matthew Sag (@matthewsag)Photo by Matthew Sag (@matthewsag)Photo by Matthew Sag (@matthewsag)Photo by Matthew Sag (@matthewsag)Photo by Matthew Sag (@matthewsag)Photo by Matthew Sag (@matthewsag)Photo by Matthew Sag (@matthewsag)Photo by Matthew Sag (@matthewsag)
Matthew Sag retweeted
If you're still concerned about citation hallucination by LLMs, you're about 2 years behind here.
83
301
52
3,524
321,365
The Constitution has a Privileges AND Immunities Clause. The Fourteenth Amendment has a Privileges OR Immunities Clause. Any lawyer can tell you they are different provisions, but run both through a standard text-analysis pipeline and they come out the same: “privileges immunities.” That happens because most pipelines start by deleting “stopwords,” meaning “the,” “of,” “and,” “or,” and a few hundred other common words that are assumed to carry little meaning. The habit comes from 1950s information retrieval, where dropping common words was mostly a way to shrink an index. Until now, no one had tested whether it helps a model classify judicial opinions more accurately. My new article, Stop Removing Stopwords, just out in the Journal of Law and Empirical Analysis, runs that test. Using thousands of Supreme Court opinions, I asked whether stopword removal helps a classifier sort opinions by ideological direction or by whether they decide a constitutional question. I tried three standard stoplists (scikit-learn, NLTK, and spaCy). Then I built custom stoplists by deleting each of about 18,500 words, one at a time, and measuring how the deletion affected classification accuracy. The answer is no -- stoplists were not helpful for either task. The standard lists made the classifier worse in every test, and the custom lists produced no gain that could be detected. Even “and” and “or” carry real information. Whether deleting one helps depends on the phrase it sits in, but a stoplist removes a word either everywhere or nowhere. The lesson goes beyond stopwords. If you build or buy tools that analyze legal text, ask what the pipeline throws away before the model ever sees it. Preprocessing steps that look like housekeeping can change what your results mean. Many thanks to Lee Epstein and the reviewers for careful editing and thoughtful suggestions, and to @Sage_Publishing and @UNLCollegeofLaw. Link to the full paper in the comments. #LegalTech #TextAsData
11
12
3
59
3,298
Matthew Sag retweeted
Knowledge work has moved from the 80/20 rule to the 99/1 rule: AI hands you something 99% finished in a minute. Doing 1% of the work sounds restful. Instead, a March HBR/BCG study coined “AI brain fry”: workers pushed into heavy AI oversight report 33% more decision fatigue and 39% more major errors, with strain surging past 3 concurrent AI tools. Why? Because the old 80% slog (formatting decks, tidying models) was disguised recovery where your brain worked out the argument. Delete the cruise, and the whole day is take-offs and landings.
40
115
40
1,235
236,872
Matthew Sag retweeted
So, @CopyrightOffice has finished scanning all of the registration record books: copyright.gov/newsnet/2026/1…. This is a huge deal, although the Library of Congress's interface is terrible for these purposes. Next step needs to be making them into data.
2
1
11
890
We mourn the loss of Senior Lecturer Richard Allen Posner, a longtime #UChicagoLaw scholar, former federal appellate judge, and pioneer in the field of law and economics, who died on November 10. Read more about his legacy: buff.ly/e3c3WdL
98
1,047
394
3,533
641,871
I get a lot of spam email from other law schools, and I just noticed that I had to block dean@law.[school withheld].edu. That really seems suboptimal. Please don't send law school spam from your dean account.
221
Grading presentations, November 2026:
3
3
20
2,210
Matthew Sag retweeted
Dear software designers: I *always* want to open the link in my default browser, and I *always* want to paste the text unformatted. Thank you for your attention to this matter.
56
136
10
5,431
73,842
Matthew Sag retweeted
Had Claude estimate CO2 emissions per dollar by where you spend it, including AI tokens. Not super reliable but if someone wants to take this further this is yet another argument that people should really be trying AI models to see what they can do (and hopefully see that they are in fact quite disturbingly powerful)
9
7
130
6,219
Amen
In which I come out swinging against Canvas. Canvas is why students prefer AI to their professors, and I don't blame them one bit. hollisrobbinsanecdotal.subst…
3
622
Claude will sometimes make very strange choices in an empirical project if you tell it to be cost-conscious about model selection and token spend.
1
4
451
Matthew Sag retweeted
Another blow to legal scholarship's traditional model. It takes 12 - 18 months to write and publish an article. By then the models may have changed too much for your original claims to survive. And model development is accelerating.
How long does a finding about LLMs stay true? A new paper re-tested 50 widely cited findings about LLM behavior (biases, hallucination, sycophancy, etc). Within 15 months, most no longer held or had substantially reduced effects. The shelf life of LLM findings is short. Almost half (44%) of the tested claims are now expired in today’s models, 34% are weakener, 16% persisted, and 6% have strengthened relative to the original claim The only effects that got stronger were about sycophancy, LLM ingroup bias, and creativity. This means we should be cautious when interpreting the long term durability of research on LLMs. zou-group.github.io/llm-clai…
4
1
1
8
1,261
I was astonished to read an amicus brief citing the disgraced whack-a-mole paper in the open AI litigation.
1
5
600
I filed an amicus brief in the OpenAI copyright litigation. If you want to read a summary matthewsag.com/amicus-brief-… but the original is only 6,000 words. matthewsag.com/wp-content/up…
2
2
19
663
Matthew Sag retweeted
FYI, if you are using ChatGPT for bluebooking / citation formatting, it will add a reference to itself in the hyperlink. This makes it look like you used ChatGPT to find the source even if you didn't. It's not visible in the text or CTRL-F but is visible if you click or hover over the link. You can remove it by right clicking, "edit hyperlink" and deleting "&utm_source" and everything that follows it.
5
8
4
185
22,031
Matthew Sag retweeted
A consequential choice will be made soon by government and companies as they are increasingly flooded with agentic AI inquiries that overcome the frictions that gated access (see the graphs - it has already started!) Two options: do you make services worse or better as a result?
267
178
79
1,371
165,675
I had an idea for a start up a couple of weeks ago. Thought it over for a bit. Wrote a plan for the site in a faculty meeting. Built it with @claudeai in a day. Expect to soft launch next week. These are amazing times!
9
237
Here’s a useful test for any AI bill: Could a university lab or two-person startup actually comply with it? If the answer is “only if they have the legal budget of the biggest AI companies in the world,” you’re not making AI safer. You’re entrenching the biggest labs. @neil_chilson
2
17
5
33
4,741