This essay also appears on Proofs and Prompts.
Three years ago, AI systems could not reliably add two numbers. A year ago, internal models at OpenAI and DeepMind received the equivalent of a gold-medal score on the IMO. Now, these systems are autonomously resolving major open questions. It’s hard to imagine this trend continuing for another year, but I expect it will. It is clear that this will require a radical rethinking of our profession.
A few weeks ago, I gave a talk titled The End of Mathematics. If you only read the title1, you might guess that this talk was about how, soon, AI will “solve” math. That’s not what it was about. The talk instead laid out a gloomy vision of the future, in which, despite the possibility of AI systems that are robustly superhuman at mathematics, the design of our institutions causes human understanding of mathematics, and possibly even mathematical progress in the abstract, to stall. I think we will avoid this future, but I also think it is plausibly the default if academic mathematics does not adapt. Despite my relative enthusiasm for the use of AI to do mathematics, I share this view with many of its detractors.
Here I want to lay out, instead, a positive vision of the future of mathematics, and the human practice of mathematics. I claim we can deepen human understanding even as the production of interesting mathematics becomes less dependent on it.
This essay will take as a premise that AI systems that are robustly superhuman at most or all aspects of mathematics will be here soon. But the concrete changes to our institutions I propose only require accepting the weaker premise that the production of mathematical text is becoming increasingly disconnected from mathematical understanding.
What are we even trying to do here?
I think it has now become clear that there is no consensus in the mathematical community as to what our goals are. Some of us want to solve problems; some of us think of mathematics as play or as poetry. For some: “ Wir müssen wissen – wir werden wissen.”2 Some of us think we are penetrating the mysteries of the platonic realm. Some of us think the goal is to embody love of and understanding of mathematics,3 and to transmit that love and understanding to the next generation.
My personal, if self-referential, answers are:
-
We’re trying to produce and understand high quality mathematics.
-
We’re trying to produce high quality mathematicians.
These goals should be construed broadly. What high quality mathematics consists of has changed quite dramatically over time; we come to its definition as a community. We are not just training PhD students to do research in mathematics. A substantial part of our job, though perhaps an underemphasized one, is to educate the general public about high quality mathematics and mathematical thinking.4
Whatever our goals are, we’ve operationalized them primarily through proving theorems. Almost all papers or PhD theses have a main theorem, and ostensibly a proof of it. But it should be clear that the goal of mathematics is not to prove theorems; if it was, it would be trivial to automate. A computer or monkey could easily start at the axioms of ZFC and iteratively apply deduction rules to them, with no attention whatsoever paid to their meaning. It has had particular significance when a theorem resolves an open problem, especially one that has resisted substantial effort. Again this is easily automated; our computer or monkey can simply conjecture all mathematical propositions in alphabetical order.
The general attitude of our community towards a technology that can prove theorems and solve open problems suggests that these operationalizations of our values are at best incomplete.
The prospect of automating mathematics by enumerating all conjectures, and all proofs of ZFC, is probably not so disturbing to you. But let us for a moment assume the computer or monkey is very smart; perhaps it understands the results it is proving, and writes beautiful expositions thereof. Perhaps it has a good sense of what we find interesting, and is primarily focusing on those questions. Perhaps it has, in the course of enumerating theorems of ZFC, answered many of our most pressing open questions, and is asking many more fundamental open questions. Is there still a need for human mathematicians?
I think so. This machine might produce answers we value, but it would not, in itself, produce human understanding of those answers. In fact I think we are at the beginning of an incredible, wonderful explosion of mathematics, and if we value human understanding, there will be more need for human mathematicians than ever before. But the profession will have to change.
In the course of this change, we will have to decide what to hold on to and what to throw away. Some things I would like to preserve: learning seminars; serendipitous conversations that spark an idea; students knocking on a professor’s door to chat about math. A robust community learning exciting new mathematics. Thousands of people that, together, slowly start to resolve their confusion.
I worry that much of what has been written on this topic, including some of my own past writing, focuses too much on trying to preserve the precise shape of the institutions of academic mathematics, rather than our values. How can we preserve the journal and peer review system?5 How can we protect the arXiv? How can we keep our role as gatekeepers? If you have internalized the fact that existing AI systems can produce relatively high quality results for the marginal cost of a few dollars, the idea that any semblance of the current equilibrium can survive what’s coming is absurd.
As we try to find a new equilibrium, we could try to chase the edge of model capabilities. Right now AI systems arguably underperform us at theory-building, asking questions, exposition, … so we could prioritize and reward those skills. I think this is unwise: compare the speed at which the academy adapts to the speed at which model capabilities improve. We need to consider the endgame. If the models remain incapable in some domain, we can adjust later.
Before I propose some relatively concrete steps we can take, let me remark on what we’re trying to protect mathematics from. There is a lot of anger at AI labs, and certain individuals at those labs. But whatever our judgment of the labs, we need a plan that does not depend on AI capabilities disappearing. The basic issue is not the labs’ behavior, ethical or not.6 It’s the technology itself. I think there is some belief that the labs will “move on” from math next year, be nationalized or broken up, or that a financial bubble will pop, somehow returning things to normal, or… But there is no way our institutions can survive unchanged when anyone with a laptop and a few hundred dollars can generate what would have been an Annals paper last year. AI does not care if you are anti-AI.
Producing high-quality mathematicians
The most urgent question our profession needs to answer right now is: what should our students be doing? It’s now possible to produce a PhD thesis one hasn’t even read; in terms of demonstrating understanding, mathematical text is worth the paper it is printed on.7 The value of the text no longer reliably conveys a signal about the person who produced it.
In my view we should welcome interesting mathematical results regardless of provenance. But our institutions have historically relied on the same signal to indicate both mathematical progress and mathematical expertise. These now must be distinguished.
I propose the following reconceptualization of the goal of a mathematics PhD: to become a world expert on some interesting, deep topic, and to be able to convey that interest and understanding to others. Part of operationalizing this might be a thesis, but the degree would be awarded primarily on the basis of a rigorous defense, in which the student explains the topic to their examiners until they are satisfied. While we might require the topic to be original, its provenance—AI or not—is irrelevant.8
How different would this look from current PhDs? I think students would still meet with an advisor, who might suggest a topic. That topic could be explored with AI assistance, or not, but the student would be responsible for understanding it; it might be much more open-ended and larger than the typical PhD is currently. The student would be trained to ask interesting questions and try to resolve them, by whatever means. To keep students on track, there might be regular meetings in which the student is asked to independently work through an unfamiliar example, apply a technique in a new case, etc.
The allocative aspects of our job (hiring, graduate admissions, etc.) are in dire need of reform if we want to retain human mathematical expertise. Broadly speaking I think we should focus on rewarding skill in the parts of our jobs that cannot be automated: the internal (e.g. understanding mathematics) and social-relational parts, and operationalizations that hew as closely to those aspects of the profession as possible. For example, talks and sustained mathematical discussion now demonstrate understanding much better than papers. Once AI systems improve at exposition and “digestion,” this will be even more the case. We already interview faculty hires; we must now do the same for graduate admissions.
I think we should try to foster a robust seminar culture in which speakers are expected to explain their topic to the audience’s satisfaction. Much has been written recently (by myself among others) about the fact that we are primarily interested in understanding, not merely the truth value of mathematical statements. If that is the case, let us make sure we actually understand each other.
Right now the use of AI systems to do mathematics above some minimum bar relies on the fact that our community has produced many open conjectures, whose interest is evidenced by the existence of human mathematicians who care about them.9 The recent importance of this fact suggests to me our community plays a very important function that we have, arguably, underrated: namely, figuring out what is interesting. It is not entirely clear to me how to operationalize this, but one possibility might be to reward the construction of research programs (either with help from AI systems or otherwise) that persuade others of their worthiness.
To be clear, I am not saying that AI systems will not be able to ask interesting questions, make interesting conjectures, pursue interesting programs, and so on. I think they most likely will, resulting in the production of an abundance of PDFs. The contents of some of those PDFs may even have important applications. But others will primarily be of interest because they tell us something fundamental about basic mathematical objects, and accrue value only if we can and do engage with them. It seems to me that it will be up to us to build a community of researchers to do so, and we should reward mathematicians who do. And even if the AI is asking excellent questions, there is no reason to think it will ask the same questions we would.
All of these changes are oriented towards increasing the amount we talk to each other about mathematics. It seems to me that this would be positive even in a world with no AI.
I think there is room in this world both for mathematicians who, like me, are enthusiastic about AI, and for those who do not use it. But as the models begin to produce huge quantities of mathematics, it will not be possible to avoid their outputs entirely.
Producing high-quality mathematics
As we think about how to reshape our profession, it’s important to understand that, whether one likes it or not,10 it’s impossible to stop people, amateur or professional, from pushing a button to produce mathematics. The idea that we will persuade people not to play around with math, or that we will be able to “reserve” problems for graduate students, is just not realistic.11 And we shouldn’t want to do this!
There is now more interest in math than at any other time in history. We should be ecstatic for mathematics’s sake, even as we are concerned about mathematicians and mathematical expertise. And by and large, the value of this button-pressing comes from the mathematical community. If a conjecture falls in the woods and no one is around to hear it, who cares?12 For the abundance of new mathematics to have value outside application, we will need an abundance of new mathematicians. And for results with applications, we will want people to be capable of understanding their assumptions and consequences.
I wrote above that solving problems and resolving open conjectures is an incomplete operationalization of our values. But nonetheless it is important to solve problems and resolve conjectures! The provenance of such solutions only matters insofar as it intersects with the existing structure of the profession (incentives, prestige, and so on). It is obvious that structure needs to change in any case.
Mathematics used to be the cheapest of the sciences. I think the biggest change we are facing is that now, some portion of our questions will be answerable via a cash injection. I know some of my colleagues find this distressing. Previously those questions might have brought together a research community, led to interesting auxiliary developments, and so on. This contingent progress may now no longer occur.
But don’t you believe in mathematics!? There will always be more to learn. If a basic question can be resolved for the cost13 of a nice dinner, we should be delighted. But that’s only the beginning. We will ask what the answer explains, and what it helps us understand. It will lead to many more new questions, some of which can in turn be resolved for the cost of a nice dinner, and others which renew our confusion and lead to the development of a research community.
Our industrious new helpers will be churning out an unbelievable amount of math, pursuing our interests or perhaps their own. We will have our own questions, and confusions; sometimes they will be resolved by the models, and sometimes they won’t. Sometimes the answers will be complicated, and we’ll devote a learning seminar to them. Sometimes progress will be minimal, but the question itself will be so motivating it gives rise to a research community.
A student will be confused. They will knock on their professor’s door. Maybe the two of them will ask a model for help, or maybe not, but first they might spend some time at the blackboard thinking through the question. And the model might give them a beautiful explanation, but we all know that’s not enough; no one can understand mathematics for us. We have got to do the work.
There is so much more to learn—an infinite amount. We’ve always been at the beginning, and we always will be.
Acknowledgments
I am grateful for comments from Mohammed Abouzaid, alz, Boaz Barak, Frank Calegari, Ben Church, Jennifer Cutler, doomslide, Elden Elmanto, Francesco Fournier-Facio, Tony Feng, Dan Freed, Peli Grietzer, Michael Groechenig, Stephanie Koh, Joshua Lam, Mark Sellke, Ravi Vakil, and Amal Vayalinkal.
-
I regret choosing this title. ↩
-
Hilbert’s full opinion is as relevant today as ever: ‘We must not believe those, who today, with philosophical bearing and deliberative tone, prophesy the fall of culture and accept the ignorabimus. For us there is no ignorabimus, and in my opinion none whatever in natural science. In opposition to the foolish ignorabimus our slogan shall be Wir müssen wissen – wir werden wissen (“We must know – we will know”).’ ↩
-
I owe this phrasing to Peli Grietzer. ↩
-
Note that this list consists mostly of internal and social-relational functions (understanding, coming to a determination of what’s interesting, training, and so on). This is in contrast to our operationalizations (proving theorems, solving problems, etc.). ↩
-
This system was already close to breaking before AI; it is overdue for radical reform. ↩
-
Obviously some of it has not been ethical. But even if every lab had behaved perfectly, the capabilities of AI systems would still force us to radically adapt our institutions. ↩
-
Which is not to say the text is necessarily uninteresting. ↩
-
This is a practical necessity. There is no way to enforce restrictions on provenance, and attempting to do so will only create incentives to conceal use of AI. But I find it unlikely that someone whose only contribution was to push a button, and who did not engage deeply with the material, would be able to pass a rigorous defense. ↩
-
To be clear, many open conjectures are less interesting than one might have hoped, post hoc, and are generally not an end in themselves. They are often meant to measure our failure to understand some object, but they are sometimes resolved without improving that understanding. ↩
-
On balance, I think I like it, though I am sometimes annoyed to find slop PDFs in my inbox. It took me some time to understand that these PDFs expressed a need for understanding; a person elicited them, often without being able to meaningfully engage with their contents, and needed to know that someone could engage, and that someone cared. ↩
-
That we cannot reserve a problem for a graduate student does not mean we can’t give them the opportunity to work on it. This is compatible with the reconceptualization of a PhD outlined previously. ↩
-
Some have suggested that interest in using AI to answer mathematical questions may soon fade. It is hard for me to see how this will happen as long as questions we care about remain unanswered. ↩
-
By this I mean marginal cost. Michael Groechenig points out to me that it is unclear that we should directly compare the cost of a machine proving a theorem to the cost of a human doing so, as the products of this work are arguably different. Only one of them produces understanding and expertise in a human being, which I think we might value independent of the result itself. ↩
