ACC Seminar: Applications of Transformers in Mathematics

ABSTRACT

Since the introduction of the transformer architecture in 2017 [Google DeepMind], large language models (LLMs) have a meteoric rise. We will discuss how the transformer works and state of the art methods for incentivizing reasoning by LLMs [Deepseek]. We will discuss some of the recent successes that LLMs have had in solving mathematics problems as well as the difficulties that still exist [DeepMind et al.]. Finally, we will discuss a result from March 2026 which proves that there are theoretical limitations to embedding-based retrieval [DeepMind] and pivot toward other machine learning methods that have contributed to recent mathematical discoveries.

BIOGRAPHY

Paul Schwartz received his PhD from the University of Florida in 2022 under the direction of Kevin Keating. He currently serves as a Lecturer of Mathematics at Stevens Institute of Technology.

Attendance: This is a technical talk open to all.
A campus map is available at https://tour.stevens.edu.
Additional information is available at https://web.stevens.edu/algebraic/.