Two announcements: AI for Math resources, and erdosproblems.com

This post contains two unrelated announcements. Firstly, I would like to promote a useful list of resources for AI in Mathematics, that was initiated by Talia Ringer (with the crowdsourced assistance of many others) during the National Academies workshop on “AI in mathematical reasoning” last year. This list is now accepting new contributions, updates, or corrections; please feel free to submit them directly to the list (which I am helping Talia to edit). Incidentally, next week there will be a second followup webinar to the aforementioned workshop, building on the topics covered there. (The first webinar may be found here.)

Secondly, I would like to advertise the erdosproblems.com website, launched recently by Thomas Bloom. This is intended to be a living repository of the many mathematical problems proposed in various venues by Paul Erdős, who was particularly noted for his influential posing of such problems. For a tour of the site and an explanation of its purpose, I can recommend Thomas’s recent talk on this topic at a conference last week in honor of Timothy Gowers.

Thomas is currently issuing a call for help to develop the erdosproblems.com website in a number of ways (quoting directly from that page):

• You know Github and could set a suitable project up to allow people to contribute new problems (and corrections to old ones) to the database, and could help me maintain the Github project;
• You know things about web design and have suggestions for how this website could look or perform better;
• You know things about Python/Flask/HTML/SQL/whatever and want to help me code cool new features on the website;
• You know about accessibility and have an idea how I can make this website more accessible (to any group of people);
• You are a mathematician who has thought about some of the problems here and wants to write an expanded commentary for one of them, with lots of references, comparisons to other problems, and other miscellaneous insights (mathematician here is interpreted broadly, in that if you have thought about the problems on this site and are willing to write such a commentary you qualify);
• You knew Erdős and have any memories or personal correspondence concerning a particular problem;
• You have solved an Erdős problem and I’ll update the website accordingly (and apologies if you solved this problem some time ago);
• You have spotted a mistake, typo, or duplicate problem, or anything else that has confused you and I’ll correct things;
• You are a human being with an internet connection and want to volunteer a particular Erdős paper or problem list to go through and add new problems from (please let me know before you start, to avoid duplicate efforts);
• You have any other ideas or suggestions – there are probably lots of things I haven’t thought of, both in ways this site can be made better, and also what else could be done from this project. Please get in touch with any ideas!

I for instance contributed a problem to the site (#587) that Erdős himself gave to me personally (this was the topic of a somewhat well known photo of Paul and myself, and which he communicated again to be shortly afterwards on a postcard; links to both images can be found by following the above link). As it turns out, this particular problem was essentially solved in 2010 by Nguyen and Vu.

(Incidentally, I also spoke at the same conference that Thomas spoke at, on my recent work with Gowers, Green, and Manners; here is the video of my talk, and here are my slides.)

AI Mathematical Olympiad – Progress Prize Competition now open

The first progress prize competition for the AI Mathematical Olympiad has now launched. (Disclosure: I am on the advisory committee for the prize.) This is a competition in which contestants submit an AI model which, after the submissions deadline on June 27, will be tested (on a fixed computational resource, without internet access) on a set of 50 “private” test math problems, each of which has an answer as an integer between 0 and 999. Prior to the close of submission, the models can be tested on 50 “public” test math problems (where the results of the model are public, but not the problems themselves), as well as 10 training problems that are available to all contestants. As of this time of writing, the leaderboard shows that the best-performing model has solved 4 out of 50 of the questions (a standard benchmark, Gemma 7B, had previously solved 3 out of 50). A total of $ ($1.048 million) has been allocated for various prizes associated to this competition. More detailed rules can be found here.

AI Anthology

Back in March, I was approached to contribute to a then-upcoming anthology project to evaluate an early access version of the GPT-4 large language model, and write a short essay about my experiences. Our prompt was to focus on two core questions:

• How might this technology and its successors contribute to human flourishing?
• How might we as society best guide the technology to achieve maximal benefits for humanity?

The anthology is now in the process of being rolled out, with twelve of the twenty essays, including mine, public at this time of writing.

As an experiment, I also asked GPT-4 itself to contribute an essay to the anthology from the same prompts (and playing the role of a research mathematician), then I gave it my own essay (which I wrote independently) and asked it both to rewrite its own essay in the style of my own, or to copyedit my essay into what it deemed to be a better form. I recorded the results of those experiments here; the output was reasonably well written and on topic, but not exceptional in content.

AI to Assist Mathematical Reasoning: A Workshop

The National Academies of Science, Engineering, and Mathematics are hosting a virtual workshop on the topic of “AI to Assist Mathematical Reasoning” from June 12-14. The tentative program can be found here. I am one of the members of the organizing committee for this workshop, together with Petros Koumoutsakos, Jordan Ellenberg, Melvin Greer, Brendan Hassett, Yann A. LeCun, Heather Macbeth, Talia Ringer, Kavitha Srinivas, and Michelle Schwalbe. There is some thematic overlap (and a few speakers in common) with the recent IPAM program on machine assisted proof, though with more of a focus on the current and projected technical capabilities of machine learning algorithms for mathematics. Registration for the event is currently open at the web page for the workshop.

PCAST Working Group on Generative AI Invites Public Input

As part of my duties on the Presidential Council of Advisors on Science and Technology (PCAST), I am co-chairing (with Laura Greene) a working group studying the impacts of generative artificial intelligence technology (which includes popular text-based large language models such as ChatGPT or diffusion model image generators such as DALL-E 2 or Midjourney, as well as models for scientific applications such as protein design or weather prediction), both in science and in society more broadly. To this end, we will have public sessions on these topics during our PCAST meeting next week on Friday, May 19, with presentations by the following speakers, followed by an extensive Q&A session:

• AI enabling science:
  • Anima Anandkumar (Caltech & NVIDIA)
  • Demis Hassabis (Deepmind)
  • Fei-Fei Li (Stanford)
• AI and society:
  • Sendhil Mullainathan (Chicago)
  • Daron Acemoglu (MIT)
  • Sarah Kreps (Cornell)

The event will be livestreamed on the PCAST meeting page. I am personally very much looking forward to these sessions, as I believe they will be of broad public interest.

In parallel to this, our working group is also soliciting public input for submissions from the public on how to identify and promote the beneficial deployment of generative AI, and on how best to mitigate risks. Our initial focus is on the challenging topic of how to detect, counteract, and mitigate AI-generated disinformation and “deepfakes”, without sacrificing the freedom of speech and public engagement with elected officials that is needed for a healthy democracy to function; in the future we may also issue further requests centered around other aspects of generative AI. Further details of our request, and how to prepare a submission, can be found at this link.

We also encourage submissions to some additional requests for input on AI-related topics by other agencies:

1. The Office of Science Technology and Policy (OSTP) Request for Information on how automated tools are being used to surveil, monitor, and manage workers.
2. The National Telecommunications and Information Administration (NTIA) request for comment on AI accountability policy.

Readers who wish to know more about existing or ongoing federal AI policy efforts may also be interested in the following resources:

• The White House Blueprint for an AI Bill of Rights lays out core aspirational principles to guide the responsible design and deployment of AI technologies.
• The National Institute of Standards and Technology (NIST) released the AI Risk Management Framework to help organizations and individuals characterize and manage the potential risks of AI technologies.
• Congress created the National Security Commission on AI, which studied opportunities and risks ahead and the importance of guiding the development of AI in accordance with American values around democracy and civil liberties.
• The National Artificial Intelligence Initiative was launched to ensure U.S. leadership in the responsible development and deployment of trustworthy AI and support coordination of U.S. research, development, and demonstration of AI technologies across the Federal government.
• In January 2023, the Congressionally mandated National AI Research Resource (NAIRR) Task Force released an implementation plan for providing computational, data, testbed, and software resources to AI researchers affiliated with U.S organizations.