"Take a deep breath and work on this problem step by step" – an innocuous phrase that has proven to be a game-changer in the world of AI. In a groundbreaking study published on September 7, Google DeepMind researchers shed light on how simple prompts can significantly enhance the accuracy of artificial intelligence. The study revealed that when Google’s PaLM 2 AI model was asked to "take a deep breath and work on this problem step by step," it demonstrated an impressive 80% accuracy rate in solving math problems, a stark improvement from the mere 34% without the prompt.
The quest to improve the performance of large language models, such as ChatGPT’s GPT-4 or Google’s PaLM 2, has gained momentum in recent years. This study underscores the potential that lies in using simple prompts, prompting a surge of interest in the field. With over 8,500 math problems used in the study, the researchers aimed to create a scenario that a "bright middle school student" should be able to solve. The results were remarkable, with even the prompt "let’s think step by step" boosting the model’s accuracy to 71%.
Simple Prompts Boost AI Accuracy, Google DeepMind Study Reveals
In a recent study published on September 7, researchers from Google DeepMind explored the potential of simple prompts to enhance the performance of AI models. The study revealed an intriguing finding: the phrase "Take a deep breath and work on this problem step by step" significantly improved the accuracy of Google’s PaLM 2 AI model.
A Leap in Accuracy
The researchers aimed to uncover how simple prompts could potentially enhance the performance of large language models, such as Google’s PaLM 2 or ChatGPT’s GPT-4. The study used a set of over 8,500 math problems, designed to be solvable by a "bright middle school student," as the testing ground.
When the AI was prompted with the phrase "Take a deep breath and work on this problem step by step," the accuracy of the PaLM 2 model soared to 80% in solving these problems. This was a stark improvement from the mere 34% accuracy observed when the model was used without any prompts. Even the phrase "let’s think step by step" led to a commendable increase in accuracy, reaching 71%.
The Power of Prompts
The methodology of the study involved the automation of testing a large number of different phrases across various AI models, with the goal of determining which prompts yielded the most effective results. This study affirms the findings of a joint study in 2022 conducted by researchers at Google and the University of Tokyo, which had found that encouraging large language models to "think step by step" improved their accuracy.
The recent launch of ChatGPT has stirred interest around the optimal way to communicate with AI to achieve desired outcomes. Some companies have started hiring "prompt engineers" who specialize in formulating questions and phrases that refine AI responses. Anna Bernstein, a prompt engineer for AI company Copy.ai, suggests using a thesaurus and paying careful attention to verbs to maximize the effectiveness of prompts.
Sharing Prompts in Communities
The concept of "prompt libraries" has been gaining traction, with groups assembling and sharing effective phrases to optimize AI performance. For instance, OpenAI’s Discord community shares sample phrases to help users get the most out of ChatGPT, such as assistance with job interviews.
Google and the authors of the study have not yet responded to requests for comment from Insider.
Takeaways
This study underscores the power of language and its impact on the functioning of AI. It’s fascinating how seemingly simple phrases can dramatically enhance the performance of AI models when used as prompts. As AI continues to advance and permeate various aspects of our lives, understanding how to communicate effectively with these systems will become increasingly crucial. The rise of roles like "prompt engineers" and the creation of prompt libraries signal a new frontier in our interaction with AI.
