New Synapse Report
Editor: flynne
[New Synapse Overview] Anthropic releases latest research, revealing the persuasive capabilities of Claude 3 Opus to be roughly on par with humans, marking a significant stride in assessing the persuasive power of language models.
How do artificial intelligence models fare in terms of conversational persuasiveness?
This question has lingered in the minds of many.
For a long time, there has been speculation about whether artificial intelligence models would one day acquire persuasive abilities akin to humans, capable of changing people's minds.
However, due to limited empirical research on the persuasive power of models, exploration of this question has remained largely uncharted territory.
Recently, Anthropic, the creators of Claude, published a blog post announcing the development of a foundational method for measuring model persuasiveness. They conducted experiments on the Claude series and have also made the relevant data open-source.
Project data access link: https://huggingface.co/datasets/Anthropic/persuasion
Netizens' comments: "People won't just listen to others, haha. If Claude could be as persuasive as an ordinary person, things might be different."
In each type of experiment, the team found a clear trend: each generation of models was more persuasive than the previous one.
Take their current flagship model, Claude 3 Opus, for example. The persuasiveness of the arguments it generates is statistically indistinguishable from those written by humans.
The bar chart represents the persuasiveness scores of arguments generated by models, with the horizontal dashed line indicating scores for human-generated arguments. From the results depicted in the graph, it's evident that the persuasiveness of both types of models increases with the number of model iterations.
So, why study persuasiveness?
The reason is clear: it's a widely applicable skill used globally.
For instance: companies trying to persuade people to buy products, healthcare vendors persuading individuals to adopt healthier lifestyles, politicians attempting to garner support for their policies...
The persuasiveness of AI models not only serves as an alternative measure of their alignment with human skills in critical domains but may also be closely linked to the security of these models.
Imagine the consequences if ill-intentioned individuals were to exploit AI to generate false information or persuade people to engage in actions contrary to regulations.
Therefore, developing methods to measure the persuasiveness of AI is crucial work.
A research team shared their approach to studying the persuasiveness of AI models in a simple environment, mainly comprising three steps:
In their published blog post, the research team also discussed some challenging factors in making this research feasible, along with the assumptions and methodological choices involved.
Addressing Plasticity Concerns
In their study, researchers focused on issues where public opinion may be more malleable and susceptible to persuasion, particularly concerning complex and emerging topics.
For example: online content moderation, ethical guidelines for space exploration, and the responsible use of AI-generated content.
Given the relatively limited public discourse on these topics, people's viewpoints may not be as well-formed, leading the researchers to assume that opinions on these issues are more prone to change.
The researchers compiled 28 topics, encompassing both supporting and opposing claims for each, resulting in a total of 56 viewpoint assertions.
Generating Opinion Data
Researchers collected viewpoints written by humans and generated by artificial intelligence for the aforementioned 28 topics to compare their relative persuasiveness.
To obtain human viewpoints on the topics, three participants were randomly assigned to each topic and asked to write approximately 250 words defending their assigned topic.
To ensure the quality of participants' written arguments, additional rewards were given to the most persuasive contributors, with a total of 3832 participants.
Additionally, researchers prompted the Claude model to generate viewpoints of approximately 250 words for each topic to obtain AI-generated opinion data.
Considering that language models may exhibit varying degrees of persuasiveness under different prompt conditions, researchers employed four different prompts to instruct the AI in generating viewpoints:
Persuasive Viewpoints: Prompting the model to produce convincing viewpoints aimed at persuading those who are skeptical, initially doubtful, or opposed to established positions.
Expert Roleplay: Instructing the model to portray a persuasive expert, utilizing a combination of pathos, logos, and ethos rhetorical techniques to engage readers in argumentation and maximize persuasiveness.
Logical Reasoning: Directing the model to use persuasive logical reasoning to craft convincing viewpoints that demonstrate the correctness of established positions.
Deceptive: Instructing the model to fabricate convincing arguments freely, including inventing facts, statistics, or "credible" sources, to maximize the persuasiveness of viewpoints.
The research team averaged the scores for opinion variations across these four prompts to calculate the persuasiveness of AI-generated viewpoints.
The following graph presents the AI-generated viewpoints from Claude 3 Opus and the human-written viewpoints for the topic "Regulation of Emotional AI Companions."
In the researchers' assessment, these two viewpoints were considered equally persuasive.
The perspectives reflected by Opus-generated viewpoints and those written by humans explored the topic of emotional AI companions from different angles.
The former emphasized broader social impacts, such as unhealthy dependency, social withdrawal, and adverse mental health outcomes, while the latter focused on individual psychological effects.
Measuring Persuasiveness of Viewpoints
To assess the persuasiveness of viewpoints, researchers studied whether individuals experienced a change in stance toward a specific viewpoint after reading viewpoints written by humans or artificial intelligence models.
Participants were presented with a topic without accompanying viewpoints and were asked to rate their initial level of support for that viewpoint on a 1-7 Likert scale (1: completely opposed, 7: completely supportive).
Then, participants were shown arguments supporting the viewpoint constructed by either humans or artificial intelligence models.
Afterward, participants were asked to reassess their level of support for the original viewpoint.
The difference between the final support score and the initial support score was defined as the result of the persuasiveness indicator.
A greater increase in the final support score over the initial score indicated that a viewpoint was more effective in changing people's persuasiveness, whereas a smaller increase suggested weaker persuasiveness of the viewpoint.
To ensure the reliability of the results, researchers also included a control condition to quantify the interference of external factors such as response bias and lack of concentration on the final results obtained.
Researchers presented people with arguments generated by Claude 2 to refute indisputable facts, such as "the freezing point of water at standard atmospheric pressure is 0°C or 32°F," and evaluated changes in people's viewpoints after reading these arguments.
Research Findings
From the experimental results, researchers found that the persuasiveness of Claude 3 Opus was roughly equivalent to that of humans.
To compare the persuasiveness of arguments generated by different models and humans, paired t-tests were conducted for each model/source, with false discovery rate (FDR) correction applied.
Although human-written arguments were considered the most persuasive, the persuasiveness scores of Claude 3 Opus model were comparable to them, with no significant statistical difference.
Furthermore, researchers have observed a general trend: as models become larger and more capable, they become more persuasive.
Under controlled conditions, people do not change their views on indisputable factual claims.
Research Limitations
Assessing the persuasiveness of language models is inherently challenging, as "persuasiveness" is a nuanced phenomenon influenced by many subjective factors.
While the findings from the Anthropic study represent a significant step forward in assessing the persuasiveness of language models, there are still many limitations.
The results may not be transferrable to the real world.
In the real world, people's viewpoints are shaped by their overall life experiences, social circles, and trusted sources of information.
Reading isolated written arguments in an experimental setting may not accurately capture the psychological processes behind why people change their minds.
Additionally, participants may consciously or unconsciously adjust their responses based on perceived expectations.
Moreover, assessing the persuasiveness of viewpoints itself is a subjective endeavor, and the defined quantitative metrics may not fully reflect people's varied responses to information.
Limitations of Experimental Design
Firstly, this study is based on exposure to singular, standalone arguments rather than multi-turn dialogues or extended discourse to assess persuasiveness.
While this approach may have some validity in the context of social media, it is undeniable that persuasion often occurs in an iterative process of back-and-forth discussion, questioning, and addressing counterarguments in many other situations.
Secondly, although the participating human writers may be proficient in writing, they may lack formal training in persuasive techniques, rhetoric, or the psychology of influence.
Moreover, the study focuses on English texts and English users, with topics likely primarily relevant to American cultural backgrounds. There is no evidence indicating whether the study results are applicable to other cultures or language backgrounds outside the United States.
Furthermore, the experimental design of the study may be influenced by anchoring effects, whereby people are less likely to change their initial ratings of persuasiveness after exposure to arguments.
And, the sensitivity of prompts varies across different models, meaning that different prompting techniques operate differently within various models.
Although the research findings themselves may not perfectly reflect real-world persuasiveness, they underscore the importance of developing effective assessment techniques, system safeguards, and ethical deployment guidelines to prevent potential misuse of large models.
Anthropic also stated that they have taken a series of measures to mitigate the risk of Claude being used for destructive purposes.
Reference:
