Auditing the impact of social media’s policy shift on anti-vaccine discourse: A large language model-driven empirical study

The sudden termination of X’s (formerly Twitter) misinformation policy on November 23, 2022, provides an opportunity to assess the effects of lifting content moderation restrictions on vaccine-related discourse. This study examines changes in the prevalence, thematic composition, and engagement of anti-vaccine discourse following X’s policy shift, analyzing tweets from a seven-day period before and after the policy termination (November 16–30, 2022), excluding the announcement date itself from regression analyses. Using GPT-4o for stance classification, thematic categorization, and stance consistency assessment, with validation through external benchmarks and cross-annotator agreement, we find that anti-vaccine tweets increased significantly post-policy (OR = 1.60, 95% CI: 1.50–1.72), particularly via retweets, suggesting content amplification. Sensitivity analyses excluding highly retweeted content revealed that the policy change was also associated with increased creation of new anti-vaccine content. Thematically, health concerns over vaccination became more prominent, while conspiracy-related and anti-mandate narratives declined in relative prevalence. Stance consistency in quote tweets increased, indicating reinforced ideological alignment in anti-vaccine discourse. These results suggest that content moderation policies may constrain both the volume and amplification of anti-vaccine content, with policy removal associated with rapid shifts in discourse patterns.