Trust the Crowd-wisdom or AI? Research on the Impact of Human-AI Integration Fact Checking on User Information Engagement Behavior

Wu Lianren; Hu Yanan; Li Jinjie; Qi Jiayin

doi:10.13266/j.issn.2095-5472.2025.020

PDF(1723 KB)

Knowledge Management Forum ›› 2025, Vol. 10 ›› Issue (4) : 309-320. DOI: 10.13266/j.issn.2095-5472.2025.020 CSTR: 32306.14.CN11-6036.2025.020

Trust the Crowd-wisdom or AI? Research on the Impact of Human-AI Integration Fact Checking on User Information Engagement Behavior

Wu Lianren ¹ ,
Hu Yanan ² ,
Li Jinjie ³ ,
Qi Jiayin ¹

Author information +

History +

Abstract

[Purpose/Significance] Fact-checking is an effective strategy to combat the dissemination of misinformation. Exploring the mechanism of how fact-checking types (crowdsourced and AI fact-checking) influence users' information participation behavior can help social platforms improve and optimize their measures for misinformation governance. [Method/Process] Based on real data from social media platforms, this study used analysis of variance to investigate the impact of fact-checking on users' information participation behavior, while considering the moderating effect of source credibility. [Result/Conclusion] The empirical results show that crowdsourced fact-checking significantly affects users' information engagement behavior, with positive fact-checking promoting information participation behavior and negative fact-checking inhibiting it. AI fact-checking positively influences users' information engagement behavior, and the combination of AI and crowdsourced fact-checking also positively affects users' information engagement behavior. Source credibility positively moderates the relationship between crowdsourced fact-checking and information engagement behavior, as well as the relationship between AI fact-checking and information engagement behavior. This study reveals the impact of different types of fact-checking and their integration on information engagement behavior, providing theoretical support for social media platforms to fully utilize collective intelligence and AI for fact-checking and misinformation governance.

Key words

human-AI integration / crowdsourced fact-checking / AI fact-checking / information engagement behavior / source credibility

Cite this article

EndNote

Ris (Procite)

Bibtex

Download Citations

Wu Lianren , Hu Yanan , Li Jinjie , et al. Trust the Crowd-wisdom or AI? Research on the Impact of Human-AI Integration Fact Checking on User Information Engagement Behavior[J]. Knowledge Management Forum. 2025, 10(4): 309-320 https://doi.org/10.13266/j.issn.2095-5472.2025.020

References

List( Publishing order | Descend order by publishing year | Descend order by cited within ) Chart analysis

[1]

张新生, 王润周, 马玉龙. AIGC背景下虚假信息治理挑战、机会与策略研究 [J/OL]. 情报科学, 1-23[2025-06-22]. http://kns.cnki.net/kcms/detail/22.1264.G2.20241111.1002.024.html.

http://kns.cnki.net/kcms/detail/22.1264.G2.20241111.1002.024.html

ZHANG X S, WANG

L Z

, MA

Y L

. Challenges, opportunities and strategies of misinformation governance at AIGC background[J]. Information science, 2025: 1-23.

Cited in this article [1]

[2]	ALLEN J, ARECHAR A A, PENNYCOOK G, et al. Scaling up fact-checking using the wisdom of crowds[J]. Science advances, 2021, 7(36): eabf4393. Cited in this article [1]

[3]	RUGGERI K, STOCK F, HASLAM S A, et al. A synthesis of evidence for policy from behavioural science during COVID-19[J]. Nature, 2024, 625(7993): 134-147. Cited in this article [1]

[4]	BODE L, VRAGA E K, TULLY M. Correcting misperceptions about genetically modified food on social media: examining the impact of experts, social media heuristics, and the gateway belief model[J]. Science communication, 2021, 43(2): 225-251. Cited in this article [1]

[5]	HUANG G, WANG S. Is artificial intelligence more persuasive than humans? a meta-analysis[J]. Journal of communication, 2023, 73(6): 552-562. Cited in this article [1]

[6]	王喆诚, 于瑞峰. 社会存在、警惕性及信息卷入度对个体事实核查的影响[J]. 心理技术与应用, 2023, 11(1): 42-49. WANG Z C, YU R F. The impact of social presence, vigilance, and information involvement on individual fact - checking[J]. Psychological techniques and applications, 2023, 11(1): 42-49. Cited in this article [1]

[7]	HANNAK A, MARGOLIN D, KEEGAN B, et al. Get back! you don’t know me like that: the social mediation of fact checking interventions in Twitter conversations[J]. Proceedings of the international AAAI conference on Web and social media, 2014, 8(1): 187-196. Cited in this article [2]

[8]	PILARSKI M, SOLOVEV K O, PRÖLLOCHS N. Community notes vs. snoping: how the crowd selects fact-checking targets on social media[J]. Proceedings of the International AAAI conference on Web and social media, 2024, 18(1): 1262-1275. Cited in this article [1]

[9]	LIU X, QI L, WANG L, et al. Checking the fact-checkers: the role of source type, perceived credibility, and individual differences in fact-checking effectiveness[J]. Communication research, 2023, 6(7): 00936502231206419. Cited in this article [1]

[10]	CHUAI Y W, TIAN H Y, PRLLOCHS Net al. Did the roll-out of community notes reduce engagement with misinformation on X/Twitter? [J]. Proceedings of the ACM on human-computer interaction, 2024, 428(8): 1-52. Cited in this article [1]

[11]	GUO Z, SCHICHTKRULL M, VLACHOS A. A survey on automated fact-checking [J]. Transactions of the Association for Computational Linguistics, 2022, 10(2): 178-206. Cited in this article [1]

[12]	ZHOU X, ZAFARANI R. A survey of fake news: fundamental theories, detection methods, and opportunities[J]. ACM computing surveys, 2021, 53(5): 1-40. Cited in this article [1]

[13]	管佖路, 顾理平. 价值冲突与治理出路：虚假信息治理中的人工智能技术研究[J]. 新闻大学, 2022(3): 61-75, 119. GUAN B L, GU L P. Value conflict and governance solution: examining the role of artificial intelligence in tackling misinformation[J]. Journalism research, 2022(3): 61-75, 119. Cited in this article [1]

[14]	MOLINA M D, SUNDAR S S. Does distrust in humans predict greater trust in AI? role of individual differences in user responses to content moderation [J]. New media & society, 2024, 26(6): 3638-3656. Cited in this article [1]

[15]	QUELLE D, BOVET A. The perils and promises of fact-checking with large language models[J]. Frontiers in artificial intelligence, 2024, 7(3): 1341697. Cited in this article [1]

[16]	AUGENSTEIN I, BALDWIN T, CHA M, et al. Factuality challenges in the era of large language models and opportunities for fact-checking[J]. Nature machine intelligence, 2024, 6(8): 852-863. Cited in this article [1]

[17]	KUZNETSOVA E, MAKHORTYKH M, VZIATYSHEVA V, et al. In generative AI we trust: can chatbots effectively verify political information?[J]. Journal of computational social science, 2024, 8(1): 15-23. Cited in this article [1]

[18]	DEVERNA M R, YAN H Y, YANG K C, et al. Fact-checking information from large language models can decrease headline discernment[J]. Proceedings of the National Academy of Sciences, 2024, 121(50): e2322823121. Cited in this article [1]

[19]	SWIRE-THOMPSON B, LAZER D. Public health and online misinformation: challenges and recommendations[J]. Annual review of public health, 2020, 41(1): 433-451. Cited in this article [1]

[20]	CHUNG M, KIM N. When I learn the news is false: how fact-checking information stems the spread of fake news via third-person perception[J]. Human communication research, 2021, 47(1): 1-24. Cited in this article [2]

[21]	SMITH C N, SEITZ H H. Correcting misinformation about neuroscience via social media[J]. Science communication, 2019, 41(6): 790-819. Cited in this article [1]

[22]	YORK C, PONDER J D, HUMPHRIES Z, et al. Effects of fact-checking political misinformation on perceptual accuracy and epistemic political efficacy[J]. Journalism & mass communication quarterly, 2020, 97(4): 958-980. Cited in this article [1]

[23]	WINTERSIECK A, FRIDKIN K, KENNEY P. The message matters: the influence of fact-checking on evaluations of political messages[J]. Journal of political marketing, 2021, 20(2): 93-120. Cited in this article [1]

[24]	ZHANG J, FEATHERSTONE J D, CALABRESE C, et al. Effects of fact-checking social media vaccine misinformation on attitudes toward vaccines[J]. Preventive medicine, 2021, 145(8): 106408. Cited in this article [1]

[25]	CAREY J M, GUESS A M, LOEWEN P J, et al. The ephemeral effects of fact-checks on COVID-19 misperceptions in the United States, Great Britain and Canada[J]. Nature human behaviour, 2022, 6(2): 236-243. Cited in this article [1]

[26]	WALTER N, BROOKS J J, SAUCIER C J, et al. Evaluating the impact of attempts to correct health misinformation on social media: a meta-analysis[J]. Health communication, 2021, 36(13): 1776-1784. Cited in this article [1]

[27]

王钰昱, 夏志杰. 基于事实核查标签的众包方法应对虚假信息的效果差异及认知机制研究[J]. 知识管理论坛, 2024, 9(3): 330-340.

WANG

Y Y

, XIA

Z J

. Research on the difference in the effect of crowdsourcing methods based on fact - checking labels to deal with false information and its cognitive mechanism[J]. Knowledge management forum, 2024, 9(3): 330-340.

Cited in this article [1]

[28]

李瑾颉, 聂凯伦, 吴联仁, 等. 众包事实核查对信息参与行为的影响：基于来源可信度的调节 [J]. 知识管理论坛, 2024, 9(4): 367-379.

J J

, NIE

K L

, WU

L R

, et al. The impact of crowdsourced fact-checking on user information engagement behavior: the moderating of source credibility[J]. Knowledge management forum, 2024, 9(4): 367-379.

Cited in this article [1]

[29]	SU Y, LEE D K L, XIAO X. “I enjoy thinking critically, and I’m in control”: examining the influences of media literacy factors on misperceptions amidst the COVID-19 infodemic[J]. Computers in human behavior, 2022, 128(9): 107111. Cited in this article [1]

[30]	ARUGUETE N, BACHMANN I, CALVO E, et al. Truth be told: how “true” and “false” labels influence user engagement with fact-checks[J]. New media & society, 2023, 9(2): 14614448231193709.

[31]	PANIZZA F, RONZANI P, MORISSEAU T, et al. How do online users respond to crowdsourced fact-checking?[J]. Humanities and social sciences communications, 2023, 10(867): 1-11. Cited in this article [1]

[32]	NI Z, QIAN Y, CHEN S, et al. Scientific evidence and specific context: leveraging large language models for health fact-checking[J]. Online information review, 2024, 48(7): 1488-1514. Cited in this article [1]

[33]	BANAS J A, PALOMARES N A, RICHARDS A S, et al. When machine and bandwagon heuristics compete: understanding users’ response to conflicting AI and crowdsourced fact-checking[J]. Human communication research, 2022, 48(3): 430-461. Cited in this article [2]

[34]	SPINA D, SANDERSON M, ANGUS D, et al. Human-AI cooperation to tackle misinformation and polarization[J]. Communications of the ACM, 2023, 66(7): 40-45. Cited in this article [1]

[35]	ABBASS H A. Social integration of artificial intelligence: functions, automation allocation logic and human-autonomy trust[J]. Cognitive computation, 2019, 11(2): 159-171. Cited in this article [1]

[36]	MOLINA M D, SUNDAR S S. When AI moderates online content: effects of human collaboration and interactive transparency on user trust[J]. Journal of computer-mediated communication, 2022, 27(4): zmac010. Cited in this article [1]

[37]	MA H V, HUANG W W, DENNIS A R. Unintended consequences of disclosing recommendations by artificial intelligence versus humans on true and fake news believability and engagement[J]. Journal of management information systems, 2024, 41(3): 616-644. Cited in this article [1]

[38]	SUSSMAN S, SIEGAL W. Informational influence in organizations: an integrated approach to knowledge adoption[J]. Information systems research, 2003, 14(3): 47-65. Cited in this article [1]

[39]	NEKMAT E, GOWER K K, ZHOU S, et al. Connective-collective action on social media: moderated mediation of cognitive elaboration and perceived source credibility on personalness of source[J]. Communication research, 2019, 46(1): 62-87. Cited in this article [1]

[40]	MOORE D L, HAUSKNECHT D, THAMODARAN K. Time compression, response opportunity, and persuasion[J]. Journal of consumer research, 1986, 13(1): 85-99. Cited in this article [1]

[41]	叶凤云, 常琳, 秦琴, 等. 基于虚假信息特征的社交媒体大学生用户感知信任研究 [J]. 情报理论与实践, 2024, 47(10): 118-127. YE F Y, CHANG L, QI Q, et al. Research of social media college students’ perceived trust on false information features[J]. Information studies: theory & application, 2024, 47(10): 118-127. Cited in this article [1]

[42]	LUO C, LUO X R, SCHATZBERG L, et al. Impact of informational factors on online recommendation credibility: the moderating role of source credibility[J]. Decision support systems, 2013, 56(8): 92-102. Cited in this article [1]

[43]	ZHANG Z, ZHANG Z, LIU S, et al. Are high-status reviewers more likely to seek anonymity? Evidence from an online review platform[J]. Journal of retailing and consumer services, 2024, 78(6): 103792.

[44]	PANG M S, PAVLOU P A. Cure or poison? Identity verification and the posting of fake news on social media[M]//Fake news on the internet. London: Taylor & Francis Group, Routledge, 2023: 119-146.

[45]	HWANG E H, LEE S. A nudge to credible information as a countermeasure to misinformation: evidence from Twitter[J]. Information systems research, 2025, 36(1): 621-636. Cited in this article [1]

[46]	PENNYCOOK G, EPSTEIN Z, MOSLEH M, et al. Shifting attention to accuracy can reduce misinformation online[J]. Nature, 2021, 592(7855): 590-595. Cited in this article [2]

[47]	DEVERNA M R, YAN H Y, YANG K C, et al. Fact-checking information from large language models can decrease headline discernment[J]. Proceedings of the National Academy of Sciences, 2024, 121(50): e2322823121. Cited in this article [1]

[48]	XUE H, ZHANG J, SHEN C, et al. The majority of fact-checking labels in the United States are intense and this decreases engagement intention[J]. Human communication research, 2024, 50(4): 530-544. Cited in this article [1]

吴联仁：研究选题与框架拟定，论文理论部分撰写与修改；

胡亚男：数据收集与分析；

李瑾颉：论文实证部分撰写与修改；

齐佳音：论文修改与指导。

Funding

general program of National Natural Science Foundation of China titled “Research on the Nudging Governance of Infodemic Based on Public Social Media Engagement Intervention”(72274119)