Automatic Summary Generation of News for People's Daily Online Corpus

Liang Yuan; Wang Dongbo; Huang Shuiqing

doi:10.13266/j.issn.2095-5472.2022.038

PDF(2352 KB)

Knowledge Management Forum ›› 2022, Vol. 7 ›› Issue (4) : 452-464. DOI: 10.13266/j.issn.2095-5472.2022.038

Automatic Summary Generation of News for People's Daily Online Corpus

Liang Yuan ¹^,² ,
Wang Dongbo ¹^,² ,
Huang Shuiqing ¹^,²

Author information +

History +

Abstract

[Purpose/Significance] This paper conducts a study for the mainstream news media for People's Daily Online corpus, aiming to provide ideas and practical support for the study of automatic text summarization, which can then be applied to news and other related text information processing, and contribute to knowledge aggregation services and information access research. [Method/Process] The experimental corpus of this research was the sub-corpus of the People’s Daily Online in January 2015, June 2015 and January 2016 in the new era People’s Daily (NEPD). Based on TF-IDF, Textrank and other extractive automatic summarization algorithms, based on the generative automatic abstractive summarization model for the pointer-generator network, the research was carried out and analyzed and evaluated the summarization results. [Result/Conclusion] The experiment builds a news extraction automatic abstractive algorithm the Pointer-Generator Networks model for the People’s Daily corpus, and constructs a network model of news generative automatic summary pointer generation for People's Daily Online corpus. Fruitful experimental results are evaluated by Rouge indicator (including 3 indicators: Rouge-1, Rouge-2 and Rouge-L). This article provides corpus support and practical support for the automatic news summarization system.

Key words

People's Daily / extractive automatic summarization / generative automatic summarization / NEPD / pointer-generator networks

Cite this article

EndNote

Ris (Procite)

Bibtex

Download Citations

Liang Yuan , Wang Dongbo , Huang Shuiqing. Automatic Summary Generation of News for People's Daily Online Corpus[J]. Knowledge Management Forum. 2022, 7(4): 452-464 https://doi.org/10.13266/j.issn.2095-5472.2022.038

References

List( Publishing order | Descend order by publishing year | Descend order by cited within ) Chart analysis

[1]	王帅,赵翔,李博,等.TP-AS:一种面向长文本的两阶段自动摘要方法[J].中文信息学报, 2018,32(6):71-79. Cited in this article [1]

[2]	黄水清,王东波.新时代人民日报分词语料库构建、性能及应用(一)——语料库构建及测评[J].图书情报工作,2019,63(22):5-12. Cited in this article [1]

[3]	黄水清,王东波.基于人民日报语料的中央一号文件词频历时分析[J].农业图书情报学报, 2020,32(3):4-9. Cited in this article [1]

[4]	莫燕,王永成.中文文献摘要的自动编制[J].现代图书情报技术,1993(3):10-12. Cited in this article [1]

[5]	王永成.自动编制文献摘要及知识的自动提取[J].现代图书情报技术,1993(3):13-28. Cited in this article [1]

[6]	王永成,许慧敏.OA中文文献自动摘要系统[J].情报学报,1997(2):49-53. Cited in this article [1]

[7]	王知津.基于句子选择的自动文本摘要方法及其评价[J].现代图书情报技术, 1998(1):46-51,58. Cited in this article [1]

[8]	史磊,王永成.英文文献自动摘要系统研究[J].情报学报,1999(6):504-508. Cited in this article [1]

[9]	熊娇,王明文,李茂西,等.基于词项—句子—文档三层图模型的多文档自动摘要[J].中文信息学报,2014,28(6):201-207. Cited in this article [1]

[10]	张筱丹,胡学钢.基于向量空间模型的自动摘要冗余处理研究[J].合肥工业大学学报(自然科学版),2010,33(9):1355-1358. Cited in this article [1]

[11]	刘星含,霍华.基于互信息的文本自动摘要[J].合肥工业大学学报(自然科学版),2014,37(10):1198-1203. Cited in this article [1]

[12]	纪文倩,李舟军,巢文涵,等.一种基于LexRank算法的改进的自动文摘系统[J].计算机科学,2010,37(5):151-154,218. Cited in this article [1]

[13]	曾哲军.基于连续LexRank的多文本自动摘要优化算法研究[J].计算机应用与软件, 2013,30(10):209-212,245. Cited in this article [1]

[14]	刘静,肖璐.基于依存句法分析的多主题文本摘要研究[J].情报杂志,2014,33(6):167-171. Cited in this article [1]

[15]	王帅,赵翔,李博,等.TP-AS:一种面向长文本的两阶段自动摘要方法[J].中文信息学报, 2018,32(6):71-79. Cited in this article [1]

[16]	吴云,杨长春,梅佳俊,等.词句协同自动摘要提取方法[J].计算机工程与设计, 2018,39(9):2776-2779,2810. Cited in this article [1]

[17]	陈晨,张璐,伍之昂.词句协同排序的自动摘要算法[J].江苏大学学报(自然科学版),2016,37(04):443-449. Cited in this article [1]

[18]	丁建立,李洋,王家亮.基于双编码器的短文本自动摘要方法[J].计算机应用, 2019,39(12):3476-3481. Cited in this article [1]

[19]	冯读娟,杨璐,严建峰.基于双编码器结构的文本自动摘要研究[J].计算机工程, 2020,46(6):60-64. Cited in this article [1]

[20]	廖涛,刘宗田,王先传.基于事件的文本表示方法研究[J].计算机科学,2012,39(12):188-191. Cited in this article [1]

[21]	徐馨韬,柴小丽,谢彬,等.基于改进TextRank算法的中文文本摘要提取[J].计算机工程, 2019,45(3):273-277. Cited in this article [1]

[22]	陈海华,黄永,张炯,等.基于引文上下文的学术文本自动摘要技术研究[J].数字图书馆论坛,2016(8):43-49. Cited in this article [1]

[23]	黄水清,李志燕,梁刚.面向计算机类文献的自动摘要系统的研究与实现[J].图书与情报,2006(3):93-97. Cited in this article [1]

[24]	张晗,赵玉虹.基于语义图的医学多文档摘要提取模型构建[J].图书情报工作, 2017,61(8):112-119. Cited in this article [1]

[25]	陈志敏,姜艺,赵耀.基于用户查询扩展的自动摘要技术[J].计算机应用研究, 2011,28(6):2188-2190. Cited in this article [1]

[26]	李芳,何婷婷.面向查询的多模式自动摘要研究[J].中文信息学报,2011,25(2):9-14. Cited in this article [1]

[27]	张哲铭,任淑霞,郭凯杰.结合主题感知与通信代理的文本摘要模型[J].西安电子科技大学学报,2020,47(3):97-104. Cited in this article [1]

[28]	陈燕敏,王晓龙,刘远超,等.一种基于文章主题和内容的自动摘要方法[J].计算机工程与应用,2004(33):11-14. Cited in this article [1]

[29]	罗芳,汪竞航,何道森,等.融合主题特征的文本自动摘要方法研究[J].计算机应用研究, 2021,38(1):129-133. Cited in this article [1]

[30]	杜秀英.基于聚类与语义相似分析的多文本自动摘要方法[J].情报杂志, 2017,36(6):167-172. Cited in this article [1]

[31]	吴世鑫,黄德根,李玖一.基于语义对齐的生成式自动摘要研究[J].北京大学学报(自然科学版), 2021, 57(1):6. Cited in this article [1]

[32]	方旭,过弋,王祺,等.核心词修正的Seq2Seq短文摘要[J].计算机工程与设计,2018,39(12):3610-3615. Cited in this article [1]

[33]	唐晓波,翟夏普.基于混合机器学习模型的多文档自动摘要[J].情报理论与实践,2019,42(2):145-150. Cited in this article [1]

[34]	谭金源,刁宇峰,祁瑞华,等.基于BERT-PGN模型的中文新闻文本自动摘要生成[J].计算机应用,2021,41(1):127-132. Cited in this article [1]

[35]	张克君,李伟男,钱榕,等.基于深度学习的文本自动摘要方案[J].计算机应用,2019,39(2):311-315. Cited in this article [1]

[36]	李维勇,柳斌,张伟,等.一种基于深度学习的中文生成式自动摘要方法[J].广西师范大学学报(自然科学版),2020,38(2):51-63. Cited in this article [1]

[37]	肖元君,吴国文.基于Gensim的摘要自动生成算法研究与实现[J].计算机应用与软件,2019,36(12):131-136. Cited in this article [1]

[38]	官礼和.Internet网络新闻文本自动摘要的研究[J].计算机工程与设计,2007(14):3518-3520,3545. Cited in this article [1]

[39]	韩永峰,许旭阳,李弼程,等.基于事件抽取的网络新闻多文档自动摘要[J].中文信息学报,2012,26(1):58-66. Cited in this article [1]

[40]	沈洲,王永成,许一震,等.一种面向新闻文献的自动摘要系统的研究与实践[J].计算机工程,2000(9):70-72. Cited in this article [1]

[41]	李孟爽,昝红英,贾会贞.基于多特征和Ranking SVM的微博新闻自动摘要研究[J].郑州大学学报(理学版),2017,49(2):44-48. Cited in this article [1]

[42]	王凯祥,任明.基于查询的新闻多文档自动摘要技术研究[J].中文信息学报,2019,33(4):93-100. Cited in this article [1]

[43]	黄小江,万小军,肖建国.基于协同图排序的对比新闻自动摘要[J].北京大学学报(自然科学版),2013,49(1):31-38. Cited in this article [1]

[44]	柯修,王惠临.基于混合方法的多语言多文档自动摘要系统构建及实现[J].图书馆学研究,2013(2):66-72. Cited in this article [1]

[45]	叶雷,余正涛,高盛祥,等.多特征融合的汉越双语新闻摘要方法[J].中文信息学报,2018,32(12):84-91. Cited in this article [1]

[46]	高永兵,王宇,马占飞.基于CR-PageRank算法的个人事件自动摘要研究[J].计算机工程,2016,42(11):64-69. Cited in this article [1]

[47]	陈卓群,王平.面向中文微博摘录式摘要方法研究[J].情报科学,2015,33(3):130-134. Cited in this article [1]

[48]	高永兵,钟振华,王宇,马占飞等.基于混合方法的中文微博自动摘要技术研究[J].计算机工程与科学,2016,38(6):1257-1261.

[49]	贾晓婷,王名扬,曹宇.基于加权主题分布表达的微博文本摘要生成研究[J].东北师大学报(自然科学版),2020,52(1):69-74.

[50]	Text-Summarizer-Pytorh-Chinese[EB/OL].[2021-07-07].https://github.com/LowinLi/Text-Summarizer-Pytorch-Chinese. Cited in this article [2]

[51]	彭敏,高斌龙,黄济民,等.基于高质量信息提取的微博自动摘要[J].计算机工程,2015,41(7):36-42. Cited in this article [1]

[52]	LUHN H P . The automatic creation of literature abstracts[J]. IBM journal of research and development, 1958, 2(2):159-165. Cited in this article [1]

[53]	阮一峰.TF-IDF与余弦相似性的应用（三）：自动摘要[EB/OL]. [2021-07-07].http://www.ruanyifeng.com/blog/2013/03/automatic_summarization.html. Cited in this article [1]

[54]	SEE A, LIU P J, MANNING C D. Get to the point: Summarization with pointer-generator networks[J]. arXiv preprint arXiv:1704.04368, 2017. Cited in this article [1]

[55]	程爽.浅谈《人民日报》改扩版的三个变化（3）[EB/OL].[2021-07-07].https://baike.baidu.com/redirect/7e44WWpuHPxVjlVjuIAMGFXvpzQ0nX6dtcm9N58nsqPgZqu9Xe51VC9kbRkCKxL7T3HLNLWACS5_clRah9xQ4caM3Wxuxf0d6PFTO7bT9zOcRDK1CYukrEXagCY. Cited in this article [1]