[Objective]This paper aims to analyse the differences in combating hallucinations in large language models between unstructured knowledge,exemplified by knowledge base resources,and structured knowledge,exemplified by knowledge graph resources,using the Traditional Chinese Medicine(TCM)Q&A domain as a case study.Based on these findings,strategies for improving the ability of large language models to combat hallucinations in vertical domains are discussed.[Methods]The study designs experiments using external knowledge combined with prompt engineering techniques to analyse the differences in prompt effects between knowledge base resources and knowledge graph resources in the TCM Q&A domain.It also investigates the superiority of dynamic triplet strategies and integrated fine-tuning strategies in optimising large language models against hallucinations.[Results]Experimental results show that compared to prompts from unstructured knowledge in the knowledge base,prompts from structured knowledge in the knowledge graph perform better in terms of precision,recall and F1 score,improving by 1.9％,2.42％and 2.2％respectively to reach 71.44％,60.76％and 65.31％.Further analysis of the optimisation strategies shows that the combination of the dynamic triplet strategy and fine-tuning had the best effect against hallucinations,achieving precision,recall and F1 scores of 72.47％,65.87％and 68.62％respectively.[Limitations]This study is limited to a single field,as it was only tested in the field of Traditional Chinese Medicine Q&A,and its generalisability needs to be validated in a wider range of scientific fields.[Conclusions]This study has demonstrated that in the field of Traditional Chinese Medicine,structured knowledge from knowledge graphs outperforms traditional unstructured knowledge in reducing hallucinations and improving the accuracy of model responses.It demonstrates the critical role of structured knowledge in enhancing model comprehension skills.The integration of fine-tuning strategies with knowledge resources provides an effective way to improve performance in large language models.This paper provides a theoretical rationale and methodological support for integrating external knowledge into large language models to improve knowledge performance.
[Objective]This paper aims to quantify sentence alignment scores for parallel corpora of low-resource languages,obtain high-quality parallel corpora,and improve machine translation performance.[Methods]We proposed NeuroAlign,a neural network-based unsupervised sentence embedding alignment scoring method.Parallel sentence pairs were embedded into the same vector space,alignment scores for candidate sentence pairs in the parallel corpus were calculated,and low-scoring sentence pairs were filtered out based on score ranking.Finally,we obtained high-quality bilingual parallel corpora for low-resource languages.[Results]In the BUCC2018 parallel text mining task,the F1 score improved by 0.5％～0.8％.In the CCMT2021 low-resource language neural machine translation task,the BLEU score improved by 0.1-10.9.The sentence alignment scores closely approximated human evaluation.[Limitations]Due to the scarcity of low-resource bilingual parallel corpora,our research was limited to Tibetan-Chinese,Uyghur-Chinese,and Mongolian-Chinese language pairs.[Conclusions]This new method can effectively increase sentence alignment scoring for low-resource language machine translation parallel corpora,improving the corpus quality at the data source level and enhancing machine translation performance.
[Objective]This paper reviews the research progress on semantic novelty in China and abroad.It explores relevant techniques and provides references for future studies.[Coverage]We used keywords such as"novelty of the literature","semantic novelty","literature novelty",and search expressions like"semantic novelty and literature evaluation"to retrieve literature from Web of Science,Elsevier,Springer,Google Scholar,as well as Chinese databases like CNKI,Wanfang,and VIP.A total of 70 representative literature were selected for review.[Methods]We summarized research on semantic novelty,focusing on the definition of novelty,evaluation indicators,and different evaluation methods.We also discussed the current development status and future trends of evaluating semantic novelty in scientific literature.[Results]Semantic novelty evaluation has gradually received widespread attention from the academic community.Related studies have evaluated semantic content without establishing a unified measurement index.[Limitations]Existing evaluation of literature novelty mainly focuses on external features.Fewer research papers directly addressed semantic novelty,limiting support for reviews.[Conclusions]The evaluation of semantic novelty in scientific literature fundamentally lies in the novelty of semantic content.Quantitative research has become the mainstream method,but the calculation method of evaluation indicators needs to be clarified.Future studies on novelty evaluation should combine qualitative and quantitative methods for more comprehensive evaluations.
[Objective]By constructing a three-dimensional research framework of"scenario-problem-method"for altmetrics,this article aims to enrich the research design of altmetrics analysis and promote the healthy and sustainable development of altmetrics.[Methods]Drawing on mature frameworks in science of science and informetrics,and combining the characteristics of altmetrics,a research framework is constructed from four dimensions:application scenarios,research questions,key methods,and exploratory methods.[Results]The application scenarios of altmetrics are summarized in scientific evaluation,scientific communication and knowledge diffusion.Specifically,indicator application,influencing factors and indicator construction are proposed for scientific evaluation scenarios.Communication strategies,communication structures,communication trends,and science-social interaction are proposed for scientific communication scenarios.And research questions on knowledge diffusion include diffusion strategy,diffusion structure and diffusion effect.And we combine three key analytical methods of causal inference,network analysis,and machine learning to explain each research design according to the problem.[Limitations]The frameworks proposed in this study faces challenges in terms of operability and implementation.Further empirical testing is needed in the future.[Conclusions]The framework proposed in this article is conducive to promoting altmetrics to enter the connotative development phase.
[Objective]This paper aims to improve the accuracy of automatic extraction of technical words and function effects of patents.[Methods]First,ChatGPT is used as the Teacher-model,and ChatGLM3 is used as the Student-model.Through knowledge distillation,the training data extracted by ChatGPT are used to fine-tune ChatGLM3,resulting in multiple technical word extraction models and a function word extraction model.These models are performed to extract technical words and function words from the abstract,the first claim,and the technical effect segments of patents,respectively.[Results]Compared to ChatGPT,the fine-tuned technical word extraction models and the function word extraction model show higher accuracy and lower recall rates.The ChatGLM3 fine-tuning model of the first claim has the highest accuracy of 0.734 and Fl values of 0.724,respectively.The accuracy of the function word extraction model reached 0.649,which was higher than the accuracy of the commercial tool's 0.530.[Limitations]This study needs to be further optimized in the following aspects.The technical field and patent language are single,the amount of verification data is small,and the data cleaning rules are not comprehensive enough.[Conclusions]This research scheme improves the accuracy of large language models in automatically extracting technical effects through knowledge distillation operation.Additionally,this study supports mining cutting-edge innovative and hotspot technologies from patents,facilitating higher quality intelligent patent analysis.
