Introduction. Large language models (LLMs) have become increasingly significant in crisis information management due to their advanced natural language processing capabilities. This study aims to develop a comprehensive evaluation benchmark to assess the effectiveness of LLMs in generating crisis information. Method. CIEeval, an evaluation dataset, was constructed through steps such as information extraction and prompt generation. CIEeval covers 26 types of crises across sub-domains including water disasters, environmental pollution, and others, comprising a total of 4.8k data entries. Analysis. Eight LLMs applicable to the Chinese context were selected for evaluation based on multidimensional criteria. A combination of manual and machine scoring methods was utilized. This approach ensured a comprehensive understanding of each model's performance. Results. The manual and machine scores showed significant correlation. Under this scoring method, Claude 3.5 Sonnet performed the best, particularly excelling in complex scenarios like natural and accident disasters. In contrast, while scoring slightly lower overall, Chinese models like ERNIE 4.0 Turbo and iFlytek Spark V4.0, showed strong performance in specific crises. Conclusion. The evaluation benchmark validates the best LLM for crisis information generation (Claude 3.5 Sonnet) and provides valuable insights for LLMs to optimize and apply LLM in crisis information.
[Purpose/significance]Comprehending the development process of scientific knowledge contributes to the scientific research.For knowledge evaluation and service,it is crucial to trace the structure and progress of knowledge in subdivided fields from a micro perspective.[Method/process]This article took the knowledge unit in medical informatics as an example.This paper used the semantic type of triples to define the treatment-related subdivision fields,constructed the knowledge unit citation networks of 125 diseases at different intervals,and identified the knowledge communities with the Leiden algorithm.From the dimensions of knowledge evolution and knowledge competition state,we aim to reveal the evolutionary characteristics of disease individuals.The indexes of Richness,Balance,and Difference are calculated to reveal the diversity characteristics of disease individuals as well as the overall disease population.[Result/conclusion]The research demonstrates that the knowledge communities can reflect the knowledge structure and evolution state of disease individuals.The overall diversity characteristics of diseases include:the common-ality of indicators indicates that the number of all disease knowledge communities is increasing,and the differences in scale and composition between communities are expanding.Different diseases show conventional,early-controversial,and generalized evolu-tionary patterns,with the earlier diseases being less balanced and more different.
[Purpose/significance]Trust is a crucial component in sustaining collaboration and facilitating knowledge transfer among scientists,transforming knowledge from being individually-owned to being shared among many.Studying trust mechanisms in scientific knowledge transfer not only helps maintain relationships among collaborators but also accelerates the process of knowledge transfer,thereby promoting scientific development further.[Method/process]We initially quantified the trust among scientific col-laborators using the American Physical Society dataset(APS)and examined the trust distribution pattern within this group.Subse-quently,we categorized knowledge transfer among these collaborators into two types:implicit and explicit.We then delineated the paths of knowledge transfer within the collaborative network.Finally,we employed regression analysis to verify the impact of the de-gree of trust among collaborators on the volume of knowledge transfer.[Result/conclusion]Trust and knowledge transfer among scientific collaborators exhibit a distinct time-series resonance pattern.The trust within collaborative teams is less than that within teams with explicit knowledge transfer.The correlation between explicit transfer and trust is stronger,although the dependence is weaker.This paper expands the study of trust emotions in scientific collaboration to include trust behaviors.Furthermore,it mitigates the limitations of measuring static trust in a cross-sectional collaboration by quantifying the trust among scientific collaborators.
[目的/意义]梳理国内外开放科学评价的研究成果,以期为其理论研究和实践发展提供参考.[方法/过程]运用VOSviewer软件,从开放数据、开放获取、公众科学、研究方法与模型、开源软件、开放教育资源、众包、开放科学主题8个方面综述国内外开放科学评价的研究进展.[结果/结论]国内外学者对上述8个方面的评价研究进行了广泛探索,且取得了明显进展.[局限]现有研究存在开放科学评价的研究领域广而散、研究方法与模型有待丰富、实践评价调查有待加强等问题.未来应加强开放科学评价的理论研究,丰富研究方法和模型,加强开放科学实践测评,从而更好地促进开放科学实践发展.
[目的/意义]本体广泛应用于知识图谱、信息检索等众多领域,逐渐成为近年来一大研究热点.文章归纳总结了国内外本体构建及应用方面研究成果,以期为相关领域学者提供研究参考.[方法/过程]首先,从本体构建原则、表示语言和构建方法三个维度对本体构建基础研究进行梳理.其次,运用关键节点识别和社团挖掘算法对本体构建主题进行演化分析,并进一步结合中国知网和Web of Science代表性文献梳理近年来本体构建研究热点.最后,结合前述分析总结本体研究主要应用领域,并对本体相关研究进行总结与展望.[结果/结论]结合本体研究现状,半自动化及自动化本体构建、本体融合等技术研究以及本体的应用有望成为未来研究热点.
