In evidence synthesis, data extraction is a crucial procedure, but it is time intensive and prone to human error. The rise of large language models (LLMs) in the field of artificial intelligence (AI) offers a solution to these problems through automation. In this case study, we evaluated the performance of two prominent LLM-based AI tools for use in automated data extraction. Randomized trials from two systematic reviews were used as part of the case study. Prompts related to each data extraction task (e.g., extract event counts of control group) were formulated separately for binary and continuous outcomes. The percentage of correct responses ( Pcorr ) was tested in 39 randomized controlled trials reporting 10 binary outcomes and 49 randomized controlled trials reporting one continuous outcome. The Pcorr and agreement across three runs for data extracted by two AI tools were compared with well-verified metadata. For the extraction of binary events in the treatment group across 10 outcomes, the Pcorr ranged from 40% to 87% and from 46% to 97% for ChatPDF and for Claude, respectively. For continuous outcomes, the Pcorr ranged from 33% to 39% across six tasks (Claude only). The agreement of the response between the three runs of each task was generally good, with Cohen’s kappa statistic ranging from 0.78 to 0.96 and from 0.65 to 0.82 for ChatPDF and Claude, respectively. Our results highlight the potential of ChatPDF and Claude for automated data extraction. Whilst promising, the percentage of correct responses is still unsatisfactory and therefore substantial improvements are needed for current AI tools to be adopted in research practice.What is already known What is new Potential impact for Research Synthesis Methods readers outside the authors’ field ### Competing Interest StatementThe authors have declared no competing interest.### Funding StatementNational Natural Science Foundation of China (72204003) Teachers Research Foundation Project of Nanjing University of Posts and Telecommunications (NYY222042)### Author DeclarationsI confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.YesI confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.YesI understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).YesI have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.YesAll data produced in the present work are contained in the manuscript

The growing number of preprints allows reviewers to identify the authors’ identities prior to the peer review process. Yet, it remains unclear whether the preprint exposure of prestigious authors to reviewers is correlated with review features. Here, we employed the linear regression model to examine this relationship. By collecting open peer review reports of 2,059 papers published in Nature Communications in 2019 within the fields of biological and health sciences, we found no obvious difference in review features when the identities of authors with different academic prestige are potentially exposed to reviewers. Specifically, no significant effect was observed on the number of questions raised and the sentiments of the review reports (positivity and subjectivity) in the first round of the peer review process. Moreover, we found no evidence that review features from anonymous reviewers were more positively or subjectively expressed than those with reviewers’ names publicly available. The results persisted even when assuming all papers were under single-blind peer review, which were validated by using the eLife data. This study indicates that papers with both prestigious and less well-known authors are treated equally during the open peer review process, which contributes to the ongoing discourse on the fairness of peer review within the scientific community.

Peer review is crucial in improving the quality and reliability of scientific research. However, the mechanisms through which peer review practices ensure papers become top-cited papers (TCPs) after publication are not well understood. In this study, by collecting a data set containing 13, 066 papers published between 2016 and 2020 from Nature communications with open peer review reports, we aim to examine how textual features embedded within the peer review reports of papers that reflect the reviewers' emotions may predict the papers to be TCPs. We compiled a list of 15 textual features and classified them into three categories: peer review features, linguistic features, and sentiment features. We then chose the XGBoost machine learning model with the best performance in predicting TCPs, and utilized the explainable artificial intelligence techniques SHAP to interpret the role of feature importance on the prediction results. The distribution of feature importance ranking results demonstrates that sentiment features play a crucial role in determining papers' potential to be highly cited. This conclusion still holds, even when the ranking of the feature importance changes in the subgroup analysis of dividing the samples into four disciplines (biological sciences, health sciences, physical sciences, and earth and environmental sciences), as well as two groups based on whether reviewers' identities were revealed. This research emphasizes the textual features retrieved from peer review reports that play role in improving manuscript quality can predict the post-publication research impact.

Introduction Air pollution exposure has influenced a broad range of mental health conditions. It has attracted research from multiple disciplines such as biomedical sciences, epidemiology, neurological science, and social science due to its importance for public health, with implications for environmental policies. Establishing and identifying the causal and moderator effects is challenging and is particularly concerning considering the different mental health measurements, study designs and data collection strategies (eg, surveys, interviews) in different disciplines. This has created a fragmented research landscape which hinders efforts to integrate key insights from different niches, and makes it difficult to identify current research trends and gaps.Method and analysis This systematic map will follow the Collaboration for Environmental Evidence’s guidelines and standards and Reporting Standards for Systematic Evidence Syntheses guidelines. Different databases and relevant web-based search engines will be used to collect the relevant literature. The time period of search strategies is conducted from the inception of the database until November 2022. Citation tracing and backward references snowballing will be used to identify additional studies. Data will be extracted by combining of literature mining and manual correction. Data coding for each article will be completed by two independent reviewers and conflicts will be reconciled between them. Machine learning technology will be applied throughout the systematic mapping process. Literature mining will rapidly screen and code the numerous available articles, enabling the breadth and diversity of the expanding literature base to be considered. The systematic map output will be provided as a publicly available database.Ethics and dissemination Primary data will not be collected and ethical approval is not required in this study. The findings of this study will be disseminated through a peer-reviewed scientific journal and academic conference presentations.

Despite considerable progress in understanding the journal evaluation system in China, empirical evidence remains limited regarding the impact of changes in journal rank (CJR) on scientific output. By employing the difference-in-differences (DID) framework, we exploit panel data from 2015 to 2019 to examine the effect of changes in journal ranks on the number of publications by Chinese researchers. Our analysis involves comparing two groups—journals that experienced a change in ranking and journals that did not—before and after the change in ranking. Our analysis reveals a statistically significant negative effect. The results suggest that CJR has led to a 14.81

Pronoun usage’s psychological underpinning and behavioral consequence have fascinated researchers, with much research attention paid to second-person pronouns like “you,” “your,” and “yours.” While these pronouns’ effects are understood in many contexts, their role in bilateral, dynamic conversations (especially those outside of close relationships) remains less explored. This research attempts to bridge this gap by examining 25,679 instances of peer review correspondence with Nature Communications using the difference-in-differences method. Here we show that authors addressing reviewers using second-person pronouns receive fewer questions, shorter responses, and more positive feedback. Further analyses suggest that this shift in the review process occurs because “you” (vs. non-“you”) usage creates a more personal and engaging conversation. Employing the peer review process of scientific papers as a backdrop, this research reveals the behavioral and psychological effects that second-person pronouns have in interactive written communications.

The practice of uploading preprints of scientific manuscripts prior to journal submission has become increasingly popular. As such, it is essential to understand the impact of the preprint version of a manuscript on the peer review process to facilitate the development of open peer review practices. In the current research, we analyze a dataset comprising 1,078 biomedical papers published in Nature Communications and eLife in 2019, along with their manuscript information posted on preprint servers and their peer review histories. Our investigation focuses on the relationship between the readability of manuscript before journal submission, as represented by preprints, and the sentimental features expressed by reviewers. Based on empirical analysis utilizing a linear regression model, it has been found that reviewers are inclined to express positive sentiments towards preprints characterized by technical language, as indicated by low value on the readability indices. Additional subgroup analysis suggests that this positive effect is more pronounced in papers with lower social and scientific impact, as indicated by online attention scores and scholarly views after publication, respectively. Overall, results of our analysis reveals that the utilization of technical language characterized by lower readability level in academic papers does not seem to hinder the peer review process in biomedical science, which has significant implications for the open peer review practice.

In the era of print reading, being selected as a cover paper holds a crucial role in attracting greater attention and bolstering academic influence. It is important to assess its effect on scholarly attention and academic influence, particularly in light of the evolving reading habits among researchers. In this study, we empirically estimate the impact of ‘being selected as a cover paper’ on scholarly online attention (proxied by altmetric score) and academic influence (measured by citation counts). This analysis is based on a data set comprising 25,238 papers selected from 10 high-impact materials science journals (with journal impact factors exceeding 10) published between 2016 and 2020. Our findings indicate a positive correlation between ‘being selected as a cover paper’ and scholarly online attention, while its impact on academic influence is insignificant. Our results remain robust even when excluding the top 1% mostly cited papers, employing the negative binomial model and considering various time windows for estimation. Heterogeneity analysis indicates that the impact of ‘being selected as a cover paper’ on scholarly online attention holds across nearly all topics, consistent with the baseline result. In addition, online platforms, such as Twitter and News outlets, exhibit a higher frequency of sharing research featured as cover papers. We offer suggestive evidence that ‘being selected as a cover paper’ is not solely contingent on its quality. These findings contribute to the development of a precise, dynamic and multi-dimensional evaluation framework, crucial for navigating the revolution of science communication.

With the outbreak of COVID-19 pandemic, simulation modelling approaches have become effective tools to simulate the potential effects of different intervention measures and predict the dynamic COVID-19 trends. In this scoping review, Studies published between February 2020 and May 2022 that investigated the spread of COVID-19 using four common simulation modeling methods were systematically reported and summarized. Publication trend, characteristics, software, and code availability of included articles were analyzed. Among the included 340 studies, most articles used agent-based model (ABM; n = 258; 75.9 %), followed by the models of system dynamics (n = 42; 12.4 %), discrete event simulation (n = 25; 7.4 %), and hybrid simulation (n = 15; 4.4 %). Furthermore, our review emphasized the purposes and sample time period of included articles. We classified the purpose of the 340 included studies into five categories, most studies mainly analyzed the spread of COVID-19 under policy interventions. For the sample time period analysis, most included studies analyzed the COVID-19 spread in the second wave. Our findings play a crucial role for policymakers to make evidence-based decisions in preventing the spread of COVID-19 pandemic and help in providing scientific decision-makings resilient to similar events and infectious diseases in the future.

The COVID-19 pandemic and its resultant lockdowns have interrupted the way scientists live and work. This nevertheless caused an unforeseen impact of COVID-19: the pandemic substantially increased editorial speed. Here, we causally identify the impact of the pandemic on the editorial decision time, based on a quasi-experimental regression discontinuity (RD) design that compares (N = 339,199) papers submitted in the lead-up to and aftermath of the COVID-19 pandemic. We find that editors make acceptance decisions significantly quicker after the pandemic, reducing the editorial decision time of revised papers by 8.9 days on average. The pandemic, however, has unequal impacts on editors. The results reveal a larger reduction in editorial decision time for editors of high-tier journals, in the field of social science, or with busy work schedules. Finally, our findings also allude to the potential for the increase of editorial speed, and will stimulate policy changes in scientific enterprises that strive for accelerated publishing.

Peer review plays an essential role in scientific research, but the influence of reviewers' academic status is often overlooked during this process. By accessing peer review reports, in this study we empirically investigate this effect. Specifically, we analyzed 2,580 peer review histories from eLife submissions between 2016 and 2021 to examine the relationship between reviewers' academic status and their language usage in the first round of peer review. We focused on two types of language features: emotional features (e.g., positivity and subjectivity) and linguistic features (e. g., number of long words and complex words). Our findings revealed no significant reviewer bias of academic status, such that the reviewers' comments with emotional features were not signif-icantly associated with reviewers' awareness of being more prestigious than the last corre-sponding author of the manuscripts. More accomplished reviewers, however, were more likely to use longer and more complex words. Additionally, the results of linguistic features remained robust in the group where the last author served as the last corresponding author. Overall, our findings suggest that the quality of peer review remains the primary consideration for reviewers when evaluating submissions. These results have significant implications for open peer review practices and the fair assessment of the peer review process.

The relationship between environmental regulation (ER) and any associated innovative technologies has been studied in the previous decades; however, the estimated results have varied with no obvious consensus. To analyse what drives the different estimates in existing studies, we investigated the regulation–innovation relationship through a meta-analysis of 1276 estimates reported in 49 studies. In our analysis, 41 aspects of study design were controlled, Bayesian model averaging (BMA), and frequentist model averaging (FMA) methods were used to address model uncertainty problems. Our results suggested that controlling resources and ignoring endogeneity problems both played robust and methodical roles in explaining the differences in individual study results. Additionally, our results also indicated that five factors (middle year, publication year, usage of province-level data, linear model function, and the difference-in-differences (DID) model) consistently explained the differences in the reported estimates. We found that the ER had almost zero influence on technical innovation; more than one flexible policy instrument was required to trigger innovative activities among firms and sectors.

Severe outdoor air pollution in China has long constituted a threat to public health and environmental governance efficiency. Surveillance of online search behaviours that reflect real-time public risk perception and behavioural responses on environmental issues is essential in improving environmental governance. However, owing to the absence of an appropriate analytical framework, quantitative evidence of analysing online data is scarce in the extant research. To address this research gap, we analysed Baidu search indexes (BSI) keywords relevant to the public risk perception and behavioural responses on outdoor air pollution by a surveillance framework consisting of two parallel steps (prediction and monitoring). The proposed framework may improve environmental risk governance efficiency and benefit environmental risk governance policy decision-making through considering real-time public risk perceptions and behavioural response information. Using online search behaviour data, this study adds to the methodology for improving the efficacy of environmental social risk governance.

The detection of emerging trends is of great interest to many stakeholders such as government and industry. Previous research focused on the machine learning, network analysis and time series analysis based on the bibliometrics data and made a promising progress. However, these approaches inevitably have time delay problems. For the reason that leader papers of "emerging topics" share the similar characters with the "cover papers", this study present a novel approach to translate the "emerging topics" detection to "cover paper" prediction. By using "AdaBoost model" and topic model, we construct a machine learning framework to imitate the top journal (chief) editor's judgement to select cover paper from material science. The results of our prediction were validated by consulting with field experts. This approach was also suitable for the Nature, Science, and Cell journals.

Piezoelectret (also known as ferroelectret) is a kind of cellular electret material with strong piezoelectric effect. Such a material exhibits flexibility, low density and small acoustic impedance. Therefore, piezoelectret is an ideal material for air-borne flexible sound transducers. Aiming at high-sensitivity and thermal-stability sound transducers, in this work, laminated fluorinated polyethylene propylene (FEP) and polytetrafluoroethylene (PTFE) piezoelectret film with a regular cellular microstructure is prepared by a procedure involving template-based cellular structure formation and polarization. The results show that the characteristic acoustic impedance of such a laminated FEP/PTFE film is 0.02 MRayl. The quasi-static piezoelectric charge coefficient d33 up to 800 pC/N is achieved in a small applied pressure range. The maximum value of sensitivity of the microphones based on laminated FEP/PTFE piezoelectrets film can reach to 6.4 mV/Pa at 1 kHz. Besides, the frequency response curve of the device is flat in the whole audio range. For an ultrasonic transmitter with a diameter of 20 mm, driven by a voltage of 600 V (Vp), the sound pressure level (SPL) generated by it increases from 80 to 90 dB (Ref. 20 µPA) as frequency increases from 40 to 80 kHz. The thermal stability of the sensitivity for the transducers made of such a laminated FEP/PTFE piezoelectret film is much superior to that of polypropylene (PP) piezoelectret based device. The sensitivity of the present device remains 26% of the initial value after being annealed at 125 ℃ for 211 h. The improvement of thermal stability is attributed to the excellent space charge storage stability of FEP and PTFE.

Review question / Objective: The research question is: What role does the private sector play in global health? Our scoping review aims to: 1) provide a systematic overview of existing relevant research on private sector involved in global health activities; 2) identify the various types of roles that for-profit private sector play in global health; and 3) comprehensively summarize those roles and explore related research gaps in this research domain. Background: Private sectors play an important role in global health in most of the world's health systems. Some critical services provided by private sectors in combating the COVID-19 pandemic significantly mitigated the negative consequences. Thus, it is necessary to conduct a scoping review to investigate the role of the private sector in global health comprehensively and systemically.

由于中国居民医疗费用快速增长,仅依靠医疗保险并不足以缓解过重的居民医疗负担.本文发现,作为一项保障中国农村老年居民收入的社会保障制度,新农保可以显著缓解医疗负担.本文采用中国健康与养老追踪调查2015年数据,利用老年人超过60岁可以领取养老金这一自然实验,使用断点回归方法考察了领取养老金对医疗负担的因果效应.结果 显示,获得新农保可以显著降低自负医疗费用在收入中的比例,并显著降低灾难性医疗支出发生的概率.在此基础上,本文通过建立理论模型讨论了为什么在缓解医疗负担上,新农保做到了新农合没做到的事.因为医疗保险待遇的提高会因道德风险而不能有效降低自负医疗费用,而养老保险可以通过收入效应缓解医疗负担.这一解释也得到了样本数据和数值模拟的验证.最后,从更一般的社会总福利视角看,医疗保险虽然在缓解医疗负担上效果不明显,但其强大的共济性可以在很大程度上提升社会总福利.

OBJECTIVES:In China, news media are useful for educating the public about the health threats of air pollution. To explore the potential gaps between scientific findings and the public's understanding of them, the characteristics of news media articles and their corresponding scientific papers were analysed.METHODS:We used 22 articles relating to the health outcomes of exposure to outdoor air pollution published on Baidu News over the past year. An assessment tool developed by Robinson et al was used to evaluate the quality scores of news articles. Pearson correlation coefficients were used to measure the relationship between news media reporting and the characteristics of scientific papers. Misleading reporting, interpretation, or extrapolation in headlines and text bodies of news articles were examined.RESULTS:The quality scores of the news articles ranged from -4 to 8, with an overall median score of 3. Correlation results showed that the scientific papers citation in Twitter (r = .88, P < .001) and Facebook (r = .64, P < .01) were significantly and positively associated with their citations in news stories. Media misunderstanding of scientific findings was common: 15 news headlines were identified with at least one spin (misrepresentation of scientific results), and 12 news articles had seven types of spin in the body texts.CONCLUSION:Little media attention has been paid to scientific findings by Chinese researchers. Therefore, researchers and science journalists in China should make a better effort to engage in accurate and informative public discourse on domestic research.

ObjectivesOutdoor air pollution is a serious environmental problem worldwide. Current systematic reviews (SRs) and meta-analyses (MAs) mostly focused on some specific health outcomes or some specific air pollution.DesignThis evidence gap map (EGM) is to identify existing gaps from SRs and MAs and report them in broad topic areas.Data sourcesPubMed, Cochrane, Scopus and Web of Science were searched from their inception until June 2018. Citations and reference lists were traced.Eligibility criteriaSRs and MAs that investigated the impact of outdoor air pollution on human health outcomes were collected. This study excluded original articles and qualitative review articles.Data extraction and synthesisCharacteristics of the included SRs and MAs were extracted and summarised. Extracted data included authors, publication year, location of the corresponding author(s), publication journal discipline, study design, study duration, sample size, study region, target population, types of air pollution and health outcomes.ResultsAsia and North America published 93% of SRs and MAs included in this EGM. 31% of the SRs and MAs (27/86) included primary studies conducted in 5–10 countries. Their publication trends have increased during the last 10 years. A total of 2864 primary studies was included. The median number of included primary studies was 20 (range, 7–167). Cohort studies, case cross-over studies and time-series studies were the top three most used study designs. The mostly researched population was the group of all ages (46/86, 53%). Cardiovascular diseases, respiratory diseases and health service records were mostly reported. A lack of definite diagnostic criteria, unclear reporting of air pollution exposure and time period of primary studies were the main research gaps.ConclusionsThis EGM provided a visual overview of health outcomes affected by outdoor air pollution exposure. Future research should focus on chronic diseases, cancer and mental disorders.

Despite considerable air pollution prevention and control measures that have been put into practice in recent years, outdoor air pollution remains one of the most important risk factors for health outcomes. To identify the potential research gaps, we conducted a scoping review focused on health outcomes affected by outdoor air pollution across the broad research area. Of the 5759 potentially relevant studies, 799 were included in the final analysis. The included studies showed an increasing publication trend from 1992 to 2008, and most of the studies were conducted in Asia, Europe, and North America. Among the eight categorized health outcomes, asthma (category: respiratory diseases) and mortality (category: health records) were the most common ones. Adverse health outcomes involving respiratory diseases among children accounted for the largest group. Out of the total included studies, 95.2% reported at least one statistically positive result, and only 0.4% showed ambiguous results. Based on our study, we suggest that the time frame of the included studies, their disease definitions, and the measurement of personal exposure to outdoor air pollution should be taken into consideration in any future research. The main limitation of this study is its potential language bias, since only English publications were included. In conclusion, this scoping review provides researchers and policy decision makers with evidence taken from multiple disciplines to show the increasing prevalence of outdoor air pollution and its adverse effects on health outcomes.