This paper systematically examines cross-lingual disparities in pre-training source and factuality alignment in Large Language Model (LLM) answers for multilingual healthcare Q A across English, German, Turkish, Chinese (Mandarin), and Italian. To support this analysis, we (i) constructed MultiWikiHealthCare, a multilingual dataset derived from Wikipedia; (ii) used it to examine cross-lingual differences in healthcare-related coverage; (iii) evaluated the alignment between LLM-generated responses and these reference sources; and (iv) conducted a case study on factual alignment through the use of contextual information and Retrieval-Augmented Generation (RAG). Our findings reveal substantial cross-lingual disparities in both Wikipedia coverage and LLM factual alignment. Based on our Wikipedia analysis, the lowest alignment is observed between English and Chinese Wikipedia pages. Across models, responses align more with English Wikipedia, even when the prompts are non-English. We further show that providing contextual excerpts from non-English Wikipedia at inference time effectively shifts factual alignment toward target knowledge.
Converting data from machine-unreadable formats like PDFs into Markdown has the potential to enhance the accessibility of scientific research. Existing end-to-end decoder transformer models can transform screenshots of PDFs into Markdown, offering more flexibility than pipeline-based methods. Yet, decoding text token by token from scratch is inefficient, especially when dense text can be directly copied from the PDF. To address this challenge, this paper modifies Prompt Lookup Decoding (PLD) to extract candidate sequences directly from PDF files, leveraging the high n-gram overlap between PDFs and their Markdown equivalents. A new method, Copy Lookup Decoding (CLD), is introduced here to enhance PLD's candidate generation mechanism. Experiments demonstrate that CLD can accelerate the conversion process by up to 1.70× at original quality. The codebase for this paper is open-source on GitHub (https://github.com/Fireblossom/CopyLookup).
Large language models (LLMs) often encode subtle biases that reflect historical disparities in their training data. While many studies evaluate bias solely based on model outputs, the internal mechanisms that give rise to such biases remain underexplored. In this work, the framework Bias-CAVs is proposed, an innovative framework that extends the concept activation vector methodology to probe and explain internal LLM representations for bias. This approach conducts a layer-wise analysis to quantify bias projections, revealing where bias is introduced, amplified, or mitigated within the network. In a two-stage process, activations are first extracted from key layers of diverse LLM architectures (e.g., LLaMA-3, Mistral, Phi, and Gemma) and then a linear classifier (via logistic regression) is trained on standardized activations to create Bias-CAVs that distinguish between biased and neutral representations. The findings across gender, race, profession, and political domains contribute to both the understanding of LLM internal bias propagation and the development of more explainable debiasing interventions.
Instruction tuning enhances language models' (LMs) performance by refining them with specific guidelines. This paper explores the application of instruction tuning in generating and editing Internet Drafts (I-Ds), the preliminary versions of Request for Comments (RFCs). The process involves training models with detailed instructions based on earlier drafts and feedback from Working Groups (WGs) to improve the text. This approach enables the generation of drafts that adhere to established Internet Engineering Task Force (IETF) standards, significantly reducing the need for extensive manual revisions that can span years. This work marks a promising start for the future development of network protocols. By combining instruction tuning models with human expertise, we are moving towards more efficient and accurate technical documentation.
Retrieval-Augmented Generation (RAG) enhances large language models (LLMs) by integrating external knowledge retrieval. However, existing multi-hop RAG benchmarks are limited by shallow reasoning depth, simple retrieval structures, and non-descriptive answers. We introduce EcoRAG, a novel multi-hop economic QA benchmark built on knowledge graphs (KGs). EcoRAG extends retrieval depth to seven-hop reasoning, incorporates complex subgraph structures, and leverages domain-specific economic knowledge. It enables a more realistic evaluation of retrieval and reasoning by bridging the gap between structured knowledge sources and generative models, while also providing a reusable benchmark to advance multi-hop RAG research.
