Automated coding of free-text descriptions is a core challenge for National Statistical Institutes (NSIs), which must map colloquial respondent inputs to official taxonomies under strict reliability and transparency requirements. Rule-based systems, the current standard, demand extensive, ongoing human effort to author and maintain the large manual rule sets needed to cover the diversity of real world inputs. This paper evaluates embedding-based semantic retrieval as a lower-maintenance alternative, using the Italian ATECO classification as the empirical setting. We compare several knowledge base construction strategies across seven sentence transformer models (ranging from 118 million to 8 billion parameters), evaluated against 33,544 real-world queries labelled by CIRCE, Istat’s operational rule-based coder, at the 5-digit and 2-digit levels. Moderate-size models (300–700 million parameters) are competitive with much larger ones, achieving Hit@5 above 0.82 and 0.94 at the respective levels. Query length is a significant performance driver, and manual mismatch analysis reveals that roughly one in five disagreements with CIRCE reflect label noise rather than retrieval errors. Embedding-based retrieval thus offers competitive performance with no task-specific training data and a fraction of the human labour required by rule-based approaches.
The paper proposes a strategy for implementing an automated and interactive feature of persuasion for late and nonrespondents in establishment web surveys. The response rate for establishment web surveys is under constant pressure, and strategies to motivate small establishments are particularly costly. Consequently, the tested survey design feature, which emulates the approach of telephone follow-ups in interviewer-administered surveys, offers a potentially new, practical, and cost-effective strategy to counteract these shortcomings. A randomly selected sample of late and nonresponding establishments ( n = 3,000) of a well-established Swiss web survey of establishments was asked to indicate their primary nonresponse reason and was then randomly assigned to one of three appeal types: (1) a targeted appeal addressing their previously indicated nonresponse reason, (2) a general appeal addressing the same reasons as the study’s standard reminder letter, or (3) a main appeal addressing the main reason for nonresponse, “lack of time.” The findings suggest that the type of appeal has no significant impact on response or data quality, whereas the reason given for nonresponse and the timing within the survey period do have an impact. The implications of these findings for survey design research are discussed.
This work provides a guide to small area estimation of the quantile ratio index (QRI) to measure wealth inequality across socio-economic or geographical groups. Wealth exhibits extreme positive skewness, rendering traditional inequality measures unreliable, as they are highly sensitive to extreme values. The QRI’s robustness to this phenomenon addresses this problem, making it particularly interesting for small area estimation problems. Relying on well-established methods, we employ a logit-transformed area-level model with bias-corrected back-transformation, validate it through model-based simulations, and apply it to Italian Survey on Household Income and Wealth data. The results confirm the expected strong heterogeneity in wealth inequality across municipalities and household groups, now based on an indicator that is much more robust than the traditional ones. This work thereby highlights the practical value of combining robust inequality measurement with small area estimation techniques for the production of reliable wealth inequality estimates in policy-relevant domains.
National statistical offices increasingly face requests to provide large language model (LLM) capabilities in secure research environments. The governance problem is to determine which service configurations, if any, can be supported safely on confidential microdata. These services challenge traditional statistical disclosure control because interaction is adaptive, systems may retrieve documents or call analytical tools, and outputs may appear plausible without being grounded in approved data. This paper proposes a governance framework that organizes each request as a sequence of decisions: whether an LLM is needed, which service mode is justified, which threat model applies, which controls are required, and what evidence must be reviewed before release. The framework spans deterministic pipelines, retrieval-augmented generation (RAG), tool-using agents, code-execution agents, linkage configurations, and fine-tuning.
There are increasing numbers of sources available for the production of official statistics. When one is interested in making new output or redesigning the input that is used for existing output, it might not always be clear if and how the intended output can be produced from the variety of input data sets available. The current paper proposes a metadata-based framework with a search algorithm that finds out if a processing path is possible from input data sets to intended output. The framework uses the conceptual metadata of the data set as input, that is, the unit types, the identifier variables, the measurement variables and the reference periods. Our contribution to existing literature is threefold. We provide a set of basic operators, each representing a small, clearly defined step in the process of combining data sets; we can accommodate tailor-made operators (models) to cover a wide range of data configurations, and we provide an implementation in Python code to automatically answer if and how the intended output can be produced from a set of available input data sets. The framework is evaluated on a case study. We provide publicly available code and different examples on Git to facilitate the use and further development of our framework.
