The success of RL for LLM post-training stems from an unreasonably uninformative source: a single bit of information per rollout as binary reward or preference label. At the other extreme, distillation offers dense supervision but requires demonstrations, which are costly and difficult to scale. We study text feedback as an intermediate signal: richer than scalar rewards, yet cheaper than complete demonstrations. Textual feedback is a natural mode of human interaction and is already abundant in many real-world settings, where users, annotators, and automated judges routinely critique LLM outputs. Towards leveraging text feedback at scale, we formalize a multi-turn RL setup, RL from Text Feedback (RLTF), where text feedback is available during training but not at inference. Therefore, models must learn to internalize the feedback in order to improve their test-time single-turn performance. To do this, we propose two methods: Self Distillation (RLTF-SD), which trains the single-turn policy to match its own feedback-conditioned second-turn generations; and Feedback Modeling (RLTF-FM), which predicts the feedback as an auxiliary objective. We provide theoretical analysis on both methods, and empirically evaluate on reasoning puzzles, competition math, and creative writing tasks. Our results show that both methods consistently outperform strong baselines across benchmarks, highlighting the potential of RL with an additional source of rich supervision at scale.
We propose a new model for the simulation of both the implied volatility surfaces and the underlying asset price. In the spirit of [Guyon, J. and Lekeufack, J., Volatility is (mostly) path-dependent. Quant. Finance, 2023, 23(9), 1221-1258] who are interested in the dependence of volatility indices (e.g. the VIX) on the paths of the associated equity indices (e.g. the S&P 500), we first study how vanilla options implied volatility can be predicted using the past trajectory of the underlying asset price. Our empirical study reveals that a large part of the movements in the at-the-money-forward implied volatility for times-to-maturity of up to two years can be explained using the past returns and their squares. Moreover, we show that this feedback effect gets weaker when the time-to-maturity increases. Building on this new stylized fact, we fit to historical data a parsimonious version of the SSVI parameterization [Gatheral, J. and Jacquier, A., Arbitrage-free SVI volatility surfaces. Quant. Finance, 2014, 14(1), 59-71] of the implied volatility surface relying on only four parameters and show that the two parameters ruling the at-the-money-forward implied volatility as a function of the time-to-maturity exhibit a path-dependent behavior with respect to the underlying asset price. Finally, we propose a model for the joint dynamics of the implied volatility surface and the underlying asset price. The latter is modeled using a variant of the path-dependent volatility model of Guyon and Lekeufack and the former is obtained by adding a feedback effect of the underlying asset price onto the two parameters ruling the at-the-money-forward implied volatility in the parsimonious SSVI parameterization and by specifying Ornstein-Uhlenbeck processes for the residuals of these two parameters and Jacobi processes for the two other parameters. Thanks to this model, we are able to simulate highly realistic paths of implied volatility surfaces that are free from static arbitrage.
We introduce a discrete polydisperse micrograin BSDF model for the rendering of porous surface materials composed of microscopic elements of different size, shape and reflectance distributed on a bulk medium. Our approach generalizes the anisotropic monodisperse micrograin model We first reformulate it in a non-axis-aligned configuration, allowing for the later combination of different micrograin types elongated in arbitrary directions. We then extend the monodisperse model to the polydisperse case, deriving its three key components: (i) a general filling factor that controls the mix between micrograins and the bulk medium; (ii) an exact normal distribution function for surfaces composed of polydisperse micrograin distributions; and (iii) the corresponding fully-correlated shadowing and masking term. This results in an analytical single-scattering BSDF for discrete polydisperse surface materials, validated over ground truth simulations, for which we also derive a dedicated importance sampling procedure. Our model supports varying heights and anisotropy orientations of different micrograin types as input, giving additional control to simulate phenomena like retro-reflection from mixed materials, color mixture depending on lighting and observation directions, multiple directions of anisotropy, etc.
The proliferation of large and ever-growing resource description framework (RDF) datasets has sparked a need for robust and performant RDF archiving systems. In order to tackle this challenge, several solutions have been proposed throughout the years, including archiving systems based on independent copies, time-based indexes, and change-based approaches. In recent years, modern solutions combine several of the above mentioned paradigms. In particular, aggregated changesets of time-annotated triples have showcased a noteworthy ability to handle and query relatively large RDF archives. However, such approaches still suffer from scalability issues, notably at ingestion time. This makes the use of these solutions prohibitive for large revision histories. Furthermore, applications for such systems remain often constrained by their limited querying abilities, where SPARQL is often left out in favor of single triple-pattern queries. In this article, we propose a hybrid storage approach based on aggregated changesets, snapshots, and multiple delta chains that additionally provides full querying SPARQL on RDF archives. This is done by interfacing our system with a modified SPARQL query engine. We evaluate our system with different snapshot creation strategies on the BEAR benchmark for RDF archives and showcase improvements of up to one order of magnitude in ingestion speed compared to state-of-the-art approaches, while keeping competitive querying performance. Furthermore, we demonstrate our SPARQL query processing capabilities on the BEAR-C variant of BEAR. This is, to the best of our knowledge, the first openly available endeavor that provides full SPARQL querying on RDF archives.
PurposeMicrowave ablation (MWA) modeling and simulation bear great potential for locoregional treatment of liver tumors. However, accurately positioning the antenna according to a planned orientation/location is technically challenging. In cases of misplacement, maintaining the original plan may cause incomplete ablation, while repositioning the antenna may induce tumor seeding. In this work, we propose (i) a digital twin of MWA that simulates ablation outcomes and (ii) an optimizer that suggests corrections to MWA parameters without antenna reinsertion, while ensuring complete tumor ablations.MethodsA finite element scheme was used to solve the coupled microwave propagation and heat transfer equations governing MWA, with personalized dielectric and thermal properties determined from preoperative CT and MRI images. We then proposed an optimization algorithm able to adjust power input, ablation duration, and antenna position to correct for antenna misplacement.ResultsThe simulator and optimizer were evaluated against in vivo swine experimental data. Three ablations were performed in liver regions with varying vascularization. The simulations accurately predicted the ablation zones despite the presence of large vessels near the antenna, achieving Dice scores of 0.82, 0.81, and 0.79. In the case of replanning scenarios, our optimizer predicted new parameter sets that led to Dice scores of 0.83, 0.83, 0.80, a corresponding improvement of 20.3%, 40.7%, and 48.1% in average over the initial ablation result.ConclusionThis paper is the first to address intra-operative replanning of thermal ablation therapy. It demonstrates that optimal ablation results can be achieved without requiring antenna reinsertion by optimizing specific ablation parameters.
