We present a domain-grounded framework and benchmark for tool-aware plan generation in contact centers, where answering a query for business insights, our target use case, requires decomposing it into executable steps over structured tools (Text2SQL (T2S)/Snowflake) and unstructured tools (RAG/transcripts) with explicit depends_on for parallelism. Our contributions are threefold: (i) a reference-based plan evaluation framework operating in two modes - a metric-wise evaluator spanning seven dimensions (e.g., tool-prompt alignment, query adherence) and a one-shot evaluator; (ii) a data curation methodology that iteratively refines plans via an evaluator->optimizer loop to produce high-quality plan lineages (ordered plan revisions) while reducing manual effort; and (iii) a large-scale study of 14 LLMs across sizes and families for their ability to decompose queries into step-by-step, executable, and tool-assigned plans, evaluated under prompts with and without lineage. Empirically, LLMs struggle on compound queries and on plans exceeding 4 steps (typically 5-15); the best total metric score reaches 84.8

Machine learning has been extensively used in various applications related to eye-tracking research. Understanding eye movement is one of the most significant subsets of eye-tracking research that reveals the scanning pattern of an individual. Researchers have thoroughly analyzed eye movement data to understand various eye-tracking applications, such as attention mechanisms, navigational behavior, task understanding, etc. The outcome of traditional machine learning algorithms used for decoding tasks based on eye movement data has received a mixed reaction to Yarbus' claim that it is possible to decode the observer's task from their eye movements. In this paper, to support the hypothesis by Yarbus, we are decoding tasks categories while generating synthetic data samples using well-known Synthetic Data Generators CTGAN and its variations such as CopulaGAN and Gretel AI Synthetic Data generators on available data from an in-person user study. Our results show that augmenting more eye movement data combined with additional synthetically generated improves classification accuracy even with traditional machine learning algorithms. We see a significant improvement in task decoding accuracy from 28.1

Synthetic transcript generation is critical in contact center domains, where privacy and data scarcity limit model training and evaluation. Unlike prior synthetic dialogue generation work on open-domain or medical dialogues, contact center conversations are goal-oriented, role-asymmetric, and behaviorally complex, featuring disfluencies, ASR noise, and compliance-driven agent actions. In deployments where transcripts are unavailable, standard pipelines still yield derived call attributes such as Intent Summaries, Topic Flow, and QA Evaluation Forms. We leverage these as supervision signals to guide generation. To assess the quality of such outputs, we introduce a diagnostic framework of 18 linguistically and behaviorally grounded metrics for comparing real and synthetic transcripts. We benchmark four language-agnostic generation strategies, from simple prompting to characteristic-aware multi-stage approaches, alongside reference-free baselines. Results reveal persistent challenges: no method excels across all traits, with notable deficits in disfluency, sentiment, and behavioral realism. Our diagnostic tool exposes these gaps, enabling fine-grained evaluation and stress testing of synthetic dialogue across languages.

Optimizing discrete prompts for small and medium-sized language models (1-8 billion parameters) poses unique challenges due to their relatively weaker instruction-following capabilities compared to larger counterparts. We present PRompt OPtimization with Expert priors for LLMs (PROPEL), a novel knowledge-augmented iterative optimization framework leveraging the collaborative capabilities of three distinct roles assigned to large language models (LLMs): Responder, Judge, and Optimizer. Unlike prior works that focus on larger LLMs for prompt optimization with natural language feedback, our approach emphasizes improving prompt effectiveness for resource-efficient smaller and medium-sized models through structured iteration and the integration of prompt design principles as expert priors. Our experiments across diverse tasks such as long-text summarization and entity extraction demonstrate significant improvements. PROPEL improves response quality by 10-24% for Query-Based Summarization and 5-16% for Query-Based Entity Extraction over initial prompts, outperforming baseline approaches by at least 12% and 9%, respectively. Ablation studies further show that incorporating prompt design principles as priors enhances response quality by 21% for Llama-3.2 1B and 9% for Llama-3.2 3B.

The Forex market, characterized by high volatility and complexity, presents a significant challenge for accurate prediction of currency price movements. Traditional approaches often rely on either technical indicators or sentiment analysis, limiting their ability to capture the interplay between diverse data modalities. This research work introduces a novel multimodal deep learning framework that integrates technical analysis and sentiment analysis through a cross-modal attention mechanism, enabling a comprehensive understanding of market dynamics. The proposed model leverages innovative alignment techniques to synchronize sentiment from news articles with historical price trends, facilitating robust multiclass prediction of Forex price directions. To evaluate its effectiveness, the model was tested on three major currency pairs—EUR/USD, GBP/USD, and USD/JPY—using k-fold cross-validation. Multiple attention configurations, including no attention, self-attention, bi-cross attention, and a hybrid approach, were implemented to assess the impact of attention mechanisms on prediction performance. Experimental results highlight the superiority of the hybrid attention mechanism, which consistently outperformed single-modality models and other configurations across key metrics, such as Matthew's correlation coefficient, accuracy, directional accuracy, and F1-score. These findings underscore the importance of integrating sentiment and technical data for enhanced Forex prediction. This study contributes to the growing field of multimodal financial forecasting, offering a foundation for future research incorporating advanced risk metrics, real-time trading systems, and broader market applications.

Fine-tuning large language models (LLMs) with domain-specific instruction dataset has emerged as an effective method to enhance their domain-specific understanding. Yet, there is limited work that examines the core characteristics acquired during this process. In this study, we benchmark the fundamental characteristics learned by contact-center (CC) domain specific instruction fine-tuned LLMs with outof-the-box (OOB) LLMs via probing tasks encompassing conversational, channel, and automatic speech recognition (ASR) properties. We explore different LLM architectures (FlanT5 and Llama) and sizes (3B, 7B, 11B, 13B). Our findings reveal remarkable effectiveness of CC-LLMs on the in-domain downstream tasks, with improvement in response acceptability by over 48% compared to OOB-LLMs. However, we observe that the performance of probing classifiers are relatively similar and does not reflect the performance of in-domain downstream tasks. A similar observation is also noted on SentEval dataset that assess capabilities of models in terms of surface, syntactic, and semantic information through probing tasks. Our study challenges the premise that probing classifiers can reveal the fundamental characteristics learned by large language models and is reflective of the downstream task performance, via a case-study of LLMs tuned for contact center domain.

In the dynamic realm of call center communications, the potential of abstractive summarization to transform information condensation is evident. However, evaluating the performance of abstractive summarization systems within contact center domain poses a significant challenge. Traditional evaluation metrics prove inadequate in capturing the multifaceted nature of call center conversations, characterized by diverse topics, emotional nuances, and dynamic contexts. This paper uses domain-specific perturbed summaries to scrutinize the robustness of summarization metrics in the call center domain. Through extensive experiments on call center data, we illustrate how perturbed summaries uncover limitations in existing metrics. We additionally utilize perturbation as data augmentation strategy to train domain-specific metrics. Our findings underscore the potential of perturbed summaries to complement current evaluation techniques, advancing reliable and adaptable summarization solutions in the call center domain.

This research investigates the impact of preference annotation acquisition methods on the performance of LLM alignment algorithms, including Direct Preference Optimization (DPO), Identity Preference Optimization (IPO), and Conservative DPO (cDPO), compared to Supervised Fine-Tuning (SFT) in NLP tasks. We analyze the influence of LLM and human-based preferences on algorithm performance, considering data volume and quality. Additionally, we assess DPO's vulnerability to overfitting and IPO's resilience against it, addressing four main research questions. Using the GAIR dataset and Zephyr-7b as the SFT model, we reveal unexpected negative outcomes. Specifically, DPO trained on LLM preferences outperforms human preferences, contrary to expectations. Moreover, there's no correlation between preference data volume or quality and algorithm performance. Contrary to expectations, DPO shows no overfitting in both human and LLM preference datasets. Surprisingly, cDPO doesn't fare better than DPO under flip noise. Our findings highlight the complexities of preference annotation methods and underscore the importance of scrutinizing negative results in NLP algorithm research.

Purpose:The purpose of this study was to investigate gaze-scanning by pedestrians with homonymous hemianopia (HH) when walking on mid-block sidewalks. Methods:Pedestrians with right homonymous hemianopia (RHH), and left homonymous hemianopia (LHH) without and with left spatial neglect (LHSN) walked on city streets wearing a gaze-tracking system. Gaze points were obtained by combining head movement and eye-in-head movement. Mixed-effects regression models were used to compare horizontal gaze scan magnitudes and rates between the side of the hemi-field loss (BlindSide) and the seeing side (SeeingSide), among the three subject groups, and between mid-block walking and street crossing segments. Results:A total of 7021 gaze scans were obtained from 341 minutes of mid-block walking videos by 19 participants (6 with LHH, 7 with RHH, and 6 with LHSN). The average gaze magnitude and scanning rate in mid-block segments were significantly higher towards the BlindSide than the SeeingSide in LHH (magnitude larger by 1.9° (degrees), P = 0.006; scan rate higher by 4.2 scans/minute, P < 0.001) and RHH subjects (magnitude larger by 3.3°, P < 0.001; scan rate higher by 3.2 scans/minute, P = 0.002), but they were not significantly different in LHSN subjects. The scanning rate, in terms of scans/minute (mean, 95% confidence interval [CI]) was significantly lower in LHSN subjects (mean = 6.9, 95% CI = 5.6-8.7) than LHH (mean = 10.2, 95% CI = 8.0-13.1; P = 0.03) and RHH (mean = 11.1, 95% CI = 9.0-13.7; P = 0.007) subjects. Compared to street-crossings, the scan rate during the mid-block segments was lower by 3.5 scans/minute (P < 0.001) and the gaze magnitude was smaller by 3.8° (P < 0.001) over the 3 groups. Conclusions:Evidence of compensatory scanning suggests a proactive, top-down mechanism driving gaze in HH. The presence of spatial neglect (SN) appeared to negatively impact the top-down process.

In volume visualization transfer functions are widely used for mapping voxel properties to color and opacity. Typically, volume density data are scalars which require simple 1D transfer functions to achieve this mapping. If the volume densities are vectors of three channels, one can straightforwardly map each channel to either red, green or blue, which requires a trivial extension of the 1D transfer function editor. We devise a new method that applies to volume data with more than three channels. These types of data often arise in scientific scanning applications, where the data are separated into spectral bands or chemical elements. Our method expands on prior work in which a multivariate information display, RadViz, was fused with a radial color map, in order to visualize multi-band 2D images. In this work, we extend this joint interface to blended volume rendering. The information display allows users to recognize the presence and value distribution of the multivariate voxels and the joint volume rendering display visualizes their spatial distribution. We design a set of operators and lenses that allow users to interactively control the mapping of the multivariate voxels to opacity and color. This enables users to isolate or emphasize volumetric structures with desired multivariate properties. Furthermore, it turns out that our method also enables more insightful displays even for RGB data. We demonstrate our method with three datasets obtained from spectral electron microscopy, high energy X-ray scanning, and atmospheric science.

Contact centers are critical for delivering high-quality customer service to various businesses. Call summarization is a crucial task for contact center agents for compliance, to transfer contextual information to the next agent, or to serve as a reference for future interactions. Agents spend a substantial amount of time writing notes on or after a call, which reduces their productivity and adds to the cost per call. While there exist various pre-trained Large Language Models (LLM) for summarization, they often lack coverage of domain-specific information relevant to businesses. We propose a hybrid streaming notes generation system leveraging the generative capabilities of an LLM fine-tuned for contact center call summarization, but allowing businesses to focus notes generation around events of business interest. Our system reduces after-call work for agents by not only generating notes out-of-the-box but also allowing agents to edit them in real time due to its streaming nature.

Call centers serve as a critical point of contact between businesses and customers. The communication between customers and agents in such calls typically involves asking questions and responding to them. On average, a 10-minute call includes 2-3 customer questions. Such questions provide insights into customer's asks as well as identify areas of improvement for the questions where agents are taking longer time to respond. This motivates a need to peek into each call flowing through the contact center and derive business insights over such questions. To facilitate such deeper analysis and business intelligence at scale, there is a need to efficiently identify and rank the group of questions being asked over millions of calls flowing through a contact center. In this paper, we present a system for question monitoring via question extraction, rewriting and grouping, which enables contact centers to discover questions from calls at scale. Our in-house system leverages natural language processing techniques to transform customer questions into a format that is easily understandable, facilitating streamlined analysis of the data.

Contact centers handle both chat and voice calls for the same domain. As part of their workflow, it is a standard practice to summarize the conversations once they conclude. A significant distinction between chat and voice communication lies in the presence of disfluencies in voice calls, such as repetitions, restarts, and replacements. These disfluencies are generally considered noise for downstream natural language understanding (NLU) tasks. While a separate summarization model for voice calls can be trained in addition to chat specific model for the same domain, it requires manual annotations for both the channels and adds complexity arising due to maintaining two models. Therefore, it’s crucial to investigate if a model trained on fluent data can handle disfluent data effectively. While previous research explored impact of disfluency on question-answering and intent detection, its influence on summarization is inadequately studied. Our experiments reveal up to 6.99-point degradation in Rouge-L score, along with reduced fluency, consistency, and relevance when a fluent-trained model handles disfluent data. Replacement disfluencies have the highest negative impact. To mitigate this, we examine Fused-Fine Tuning by training the model with a combination of fluent and disfluent data, resulting in improved performance on both public and real-life datasets. Our work highlights the significance of incorporating disfluency in training summarization models and its advantages in an industrial setting.