In the realm of business automation, digital assistants/chatbots are emerging as the primary method for making automation software accessible to users in various business sectors. Access to automation primarily occurs through APIs and RPAs. To effectively convert APIs and RPAs into chatbots on a larger scale, it is crucial to establish an automated process for generating data and training models that can recognize user intentions, identify questions for conversational slot filling, and provide recommendations for subsequent actions. In this paper, we present a technique for enhancing and generating natural language conversational artifacts from API specifications using large language models (LLMs). The goal is to utilize LLMs in the "build" phase to assist humans in creating skills for digital assistants. As a result, the system doesn't need to rely on LLMs during conversations with business users, leading to efficient deployment. Experimental results highlight the effectiveness of our proposed approach. Our system is deployed in the IBM Watson Orchestrate product for general availability.

In the realm of business automation, conversational assistants are emerging as the primary method for making automation software accessible to users in various business sectors. Access to automation primarily occurs through application programming interface (APIs) and robotic process automation (RPAs). To effectively convert APIs and RPAs into chatbots on a larger scale, it is crucial to establish an automated process for generating data and training models that can recognize user intentions, identify questions for conversational slot filling, and provide recommendations for subsequent actions. In this paper, we present a technique for enhancing and generating natural language conversational artifacts from API specifications using large language models (LLMs). The goal is to utilize LLMs in the "build" phase to assist humans in creating skills for digital assistants. As a result, the system does not need to rely on LLMs during conversations with business users, leading to efficient deployment. Along with enabling digital assistants, our system employs LLMs as proxies to simulate human interaction and automatically evaluate the digital assistant's performance. Experimental results highlight the effectiveness of our proposed approach. Our system is deployed in the IBM Watson Orchestrate product for general availability.

Process automation has evolved from end-to-end automation of repetitive process branches to hybrid automation where bots perform some activities and humans serve other activities. In the context of knowledge-intensive processes such as IT operations, implementing hybrid automation is a natural choice where robots can perform certain mundane functions, with humans taking over the decision of when and which IT systems need to act. Recently, ChatOps, which refers to conversation-driven collaboration for IT operations, has rapidly accelerated efficiency by providing a cross-organization and cross-domain platform to resolve and manage issues as soon as possible. Hence, providing a natural language interface to bots is a logical progression to enable collaboration between humans and bots. This work presents a no-code approach to provide a conversational interface that enables human workers to collaborate with bots executing automation scripts. The bots identify the intent of users' requests and automatically orchestrate one or more relevant automation tasks to serve the request. We further detail our process of mining the conversations between humans and bots to monitor performance and identify the scope for improvement in service quality.

Commit messages play an important role in several software engineering tasks such as program comprehension and understanding program evolution. However, programmers neglect to write good commit messages. Hence, several Commit Message Generation (CMG) tools have been proposed. We observe that the recent state of the art CMG tools use simple and easy to compute automated evaluation metrics such as BLEU4 or its variants. The advances in the field of Machine Translation (MT) indicate several weaknesses of BLEU4 and its variants. They also propose several other metrics for evaluating Natural Language Generation (NLG) tools. In this work, we discuss the suitability of various MT metrics for the CMG task. Based on the insights from our experiments, we propose a new variant specifically for evaluating the CMG task. We re-evaluate the state of the art CMG tools on our new metric. We believe that our work fixes an important gap that exists in the understanding of evaluation metrics for CMG research. CCS CONCEPTS• Software and its engineering $\rightarrow$Software verification and validation.

Business processes in workflows comprise of an ordered sequence of tasks and decisions to accomplish certain business goals. Each decision point requires the input of a decision-maker to distill complex case information and make an optimal decision given their experience, organizational policy, and external contexts. Overlooking some of the essential factors or lack of knowledge can impact the throughput and business outcomes. Therefore, we propose an end-to-end automated decision support system with explanation for business processes. The system uses the proposed process-aware feature engineering methodology that extracts features from process and business data attributes. The system helps a decision-maker to make quick and quality decisions by predicting the decision and providing an explanation of the factors which led to the prediction. We provide offline and online training methods robust to data drift that can also incorporate user feedback. The system also support predictions with live instance data i.e., allow decision-makers to conduct trials on current data instance by modifying its business data attribute values. We evaluate our system on real-world and synthetic datasets and benchmark the performance, achieving an average of 15% improvement over baselines.

Accounts Payable (AP) is a resource-intensive business process in large enterprises for paying vendors within contractual payment deadlines for goods and services procured from them. There are multiple verifications before payment to the supplier/vendor. After the validations, the invoice flows through several steps such as vendor identification, line-item matching for Purchase order (PO) based invoices, Accounting Code identification for Non- Purchase order (Non-PO) based invoices, tax code identification, etc. Currently, each of these steps is mostly manual and cumbersome making it labor-intensive, error-prone, and requiring constant training of agents. Automatically processing these invoices for payment without any manual intervention is quite difficult. To tackle this challenge, we have developed an automated end-to-end invoice processing system using AI-based modules for multiple steps of the invoice processing pipeline. It can be configured to an individual client’s requirements with minimal effort. Currently, the system is deployed in production for two clients. It has successfully processed around ~80k invoices out of which 76% invoices were processed with low or no manual intervention.

Recommending a sequence of activities for an ongoing case requires that the recommendations conform to the underlying business process and meet the performance goal of either completion time or process outcome. Existing work on next activity prediction can predict the future activity but cannot provide guarantees of the prediction being conformant or meeting the goal. Hence, we propose a goal-oriented next best activity recommendation. Our proposed framework uses a deep learning model to predict the next best activity and an estimated value of a goal given the activity. A reinforcement learning method explores the sequence of activities based on the estimates likely to meet one or more goals. We further address a real-world problem of multiple goals by introducing an additional reward function to balance the outcome of a recommended activity and satisfy the goal. We demonstrate the effectiveness of the proposed method on four real-world datasets with different characteristics. The results show that the recommendations from our proposed approach outperform in goal satisfaction and conformance compared to the existing state-of-the-art next best activity recommendation techniques.

Activities in the ticket resolution process have comments and emails associated with them. Process mining uses structured logs and does not analyze the unstructured data such as comments for process discovery. However, comments can provide additional information for discovering models of process reality and identifying improvement opportunities efficiently. To address the problem, we propose to extract topical phrases (keyphrases) from the unstructured data using an unsupervised graph-based approach. These keyphrases are then integrated into the event log to derive enriched event logs. A process model is discovered using the enriched event logs wherein keyphrases are represented as activities, thereby capturing the flow relationship with other activities and the frequency of occurrence. This provides insights that can not be obtained solely from the structured data. To evaluate the approach, we conduct a case study on the ticket data of a large global IT company. Our approach extracts keyphrases with an average accuracy of around 80%. Henceforth, discovered process model succinctly captures underlying process interactions which allows to understand in detail the process realities and identify opportunities for improvement. In this case, for example, manager identified that having a bot to capture specific information can reduce the delays incurred while waiting for the information.

Predictive Business Process Monitoring tasks such as next activity prediction, next timestamp prediction, etc. are becoming crucial as new technologies are enabling intelligent automation of business processes. Recent works try to address this problem by using deep learning models that encode limited attribute information of past activities for a case independently w.r.t the other cases in execution. However, the predictions for a case can also depend on contextual information such as inter-case dependencies and domain-specific attributes, which is not considered in previous works. We propose a novel method of encoding the contextual state information i.e., encoding the state of on-going cases and multi-attribute domain-specific information along with intra-case information in an unsupervised manner. We train two widely used deep learning models i.e., LSTM and Transformer using the proposed representation, and compare their performance to show the improved results over the state-of-the-art models. We also investigate the influence of past activities and other on-going cases on prediction using self-attention, making the framework to provide interpretable predictions for a decision making business user.

Accounts payable refer to the practice where organizations procure goods and services on credit which need to be reimbursed to the vendors in due time. Once the vendor raises an invoice, it undergoes through a complex process before the final payment. In this process, tax code determination is one of the most challenging steps, which determines the tax to be levied and directly influences the amount payable to a vendor. This step is also very important from a regulatory compliance standpoint. However, it is error-prone, labor (resource) intensive, and needs regular training of the resources as it is done manually. Further, an error in the tax code determination induces penalties on the organization. Automatically arriving at a tax-code for a given product accurately and efficiently is a daunting task. To address this problem, we present an automated end-to-end system for tax code determination which can either be used as a standalone application or can be integrated into an existing invoice processing workflow. The proposed system determines the most relevant tax code for an invoice using attributes such as item description, vendor details, shipping and delivery location. The system has been deployed in production for a multinational consumer goods company for more than 6 months. It has already processed more than 22k items with an accuracy of more than 94% and high confidence prediction accuracy of around 99.54%. Using this system, approximately 73% of all the invoices require no human intervention.

The procure to pay process (P2P) in large enterprises is a back-end business process which deals with the procurement of products and services for enterprise operations. Procurement is done by issuing purchase orders to impaneled vendors and invoices submitted by vendors are paid after they go through a rigorous validation process. Agents orchestrating P2P process often encounter the problem of matching a product or service descriptions in the invoice to those in purchase order and verify if the ordered items are what have been supplied or serviced. For example, the description in the invoice and purchase order could be TRES 739mL CD KER Smooth and TRES 0.739L CD KER Smth which look different at word level but refer to the same item. In a typical P2P process, agents are asked to manually select the products which are similar before invoices are posted for payment. This step in the business process is manual, repetitive, cumbersome, and costly. Since descriptions are not well-formed sentences, we cannot apply existing semantic and syntactic text similarity approaches directly. In this paper, we present two approaches to solve the above problem using various types of available agent's recorded feedback data. If the agent's feedback is in the form of a relative ranking between descriptions, we use similarity ranking algorithm. If the agent's feedback is absolute such as match or no-match, we use classification similarity algorithm. We also present the threats to the validity of our approach and present a possible remedy making use of product taxonomy and catalog. We showcase the comparative effectiveness and efficiency of the proposed approaches over many benchmarks and real-world data sets.

Reimbursement process is part of many organizations as the employees make expenses for various purposes such as travel and nontravel related. The travel could be domestic or international based on the requirements and non-travel expenses include things like procurement of hardware. Typically there is a well-defined process for the expense approval and reimbursement. It is important to analyze the process to identify the improvement and automation opportunities given the prevalence of the process. In this work, we analyse the data for reimbursement process at TU/e, Netherlands which is made publicly available as part of BPI challenge 2020. Using process mining, the logs are analysed from multiple dimensions such as bottlenecks, reworks, variants, and automation amenability. Further comparative analysis is performed for domestic and international travel over the years and against each other. This analysis provides actionable insights using which the process can be improved efficiently.

Business processes are complex and involve the execution of various steps using different resources that can be shared across various tasks. Processes require analysis and process owners need to constantly look for methods to improve process performance indicators. It is non-trivial to quantify the improvement of a proposed change, without implementing or conducting randomized controlled trials. In several cases, the cost and time for implementing and evaluating the benefits of these changes are high. To address this, we propose a principled framework using Structural Causal Models which formally codify existing cause-effect assumptions about the process, control confounding and answer “what if” questions with observational data. We formally define an end to end methodology which takes process execution logs and specified BPMN model as inputs for structural causal model discovery and for performing counterfactual reasoning. We show that exploiting the process specification for causal discovery automatically ensures the inclusion of subject matter expertise, and also provides an effective computational methodology. We illustrate the effectiveness of our approach by answering intervention and counterfactual questions on example process models.

Location awareness of people inside commercial establishments can help with occupancy-based dynamic energy management and indoor navigation. In this paper, we propose MobiCeil, a novel phone-based indoor localization technique. The proposed technique is offline, automated, and uses image captured from phone's camera to identify the unique ceiling structure of any particular location in the office building. The proposed method is based on these assumptions: (a) in office, employees tend to keep their phones lying on the table, and (b) the layout of ceiling landmarks in a portion of the ceiling structure (as captured by the phone's camera on the table) is unique. We validated these assumptions by checking the phone placement of 47 employees randomly at their cubicle or meeting room, and collecting ceiling layout data from 18 meeting rooms and 6 cubicles in an IT office building. To evaluate the performance of MobiCeil, we collected images of the ceiling as seen by the phone (front and back) camera in three different rotations of the phone placed on the table, to capture a total of 960 ceiling images. Our approach achieved an accuracy of 88.2% for identifying locations, with a low computation time of 2.8s per image.

In this paper, we present Agent Assist, a virtual assistant which helps IT support staff to resolve tickets faster. It is essentially a conversation system which provides procedural and often complex answers to queries. This system can ingest knowledge from various sources like application documentation, ticket management systems and knowledge transfer video recordings. It uses an ensemble of techniques like question classification, knowledge graph based disambiguation, information retrieval, etc., to provide quick and relevant solutions to problems from various technical domains and is currently being used in more than 650 projects within IBM.

Business process modeling is an integral task needed for efficient running of business operations. Often process models remain buried in unstructured documents as images or screenshots. Such embedded process model images may become quickly obsolete as the underlying business process evolves. Thus, there is value in digitizing the unstructured images. We propose a novel automated solution to transform a process model image into the standard Business Process Model and Notation (BPMN) format. Our deep-learning based approach performs well in practice achieving good precision and recall.

Account payables are amount owed to vendors for goods and services delivered to a company. Vendors raise invoices which go through several processing steps before they are paid by a company. Companies have contractual obligations with vendors for paying the invoices within a stipulated time. Invoices that exceed this time attract penalty and affect vendor satisfaction to work with the company. It is very critical for large firms dealing with thousands of vendors for their day to day operations to meet the service level agreements with vendors to avoid penalties. Any assistance for practitioners, warning them of potential invoices that can breach the service level agreements, can help them in minimizing the penalties. In this research, we model the problem of identifying delayed invoices as a supervised classification task. There are three characteristics of this problem which are challenging from a classification perspective: (i) the status of an invoice is affected by other invoices that are simultaneously being processed, as there are limited resources to process the huge volume of invoices, (ii) feature engineering to capture the temporal aspect of the invoice and having the optimal representation of the multiple data entries created per invoice, and (iii) the number of paid late invoices are much smaller in percentage compared to paid on time invoices in the training data set, hence the classes are imbalanced. The results obtained by training an ensemble of classifiers show that penalties can be avoided on more than 82% of the invoices which are currently being penalized.

Question answering is one of the primary challenges of natural language understanding. In realizing such a system, providing complex long answers to questions is a challenging task as opposed to factoid answering as the former needs context disambiguation. The different methods explored in the literature can be broadly classified into three categories namely: 1) classification based, 2) knowledge graph based and 3) retrieval based. Individually, none of them address the need of an enterprise wide assistance system for an IT support and maintenance domain. In this domain the variance of answers is large ranging from factoid to structured operating procedures; the knowledge is present across heterogeneous data sources like application specific documentation, ticket management systems and any single technique for a general purpose assistance is unable to scale for such a landscape. To address this, we have built a cognitive platform with capabilities adopted for this domain. Further, we have built a general purpose question answering system leveraging the platform that can be instantiated for multiple products, technologies in the support domain. The system uses a novel hybrid answering model that orchestrates across a deep learning classifier, a knowledge graph based context disambiguation module and a sophisticated bag-of-words search system. This orchestration performs context switching for a provided question and also does a smooth hand-off of the question to a human expert if none of the automated techniques can provide a confident answer. This system has been deployed across 675 internal enterprise IT support and maintenance projects.

The goal of this paper is to establish a framework called SMOME for evaluating the marginal utility of adding smart sensing and metering hardware to residential premises, in terms of efficiency improvement (reduction in energy consumption) and economic benefits (reduction in energy cost). In order to isolate the effect of smart home technologies on energy efficiency, it is assumed that no changes are made to the appliances already installed. SMOME is developed for experimenting with different types of hardware architectures. Analysis algorithms suitable to the installed suite of sensing and metering hardware are described, which take into account realistic practical constraints. Validated with empirical appliance-level data, the proposed methodology allows us to perform a cost-benefit analysis of several potential smart home solutions. This analysis is meant to enable home owners to evaluate a priori the real cost saving potential of these solutions, when applied to their home.