We extend the popular transformer architecture to a multi modal model, processing both visual and textual inputs. We propose a new attention mechanism on Transformer-based architecture for the joint vision and language understanding tasks. Our model fuses multi-level comprehension between images and texts in a weighted manner, which could better curve the internal relationships. Experiments on benchmark VQA dataset CLEVR demonstrate the effectiveness of the proposed attention mechanism. We also observe the improvements in sample efficiency of reinforcement learning through the experiments on grounded language understanding tasks of BabyAI platform.
Farmer suicides have become an urgent social problem which governments around the world are trying hard to solve. Most farmers are driven to suicide due to an inability to sell their produce at desired profit levels, which is caused by the widespread uncertainty/fluctuation in produce prices resulting from varying market conditions. To prevent farmer suicides, this paper takes the first step towards resolving the issue of produce price uncertainty by presenting PECAD, a deep learning algorithm for accurate prediction of future produce prices based on past pricing and volume patterns. While previous work presents machine learning algorithms for prediction of produce prices, they suffer from two limitations: (i) they do not explicitly consider the spatio-temporal dependence of future prices on past data; and as a result, (ii) they rely on classical ML prediction models which often perform poorly when applied to spatio-temporal datasets. PECAD addresses these limitations via three major contributions: (i) we gather real-world daily price and (produced) volume data of different crops over a period of 11 years from an official Indian government administered website; (ii) we pre-process this raw dataset via state-of-the-art imputation techniques to account for missing data entries; and (iii) PECAD proposes a novel wide and deep neural network architecture which consists of two separate convolutional neural network models (trained for pricing and volume data respectively). Our simulation results show that PECAD outperforms existing state-of-the-art baseline methods by achieving significantly lesser root mean squared error (RMSE) - PECAD achieves ∼25% lesser coefficient of variance than state-of-the-art baselines. Our work is done in collaboration with a non-profit agency that works on preventing farmer suicides in the Indian state of Jharkhand, and PECAD is currently being reviewed by them for potential deployment.
We study the predict+optimise problem, where machine learning and combinatorial optimisation must interact to achieve a common goal. These problems are important when optimisation needs to be performed on input parameters that are not fully observed but must instead be estimated using machine learning. We provide a novel learning technique for predict+optimise to directly reason about the underlying combinatorial optimisation problem, offering a meaningful integration of machine learning and optimisation. This is done by representing the combinatorial problem as a piecewise linear function parameterised by the coefficients of the learning model and then iteratively performing coordinate descent on the learning coefficients. Our approach is applicable to linear learning functions and any optimisation problem solvable by dynamic programming. We illustrate the effectiveness of our approach on benchmarks from the literature.
In this paper, we present UbuntuWorld 1.0 LTS - a platform for developing automated technical support agents in the Ubuntu operating system. Specifically, we propose to use the Bash terminal as a simulator of the Ubuntu environment for a learning-based agent and demonstrate the usefulness of adopting reinforcement learning (RL) techniques for basic problem solving and troubleshooting in this environment. We provide a plug-and-play interface to the simulator as a python package where different types of agents can be plugged in and evaluated, and provide pathways for integrating data from online support forums like AskUbuntu into an automated agent's learning process. Finally, we show that the use of this data significantly improves the agent's learning efficiency. We believe that this platform can be adopted as a real-world test bed for research on automated technical support.
Raven and Song Scope are two, state-of-the-art automated sound analysis tools, based on machine learning techniques for detection of species vocalisations. Individually, these systems have been the subject of a number of reviews; however, to date there have been no comparisons made of their relative performance. This paper compares the tools based on six aspects: theory, software interface, ease of use, detection targets, detection accuracy, and potential applications. Examining these tools, we identified that they fail to detect both syllables and call structures, since Raven only aims to detect syllables while Song Scope targets call structures. Therefore, a Timed Probabilistic Automata (TPA) system is proposed which separates syllables and clusters them into complex structures.
This paper introduces the challenge of digital preservation in the area of engineering design and manufacturing and presents a methodology to apply knowledge representation and semantic techniques to develop Digital Engineering Archives. This work is part of an ongoing, multiuniversity, effort to create Cyber-Infrastructure-Based Engineering Repositories for Undergraduates (CIBER-U) to support engineering design education. The technical approach is to use knowledge representation techniques to create formal models of engineering data elements, workflows and processes. With these formal engineering knowledge and processes can be captured and preserved with some guarantee of long-term interpretability. The paper presents examples of how the techniques can be used to encode specific engineering information packages and workflows. These techniques are being integrated into a semantic Wiki that supports the CIBER-U engineering education activities across nine universities and involving over 3,500 students since 2006.
Considerable progress has been made over the last 15 years on building adaptive control systems to assist pilots in flying damaged aircraft. Once a pilot has regained control of a damaged aircraft, the next problem is to determine the best site for an emergency landing. In general, the decision depends on many factors including the actual control envelope of the aircraft, distance to the site, weather en route, characteristics of the approach path, characteristics of the runway or landing site, and emergency facilities at the site. All of these influence the risk to the aircraft, to the passengers and crew, and to people and property on the ground. We describe an emergency landing planner that takes these various factors into consideration, and proposes possible routes and landing sites to the pilot, ordering them according to estimated risk. We give an overview of the system architecture and input data, describe our modeling of risk, describe how we search the space of landing sites and routes, and give a preliminary performance assessment for characteristic emergency scenarios using the current research prototype.
We present an implementation of a plan adaptation system, BioPlanner, built for biological pathway prediction across species. BioPlanner formulates a pathway discovery problem as a Hierarchical Task Network (HTN) planning problem and solves it by adapting a plan solution of another well-studied pathway. BioPlanner provides the following functionalities: • It automatically builds HTN planning models for a biological pathway domain from the semantic web biological knowledge bases (KBs). • It retrieves plan cases from the biological KBs. • It generates hypothetical pathways using plan adaptation strategies with the aid of biological domain knowledge. • It evaluates the hypothetical plan candidates, ranks them, and recommends the most likely hypotheses to users. • It employs an information gathering multi-agent system to capture knowledge from heterogeneous sources to help the hypothetical plan generation process. We utilize BioPlanner to predict Signaling Transduction pathways for Mus musculus, Gallus gallus ,a ndDrosophila melanogaster from Homo sapiens.
We discuss experiences from evaluating the learning performance of a user-adaptive personal assistant agent. We discuss the challenge of designing adequate evaluation and the tension of collecting adequate data without a fully functional, deployed system. Reflections on negative and positive experiences point to the challenges of evaluating user-adaptive AI systems. Lessons learned concern early consideration of evaluation and deployment, characteristics of AI technology and domains that make controlled evaluations appropriate or not, holistic experimental design, implications of “in the wild” evaluation, and the effect of AI-enabled functionality and its impact upon existing tools and work practices.
Spacecraft simulation is an integral part of NASA mission planning, real-time mission support, training, and systems engineering. Existing approaches that power these simulations cannot quickly react to the dynamic and complex behavior of the International Space Station (ISS). To address this problem, this paper introduces a unique and efficient method for continuously learning highly accurate models from realtime streaming sensor data, relying on an online learning approach. This approach revolutionizes NASA simulation techniques for space missions by providing models that quickly adapt to real-world feedback without human intervention. A novel regional sliding-window technique for online learning of simulation models is proposed that regionally maintains the most recent data. We also explore a knowledge fusion approach to reduce predictive error spikes when confronted with making predictions in situations that are quite different from training scenarios. We demonstrate substantial error reductions up to 74% in our experimental evaluation on the ISS Electrical Power System and discuss the early deployment of our software in the ISS Mission Control Center (MCC) for ground-based simulations.
The Tokyo Metropolitan Government is the largest municipality in Japan and conducts building change identification work. Recently, Tokyo terminated its traditional visual identification work that has been in use for 20 years and shifted to a new automated system. This paper is intended to introduce the Fixed Assets Change Judgment (FACJ) system and its core tool, RealScape. RealScape automatically detects the changes in the height and color of buildings based on three-dimensional (3D) analysis of aerial photographs. It employs a unique pixel-by-pixel stereo processing method and enables the foot-level precision for each building. RealScape detects building changes more accurately than visual judgment operations by humans and reduces the labor costs to one third of the traditional approach and the required judgment duration to about two weeks per 100km 2 .
This paper describes a novel approach based on a combination of techniques in AI, parallel computing, and network science to address an important problem in social sciences and public health: planning and responding in the event of epidemics. Spread of infectious disease is an important societal problem -- human behavior, social networks, and the civil infrastructures all play a crucial role in initiating and controlling such epidemic processes. We specifically consider the economic and social effects of realistic interventions proposed and adopted by public health officials and behavioral changes of private citizens in the event of a ``flu-like'' epidemic. Our results provide new insights for developing robust public policies that can prove useful for epidemic planning.
Designers often use a series of sketches to explain how their design goes through different states or modes to achieve its intended function. Learning how to create such explanations turns out to be a difficult problem for engineering students. An automated “crash test dummy” to let students practice explanations would be desirable. This paper describes how to carry out a core piece of the reasoning needed in such system. We show how an opendomain sketch understanding system can be used to enter many aspects of such explanations, and how qualitative mechanics can be used to check the plausibility of the intended state transitions. The system is evaluated using a corpus of sketches based on designs from an engineering school design & communications course.
The use of statistical AI techniques in authorship recognition (or stylometry) has contributed to literary and historical breakthroughs. These successes have led to the use of these techniques in criminal investigations and prosecutions. However, few have studied adversarial attacks and their devastating effect on the robustness of existing classification methods. This paper presents a framework for adversarial attacks including obfuscation attacks, where a subject attempts to hide their identity imitation attacks, where a subject attempts to frame another subject by imitating their writing style. The major contribution of this research is that it demonstrates that both attacks work very well. The obfuscation attack reduces the effectiveness of the techniques to the level of random guessing and the imitation attack succeeds with 68-91% probability depending on the stylometric technique used. These results are made more significant by the fact that the experimental subjects were unfamiliar with stylometric techniques, without specialized knowledge in linguistics, and spent little time on the attacks. This paper also provides another significant contribution to the field in using human subjects to empirically validate the claim of high accuracy for current techniques (without attacks) by reproducing results for three representative stylometric methods.
Agents and market mechanisms are widely elaborated and applied to automate interaction and decision processes among others in robotics, for decentralized control in sensor networks and by algorithmic traders in financial markets. Currently there is a high demand of efficient mechanisms for the provisioning, usage and allocation of distributed services in the Cloud. Such mechanisms and processes are not manually manageable and require decisions made in quasi real-time. Thus agent decisions should automatically adapt to changing conditions and converge to optimal values. This paper presents a bidding strategy, which is capable of automating the bid generation and utility maximization processes of consumers and providers by the interaction with markets as well as to converge to optimal values. The bidding strategy is applied to the consumer side against benchmark bidding strategies and its behavior and convergence are evaluated in two market mechanisms, a centralized and a decentralized one.
In this paper, we introduce TinyTermite. TinyTermite is a novel AI routing algorithm that is reliable and secure against selective forwarding and replay attacks. TinyTermite is fully implemented on TinyOS based Intel Mote 2 platform and the experiments were done to compare its performance with that of the traditional Termite algorithm. Our results show that the TinyTermite reduces the energy consumption by more than 30% and enhances the security of the network. The experimental results show that TinyTermite is significantly more secure against replay and sinkhole attacks by lowering the packet loss from 88.5% to 32.9% with 12.7% normal packet loss. Finally, the experimental results also demonstrates that the TinyTermite provides high throughput and low latency.
This paper describes aspects of a project aiming at creating a general, flexible and reusable software architecture to address planning problems in space missions. It introduces recent work to realize an open software framework for supporting development of planning and scheduling space applications. The framework, which is named TRF (Timeline-based Representation Framework), aims at supporting application development within different space missions for the European Space Agency (ESA). It is currently being tested on three problem examples, all solved on top of the TRF functionalities. This paper describes the TRF three-layered software architecture and shows how it has been used to deploy a complete application, named MrSPOCK, an interactive system for Long Term Planning in the MARS EXPRESS operational mission.
We describe an AI-enhanced task management tool developed for a military environment, which differs from office environments in important ways: differing time scales, a focus on teams collaborating on tasks instead of an individual managing her own set of diverse tasks, and a focus on tasklists and standard operating procedures instead of individual tasks. We discuss the Task Assistant prototype, our process for adapting it from an office environment to a military one, and lessons learned about developing AI technology for a high-pressure operational environment.
Many complex real-world systems can be modeled using a graphical representation such as a constraint network. If structure can be exploited, many challenging computational tasks can have good typical-case runtimes even if they are theoretically intractable in general. This paper reports on some early experiments in a PhD-level research agenda. We report on a novel constraint network generator for random constraint networks that have a scale-free macrostructure. This scale-free generator is based on the well known Barabasi-Albert preferential attachment model. We show that scale-free constraint networks exhibit interesting phase transition behaviours which have not been seen for other problem classes studied so far.
Collaborative filtering (CF) recommender systems are very popular and successful in commercial application fields. One end-user concern is the privacy of the personal data required by such systems in order to make personalized recommendations. Recently, peer-to-peer decentralized architectures have been proposed to address this privacy issue. On the other hand system managers must be concerned about system robustness. In particular, it has been shown that recommender systems are vulnerable to profile injection, although model-based CF algorithms show greater stability against malicious attacks that have been studied in the state-of-the-art. In this paper we generalize the generic model for decentralized recommendation and discuss the trade-off between robustness and privacy. In this context, we argue that exposing knowledge of the model parameters allows new, highly effective, model-based attack strategies to be considered. We conclude that the security concerns of privacy and robustness stand in opposition to each other and are difficult to satisfy simultaneously.
