Geopolitical forecasting tournaments have stimulated the development of methods for improving probability judgments of real-world events. But these innovations have focused on easier-to-quantify variables, like personnel selection, training, teaming, and crowd aggregation—bypassing messier constructs, like qualitative properties of forecasters' rationales. Here, we adapt methods from natural language processing (NLP) and computational text analysis to identify distinctive reasoning strategies in the rationales of top forecasters, including: (a) cognitive styles, such as dialectical complexity, that gauge tolerance of clashing perspectives and efforts to blend them into coherent conclusions and (b) the use of comparison classes or base rates to inform forecasts. In addition to these core metrics, we explore metrics derived from the Linguistic Inquiry and Word Count (LIWC) program. Applying these tools to multiple tournaments and to forecasters of widely varying skill (from Mechanical Turkers to carefully culled "superforecasters") revealed that: (a) top forecasters show higher dialectical complexity in their rationales and use more comparison classes; (b) experimental interventions, like training and teaming, that boost accuracy also influence NLP profiles of rationales, nudging them in a "superforecaster" direction.

Geopolitical forecasting tournaments have stimulated the development of methods for improving probability judgments of real-world events. But these innovations have focused on easier-to quantify variables, like personnel selection, training, teaming, and crowd aggregation—and bypassed messier constructs, like qualitative properties of forecasters’ rationales. Here we adapt methods from natural language processing (NLP) and computational text analysis to identify distinctive reasoning strategies in the rationales of top forecasters, including: (a) cognitive styles, such as dialectical complexity, that gauge tolerance of clashing perspectives and efforts to blend them into coherent conclusions; (b) the use of comparison classes or base rates to inform forecasts; (c) metrics derived from the Linguistic Inquiry and Word Count (LIWC) program. Applying these tools to multiple forecasting tournaments and to forecasters of widely varying skill (from Mechanical Turkers to carefully culled “superforecasters”) revealed that: (a) top forecasters show higher dialectical complexity in their rationales, use more comparison classes, and offer more past-focused rationales; (b) experimental interventions, like training and teaming, that boost accuracy also influence NLP profiles of rationales, nudging them in a “superforecaster-like” direction.

Knowledge is becoming increasingly recognized as a valuable resource. Given its importance it is surprising that expert systems technology has not become a more common means of utilizing knowledge. In this chapter we review some of the history of expert systems, the shortcomings of first generation expert systems, current approaches and future decisions. In particular we consider a knowledge acquisition and representation technique known as Ripple Down Rules (RDR) that avoids many of the limitations of earlier systems by providing a simple, user-driven knowledge acquisition approach based on the combined use of rules and cases and which support online validation and easy maintenance. RDR has found particular commercial success as a clinical decision support system and we review what features of RDR make it so suited to this domain. INTRODUCTION Knowledge, in particular tacit knowledge, has been recognised as a key factor in gaining a competitive advantage (van Daal, de Haas & Weggeman, 1998). The soft and intangible nature of knowledge has led to increased utilisation of techniques This chapter appears in the book, Decision Making Support Systems: Achievements, Trends and Challenges for the New Decade, edited by Manuel Mora, Guiseppi Forgionne and Jatinder Gupta. Copyright © 2003, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited. 701 E. Chocolate Avenue, Hershey PA 17033-1240, USA Tel: 717/533-8845; Fax 717/533-8661; URL-http://www.idea-group.com ITB8729 IDEA GROUP PUBLISHING

In this research, we present a Bayesian model to aid the investment decision in early stage start-ups and ventures.This model addresses both the venture and the angel investing markets.The model is informed both by previous academic literature on entrepreneurship and by venture capital investment practices.The model is validated through an anonymized experiment where reviewers with previous experience in entrepreneurship or investment or both scored a list of 20 anonymous real companies for which we knew the outcome a priori.The experiment revealed that the model and online scoring platform that we built provide an accuracy of 83% in identifying companies that would later on fail and where the investments would be lost.The model also performs fairly well in identifying companies where the investors would not lose their money but they would either have to wait for a very long time on their returns or they would not receive large return on investment (ROI), and we also show that the model performs modestly in identifying "big exit" companies or companies where the investors would receive high ROI and in a fairly short amount of time.

The authors propose that valuation of information metrics developed near the end of the intelligence cycle are appropriate supplemental metrics for national security intelligence. Existing information and decision theoretic frameworks are often either inapplicable in the context of national security intelligence or they capture affects from inputs aside from just the information or intelligence. Applied information theory looks at the syntactic transmission of information rather than assigning it a quantitative value. Information economics determines the market value of information, which is also inapplicable in a national security intelligence context. Decision analysis can use the value of information to show the expected value of perfect information EVPI and the expected value of imperfect information EVII and although this method can be used with utility theory and not just monetary objectives, it has been shown that decision makers within the intelligence community IC have difficulty agreeing upon how to value objectives within analysis. Additionally, it is difficult to determine how decision makers use intelligence in the decision-making process, which makes existing decision theoretic methods problematic, and might include inputs from variables besides just the intelligence.

We propose to improve the accuracy of prediction market forecasts by using Bayesian networks to constrain probabilities among related questions. Prediction markets are already known to increase forecast accuracy compared to single best estimates. Our own flat prediction market substantially beat a baseline linear opinion pool during the first year. One way to improve performance is by expressing relationships among the questions. Elsewhere we describe work on combinatorial markets. Here we show how to use Bayesian networks within a flat market. The general approach is to decompose a target question (hypothesis) into a set of related variables (causal factors and evidence), when the relationship among the variables is known with some confidence. Then the marginal probabilities for the variables in the Bayes net are updated using the market estimates, with the Bayes net enforcing coherence. This paper describes the overall concept, shows the results for a particular model of the potential Greek exit from the European Union, and describes the team’s future research plan.

This publication contains reprint articles for which IEEE does not hold copyright. Full text is not available on IEEE Xplore for these articles.

Multi-agent dynamic-networks simulations are emerging as a powerful technique for reasoning about complex socio-cultural systems at sufficient fidelity that they can support policy development. Within these models the way in which the agents are modeled and the fidelity of the system are critical. Basic principles guiding the development and use of these models to support policy development are described.

AbstractOver the past three decades, significant improvements in the computer and computational sciences have enabled automated support for increasingly complex decision situations. One example of this progress is the influence diagram, which is simultaneously a graphical and mathematical model of a decision situation. Influence diagrams are a proven asset in the tool kit of decision analysts and they are making the power of decision analytic modelling more accessible to professionals in other disciplines.To develop influence diagrams, decision analysts guide decisionmakers and subject matter experts through a discovery process that necessarily migrates from the unstructured to the structured. These highly complex decision situations often require inputs from many different people with diverse expertise. One shortcoming of influence diagrams (and other methods) is that they do not specify a cohesive and comprehensive process for structuring the interactions of options, values, and uncertainties during the initial development of a decision model. This complicates (and sometimes precludes) the process of developing a comprehensive model because the model reaches a level of complexity that is very difficult for individual domain experts to think about.This paper introduces a formally specified method for eliciting influence diagram structure. We extend influence diagram notation to include special categories of value nodes that more explicitly define fundamental objectives hierarchy components. We then provide provably correct methods for decomposing the model, eliciting additional detail, and reassembling the model into a single graph. These methods make two important contributions to the modelling science: first, it is a necessary step toward providing automated model elicitation tools; second, it is an example of technology transfer from Bayesian Belief Networks to Influence Diagrams. Copyright © 2005 John Wiley & Son, Ltd.

We present in here validation studies of a new method for application in microgravity environment which measures the viscosity of highly viscous undercooled liquids using drop coalescence. The method has the advantage of avoiding heterogeneous nucleation at container walls caused by crystallization of undercooled liquids during processing. Homogeneous nucleation can also be avoided due to the rapidity of the measurement using this method. The technique relies on measurements from experiments conducted in near zero gravity environment as well as highly accurate analytical formulation for the coalescence process. The viscosity of the liquid is determined by allowing the computed free surface shape relaxation time to be adjusted in response to the measured free surface velocity for two coalescing drops. Results are presented from two sets of validation experiments for the method which were conducted on board aircraft flying parabolic trajectories. In these tests the viscosity of a highly viscous liquid, namely glycerin, was determined at different temperatures using the drop coalescence method described in here. The experiments measured the free surface velocity of two glycerin drops coalescing under the action of surface tension alone in low gravity environment using high speed photography. The liquid viscosity was determined by adjusting the computed free surface velocity values to the measured experimental data. The results of these experiments were found to agree reasonably well with the known viscosity for the test liquid used.

This paper describes the Value Added Analysis methodology which is used as part of the U.S. Army's Planning, Programming, Budgeting, and Execution System to assist the Army leadership in evaluating and prioritizing competing weapon system alternatives during the process of building the Army budget. The Value Added Analysis concept uses a family of models to estimate an alternative system's contribution to the Army's effectiveness using a multiattribute value hierarchy. A mathematical optimization model is then used to simultaneously determine an alternative's cost-benefit and to identify an optimal mix of weapon systems for inclusion in the Army budget. © 1999 John Wiley & Sons, Inc. Naval Research Logistics 46: 233–253, 1999

The method of drop coalescence is being investigated for use as a method for determining the viscosity of highly viscous undercooled liquids. Low gravity environment is necessary in this case to minimize the undesirable effects of body forces and liquid motion in levitated drops. Also, the low gravity environment will allow for investigating large liquid volumes which can lead to much higher accuracy for the viscosity calculations than possible under 1 - g conditions. The drop coalescence method is preferred over the drop oscillation technique since the latter method can only be applied for liquids with vanishingly small viscosities. The technique developed relies on both the highly accurate solution of the Navier-Stokes equations as well as on data from experiments conducted in near zero gravity environment. In the analytical aspect of the method two liquid volumes are brought into contact which will coalesce under the action of surface tension alone. The free surface geometry development as well as its velocity during coalescence which are obtained from numerical computations are compared with an analogous experimental model. The viscosity in the numerical computations is then adjusted to bring into agreement of the experimental results with the calculations. The true liquid viscosity is the one which brings the experiment closest to the calculations. Results are presented for method validation experiments performed recently on board the NASA/KC-135 aircraft. The numerical solution for this validation case was produced using the Boundary Element Method. In these tests the viscosity of a highly viscous liquid, in this case glycerine at room temperature, was determined to high degree of accuracy using the liquid coalescence method. These experiments gave very encouraging results which will be discussed together with plans for implementing the method in a shuttle flight experiment.

Abstract: Although most developing country cities are characterized by pockets ofsubstandard housing and inadequate service provision, it is not known to what degree lowincomes and malnutrition are confined to specific neighborhoods. This analysis usesrepresentative household surveys of Abidjan and Accra to quantify small-area clusteringin service provision, demographic characteristics, consumption, and nutrition. Both citiesshowed significant clustering in housing conditions but not in nutrition,...