BACKGROUND:High-quality decision-making in the pharmaceutical industry requires accurate assessments of the Probability of Technical Success of clinical trials. Failure to do so will lead to lost opportunities for both patients and investors. Pharmaceutical companies employ different methodologies to determine Probability of Technical Success values. Some companies use power and assurance calculations; others prefer to use industry benchmarks with or without the overlay of subjective modulations. At AstraZeneca, both assurance calculations and industry benchmarks are used, and both methods are combined with modulations.METHODS:AstraZeneca has recently implemented a simple algorithm that allows for modulation of a Probability of Technical Success value. The algorithm is based on a set of multiple-choice questions. These questions cover a comprehensive set of issues that have historically been considered by AstraZeneca when subjective modulations to Probability of Technical Success values were made but do so in a much more structured way.RESULTS:A set of 57 phase 3 Probability of Technical Success assessments suggests that AstraZeneca's historical estimation of Probability of Technical Success has been reasonably accurate. A good correlation between the subjective modulation and the modulation algorithm was found. This latter observation, combined with the finding that historically AstraZeneca has been reasonably accurate in its estimation of Probability of Technical Success, gives confidence in the validity of the novel method.DISCUSSION:Although it is too early to demonstrate whether the method has improved the accuracy of company's Probability of Technical Success assessments, we present our data and analysis here in the hope that it may assist the pharmaceutical industry in addressing this key challenge. This new methodology, developed for pivotal studies, enables AstraZeneca to develop more consistent Probability of Technical Success assessments with less effort and can be used to adjust benchmarks as well as assurance calculations.CONCLUSION:The Probability of Technical Success modulation algorithm addresses several concerns generally associated with assurance calculations or benchmark without modulation: selection biases, situations where little relevant prior data are available and the difficulty to model many factors affecting study outcomes. As opposed to using industry benchmarks, the Probability of Technical Success modulation algorithm allows to accommodate project-specific considerations.

Clinical trials often have short follow-ups, and long-term outcomes such as survival must be extrapolated. Current extrapolation methods often produce a wide range of survival values. To minimize uncertainty in projections, we developed a novel method that incorporates formally elicited expert opinion in a Bayesian analysis and used it to extrapolate survival in the placebo arm of DAPA-CKD, a phase 3 trial of dapagliflozin in patients with chronic kidney disease (NCT03036150). A summary of mortality data from 13 studies that included DAPA-CKD-like populations and training on elicitation were provided to six experts. An elicitation survey was used to gather the experts’ 10- and 20-year survival estimates for patients in the placebo arm of DAPA-CKD. These estimates were combined with DAPA-CKD mortality and general population mortality (GPM) data in a Bayesian analysis to extrapolate long-term survival using seven parametric distributions. Results were compared with those from standard frequentist approaches (with and without GPM data) that do not incorporate expert opinion. The group expert-elicited estimate for 20-year survival was 31

Abstract Background and Aims Elevated albuminuria in patients with chronic kidney disease (CKD) is associated with increased risks of CKD progression, cardiovascular events and all-cause death. In the DAPA-CKD study, dapagliflozin significantly reduced the risk of all-cause death in patients with elevated albuminuria compared with placebo (hazard ratio: 0.69; 95% confidence interval 0.53–0.88). To assess the cost-effectiveness of new treatments, decision makers require survival estimates over a longer period than that of a typical clinical trial, usually over a lifetime time horizon. A formal elicitation process is currently underway to obtain estimates of long-term survival of patients with albuminuric CKD from clinical experts. Their responses will be used to validate extrapolations of all-cause mortality data from DAPA-CKD, which could inform cost-effectiveness analyses for dapagliflozin. Method Targeted literature searches were conducted to collate data on all-cause mortality in patients with CKD and elevated albuminuria. Clinical trials and observational studies were included if they involved non-dialysis-dependent patients with CKD aged 18 years and over, had more than 500 participants per study arm and reported incidence of all-cause death and/or all-cause mortality/survival Kaplan–Meier (KM) curves. To estimate long-term survival, KM curves were extrapolated to 20 years by calculating standard mortality ratios (SMRs) using age- and sex-adjusted general-population lifetable data. Study and patient characteristics and mortality data from relevant studies were provided to clinical experts to inform their judgements in a formal elicitation process. After receiving training on the elicitation process, six leading disease area experts were invited to complete the elicitation survey using an Excel-based tool, which consisted of 10 calibration questions, and three questions regarding the survival of patients in the placebo arm of the DAPA-CKD study at 10 and 20 years. The elicited estimates will be weighted and aggregated using Cooke’s method. Results Literature searches identified 13 relevant articles (seven clinical trials and six observational studies), with a range of 1094 to 5674 participants. Mean age varied across studies (range: 55–70 years). Where reported, median follow-up was 9–144 months, and mean estimated glomerular filtration rate (eGFR) at baseline was 22.4–56.3 mL/min/1.73 m2. Five studies exclusively included patients with type 2 diabetes (T2D). The incidence of all-cause death was reported in nine studies and was 1.5–9.4 deaths per 100 patient-years, with the highest incidence observed in a study reporting data for patients with CKD stage 4 and 5 (8.0 and 9.4 deaths per 100 patient-years, respectively). Nine studies provided KM curves; from these, estimated survival at 2 years ranged from 86% (study population mean age 67 years, eGFR < 15 mL/min/1.73 m2) to 98% (study population mean age 58 years, mean eGFR 46.2 mL/min/1.73 m2). The SMR-extrapolated survival at 10 and 20 years was 36–80% and 2–69%, respectively. The ranges defined by the expert judgements collected to date for survival at 10 and 20 years are in line with the variability of the extrapolated KM survival curves. The elicitation process is ongoing and therefore, to avoid biasing the judgements that remain to be collected, preliminary results are not reported here. Results of the expert elicitation will be presented in full at the congress. Conclusion Initial results from the survey calibration questions suggest that the expert elicitation process provides expert judgements that are both informative and precise. The elicitation of survival estimates for patients with CKD and elevated albuminuria at 10 and 20 years will provide greater insight than extrapolated data alone, and will increase the validity of long-term survival projections for dapagliflozin cost-effectiveness analyses.

Large investments in analytics demonstrate that the pharmaceutical industry has embraced the value proposition of data science. This excitement however does not imply that companies, currently, have a solid understanding how data science creates value. Management rely on data scientists for the value delivery of advanced analytics. Objectives of data scientists and management are not necessarily aligned. Choices made by data scientists might be suboptimal from a wholistic corporate perspective. Conversely management might lack technical expertise. This situation is an example of a principal-agent problem. AstraZeneca is making significant investments in analytical capabilities. AstraZeneca beliefs that investment decisions should not be strictly determined by monetary objectives, instead corporate Core Values should be used as guiding principles. The relationship between objectives and attributes are captured in an objective hierarchy network. This model reduces the information asymmetry between data scientists and its leaders by creating clarity regarding the objectives pursued by AstraZeneca.

Summary The exploration and production (E&P) industry is facing a net-present-value (NPV) paradox. Despite the fact that the NPV method is widely criticized by practitioners and academics alike, the NPV method remains the cornerstone of E&P project valuation. We posit that this contradiction, which we labeled the NPV paradox, is likely to be caused by a combination of limitations of the method, a lack of theoretical understanding, and ambiguity regarding the implementation of the NPV method. Even though the NPV method has been described in numerous papers and textbooks, rigorous and succinct guidance on how to determine risk premiums for systematic risk is not available. We demonstrate that risk-adjusted discount rates are very sensitive to the choice of the length of the periods over which returns are determined (daily, weekly, or monthly), length of the time horizons considered (such as 10 or 25 years), and start date (such as 1965 or 1990). We discuss the fundamental implications and rationale of choices and their effects on the variables that underpin the risk-adjusted discount rate: risk-free rate, company β, market-risk premium, and the cost of debt. Although not entirely satisfactory, we argue for a moderate downward revision of discount rates for projects with timelines exceeding 20 years. This recommendation is dependent on recent advancements in public finance and the reality that the exposure to systematic risk in the long run is significantly less in many real-life E&P projects than the capital-assessment-pricing model (CAPM) implies. The inflated discount rate that is currently used, combined with the extended investment horizons that are common in the upstream sector, will for example result in an underweighting of decommissioning and future legacy costs. In addition to a set of widely recognized shortcomings of the NPV model, there are also lesser well-known issues. For example, the failure of CAPM to capture bankruptcy risk has a bearing on the project-risk premium. Also, the application of the NPV model implies a set of probabilistic assumptions around market risks that are likely to be invalidated when evaluating a set of market scenarios or using a series of probability-weighted market scenarios.

The development of unconventional plays tends to unfold in many stages, each of which involves incremental investment and a reduction in the geological uncertainty of the reservoir. These two characteristics yield a large decision space where future decisions are optimised based on the near-continuous arrival of new information. This managerial flexibility can be exploited by operators during the development of unconventional plays.We introduce a methodology that demonstrates how value can be created by a staged and partial development of a shale play that would have been unprofitable if fully developed. Compared to existing methods the novel methodology is more consistent with the characteristics of how plays are currently developed as existing methods assume that upon a successful appraisal stage a play is developed in its entirety in a single development phase.As more data become available after each development phase of the play, the potential of the remaining undrilled locations is updated using Bayes's rule. The method is couched in geostatistical principles, combined with an algorithm that allows for a continuous optimisation of drilling targets.An example of a shale gas project has been investigated that consists of 225 possible drilling targets each containing 10 well locations. A maximum of 200 wells can be produced by drilling 20 of the 225 targets. The mean performance of the well population is uncertain. The scenario with the highest mean well performance yields a value of -160 MM USD and the expected project value across all scenarios of mean well performance equals -920 MM USD, given that all 200 wells are drilled at randomly chosen drilling targets. In the model presented in this study the resource can be developed in up to 19 stages upon completion of an appraisal programme. After each development stage an assessment is made where, and if, the next batch of 10 wells should be drilled. This strategy of stage-wise development yields an expected value of 49.2 MM USD. The spatial dependency of well performance enables the algorithm to restrict the development of the play to the most prolific areas.The appraisal programme provides a view on the variability of well performance across the play. A trade off exists between the size, and the consequential accuracy, of the appraisal programme and the cost of appraising. The example illustrates that the expected project value increases from 33.3 MM USD for an appraisal programme in which two locations were appraised, to a maximum of 49.2 MM USD after the appraisal of four locations, and subsequently decreases to 29.0 MM USD after having appraised eight locations.The assumptions around the variability of Estimated Ultimate Recovery (EUR) used in our example are informed by data from 10,000 horizontal wells located in the Mississippian Barnett Shale in the Fort Worth basin in Texas.

Summary Appraisal programs undertaken by exploration and production (E&P) companies are designed to resolve subsurface uncertainties that contribute to uncertainty in the economic potential of undeveloped fields. Value-of-information (VOI) assessments allow E&P players to quantify the economic value of their proposed appraisal programs before carrying them out. This study proposes a VOI methodology that is tuned to the nature of the subsurface uncertainties in unconventional plays and is capable of assessing a wide range of appraisal strategies (defined as the number and configuration of wells). It addresses two main problems. The first is how to characterize the uncertainty (in a play) that an appraisal program is intended to reduce or resolve. The second is how to cast that characterization of the uncertainty in a VOI context so that the merits of various appraisal programs can be evaluated. This paper characterizes the subsurface uncertainty that arises because of inadequate sampling of natural geologic variability. In this work, three quantities are assumed to be uncertain: the mean and standard deviation (SD) (variability) of the expected ultimate recovery (EUR) of the population of wells to be drilled, should development go ahead, and the range of the variogram that describes the spatial correlation of EUR as a function of the distance between wells. The optimal appraisal program would presumably depend on the true values of these quantities. The methodology is illustrated by application to a typical unconventional play. The VOI of an appraisal program can be optimized in terms of the number of the appraisal wells to be drilled and the placement of those wells. This VOI increases as the placement of the set of wells is changed from being clustered in the central part of the appraised area to approaching uniform distribution across the area. To obtain the optimal well placement, the incremental learning from changing the well locations should be balanced against the incremental costs that are required to increase the spacing of the appraisal wells. The study results demonstrate how the VOI of each incremental appraisal well decreases with the number of appraisal wells and how an optimal number of appraisal wells can be determined.

The 2011 UK tax rise on hydrocarbon exploitation activities obviously increases short term tax revenues however the longer term effects are less clear. The strategic interaction between the UK government, a producer and a shipper has been analyzed in a game theoretical model. A complex interaction between players is expected given (1) dwindling resources and large decommissioning liabilities and (2) the fact that much of the hydrocarbons produced in the North Sea are exported through an infrastructure with shared ownership.

Summary This study demonstrates how portfolio insights can be created by combining well-known optimization methods that are generally used individually—genetic algorithm (GA), linear programming (LP), and portfolio filtering (PF). An integrated optimization approach combines the advantages of individual methods while mitigating their shortcomings. Effective portfolio management requires a comprehensive understanding of the tradeoffs between different portfolio choices. Given a set of constraints, portfolio-optimization techniques based on LP and GAs can be applied to identify an optimal portfolio. However, this optimal portfolio might not be the preferred portfolio. Decision makers have to understand the tradeoffs between generally conflicting objectives and constraints before one portfolio can be identified as the preferred option. Such assessment of the overall search space is not made with LP and GAs when used to identify a single best solution, and many portfolio options will have been eliminated before an understanding of these alternatives has been developed. Markowitz's mean-variance (M-V) approach and the traditional “rank and cut” approach are used typically to establish a relationship between a portfolio's value and its variance or associated development cost. Although these methods enable decision makers to compare and contrast different options, the optimization is limited to the portfolio value measure and a single other metric. This latter limitation is overcome by the more recently developed PF approach. This method is practical and transparent and allows for a quick development of strategic portfolio alternatives while considering a large number of portfolio attributes. Its main drawback is that the analyzed set of portfolios generally represents a subset of the total search space. Thus, as the number of feasible portfolio options increases, so does the chance that the optimal portfolio is not present in the population of sampled portfolios.

Summary Natural gas and electricity are commonly traded through swing contracts that enable the buyer to exploit changes in market price or market demand by varying the quantity they receive from the producer (seller). The producer is assured of selling a minimum quantity at a fixed price, but must be able to meet the variable demand from the buyer. The flexibility of such contracts enables both parties to mitigate the risks and exploit the opportunities that arise from uncertainty in production, demand, price, and so on. But how valuable are they? Traditional net present value (NPV), based on expected values, cannot value this flexibility, and the traditional options/valuation techniques could not model the complexity of the terms of such contracts. Taking gas contracts as an example, this paper seeks to (a) raise awareness of how flexibility creates value for both parties and (b) show how least-squares Monte Carlo (LSM) simulation can be used to quantify its value in dollar terms, from the perspective of both producer and buyer. Because the value of flexibility arises from the ability it gives to respond to fluctuations (e.g., in commodity prices), a useful model of swing contracts needs to reflect the nature of these fluctuations.

Summary The process of portfolio optimization provides guidance to decision makers on how to manage an asset base given corporate objectives, market conditions, and organizational capability. Many applications in the oil and gas industry are based upon Markowitz's (1952) efficient-portfolio theory. In the standard implementation of this framework, an efficient portfolio is defined as one that yields the highest value given a specific degree of risk. A corporate decision maker will aim, however, to select a portfolio that meets several often-competing objectives (i.e., maximize portfolio value while minimizing capital expenditure). The optimal portfolio choice given one constraint is typically not optimal given one of the competing constraints. This requires the portfolio manager to identify and select those portfolios that best meet all corporate constraints. Deciding which portfolio to develop is often compounded by there being several portfolios having similar economic characteristics. However, these portfolios can generally be differentiated by strategy, which may depend on nonfinancial attributes such as the geographic location of the assets or on geological settings that might require different engineering expertise. In this study, a large set of exploration portfolios and their attributes have been simulated. Through applying a series of simple and transparent filters, a few portfolios can be identified that meet all the corporate constraints. After a shortlist has been created, the portfolios can easily be characterized by strategy, and the tradeoffs between them can be assessed.

Decision making in the capital-intensive upstream oil and gas industry is complex for several reasons. One is the uncertainty of the investment opportunities. Another is that many projects are developed in joint ventures, in which stakeholders with potentially contrasting preferences must reach mutual agreements on the decisions at hand. In order for players to be successful in a joint venture, each player should understand the preferences, the positions, and the exposure of all other players.This study provides insight into the type of strategic interactions to be expected in typical joint-development programs in the upstream oil and gas industry. The decision situation considered involves a joint venture of three oil fields connected by a shared infrastructure used to export the produced hydrocarbons. A game-theoretic framework has been applied to analyze the relationships among players' preferences, uncertainties resolution, and commercial drivers. An improved understanding of the evolution of the players' project perspectives during the project development period will enable decision makers to be much more effective in influencing the project to their advantage. Understanding the preferences and tradeoffs of all of the joint venture participants will lead to improvements in selecting investment alternatives, timing and order of the investments, and the mitigation project upsides and downsides. (C) 2010 Elsevier B.V. All rights reserved.

Abstract One of the great successes of modern financial risk modelling is the application of computing technology to simulate complex continuous probability distributions associated with the value metrics of real-life assets. The Monte Carlo simulation technique is the best known example and is widely applied by economists in the E&P industry. However a Monte Carlo simulation in which project economics are aggregated into a large asset portfolio is rarely undertaken. The main reason is that present computing systems cannot handle the vast amounts of data generated in a Monte Carlo simulation of a large asset portfolio. A pragmatic solution for this issue might be to approximate the continuous distribution of feasible project outcomes using a small number of probability-weighted discrete scenarios. In a corporate portfolio simulation, a project sample would be drawn from these discrete distributions as opposed to the original continuous distributions. If either a global assumption needs revision or a single asset requires recalculating, the economics of a relatively small number of scenarios would be computed. Thus, there is no need to store or recalculate a large number of outcomes (typically more then 2,000) for each project, as would be required in a conventional Monte Carlo simulation. A prerequisite for this approach is that the frequently skewed and complex continuous probability distributions of each of the assets can precisely be described by a small number of scenarios. This study documents the level of precision that can be achieved using Swanson's rule and variations thereof. Our analyses suggest that if the P50 is at least 33% greater than the P10, the simulated portfolio mean and standard deviation are within 5% and 15% of the actual values, respectively. The approximation of the lower end of the distribution, i.e., the downside risk of a portfolio, is within 4% even for an extremely asymmetric lognormal distribution. The proposed portfolio simulation methodology addresses the largely unmet need of corporate managers to improve their understanding of key risk and value drivers and their impact on the performance of a corporate asset portfolio.

Abstract As decision-making processes in the E&P industry increasingly rely on probabilistic economic models, determining the accuracy of its methodologies becomes more problematic. Enhancements can be achieved by (1) better understanding the interdependencies between different sources of uncertainty and (2) the abandonment of fixed time series of either hydrocarbon prices or capital expenditures. Historical market data of hydrocarbon prices, steel prices, and daily rig rental rates can be used to establish the correlation between different sources of market risk. Uncertainties can be defined as "fixed" or "dynamic." Fixed uncertainties relate to factors that do not change over time, such as many geological parameters during the early stages of exploration. Most uncertainties that relate to market risk are dynamic, that is, they keep developing over time. For example, not only is the realized price of oil uncertain until the moment the oil has been sold, but the expectation of future oil prices changes. The recognition of this Bayesian property of hydrocarbon prices significantly affects projects with multiple decision points. The forecasted hydrocarbon price at a future decision point is a function of the simulated realized price at that given decision point. Traditional decision tree models apply the same series of static price decks at each decision point and therefore do not accurately reflect the impact of the evolving market outlook during the development of a project. The stochastic model developed in this study accounts for (1) the correlation between different uncertainties and (2) Bayesian price-cost forecasts. The versatility of the Least-Squares Monte Carlo simulation technique is demonstrated by a real option valuation of an asset subjected to a complex tax regime and two future stage-gate decision points.

Least-squares Monte Carlo simulation (LSM) is a promising new technique for valuing real options that has received little or no attention in the pharmaceutical industry. This study demonstrates that LSM can handle complex valuation situations with multiple uncertainties and compounded American-type options. The limited application of real option valuation (ROV) in the pharmaceutical industry is remarkable, given the importance of accurate project valuation in an industry that requires large investments in high-risk projects with long pay-back periods, which is furthermore suffering from ever-increasing development costs and shrinking profit margins. The LSM model developed in this study is constructed as an extension of a discounted cash flow model that should be familiar to economists active in the pharmaceutical industry. A number of pharmaceutical projects have been evaluated using LSM ROV, binominal real option valuation and expected net present value techniques. The different results yielded by these methods are explained in terms of differences in risking assumptions and ability to capture the value of flexibility. The analysis provides a framework to introduce the basic concepts of real option pricing to a non-specialist audience. The LSM model illustrates the potential for real-life commercial assessment as the versatility of the technique allows for an easy customisation to specific business problems.

A new technique is presented where mass fractionation during Rb isotope dilution analyses by multi-collector inductively coupled plasma mass spectrometry is corrected for by measuring the amount of fractionation on admixed Zr. Replicate analyses of natural Rb interspersed with analyses of 87Rb tracer enriched samples yield a mean 87Rb/85Rb=0.38540±19 (0.05%, 2 s.d.), assuming a natural 90Zr/91Zr of 4.588. Each Rb analysis takes 1 min, consumes 20 ng of Rb and has an internal precision of ∼0.02% (2 s.e.). Washouts between samples take 5 min. Persistent but small stable Rb backgrounds are overcome by an on-peak-zeroes (OPZ) measurement prior to data acquisition. Close examination of measured 87Rb/85Rb and 90Zr/91Zr ratios indicate small changes in relative fractionation of Rb and Zr during plasma ionisation occur when different sample introduction techniques are used (e.g., ‘wet’ vs. ‘dry’ nebulisation), although the differences are insignificant compared to the level of precision required for isotope dilution measurements. Replicate analyses of whole rock samples suggest a reproducibility for Rb concentration measurements of ≤0.5% and 87Rb/86Sr measurements of 0.2% when interfering Sr is reduced to satisfactory levels. However, it is difficult to ascertain to what extent this reproducibility reflects the limit of the technique or powder heterogeneity. Much of the error involved in the Rb isotope dilution and Sr isotope ratio measurements by multiple collector inductively coupled plasma mass spectrometry (MC-ICPMS) is derived from uncertainties as to which 87Sr/86Sr (and 87Rb/85Rb) ratios to use when correcting for isobaric interferences due to the presence of spike Sr and Rb at mass 87. If isobaric interferences are minimised by efficient separation of Rb from Sr during cation exchange chemistry, the use of natural ratios for isobaric interference corrections yields the most reproducible data, indicating that the interferences are derived from environmental blank. Larger isobaric interferences at mass 87 are indicative of inefficient chemical separations, and the measured ratio from the complementary analysis provide more reproducible data. Burning off of Rb during conventional thermal ionisation mass spectrometry (TIMS) Sr isotope analysis nullifies this isobaric interference, and therefore, TIMS remains the method of choice for reliable and precise 87Sr/86Sr determinations on spiked samples. Application of our technique to minerals separated from Tertiary to Palaeozoic plutons yields age data consistent with previous determinations. Where different two-point isochron ages can be calculated for individual plutons, the ages reproduce to ≤±0.3%. The method represents an initial improvement in Rb isotope dilution measurements over TIMS by allowing a quantifiable correction to be made for mass fractionation, confirmed by duplicate analyses of standards and samples by both TIMS and MC-ICPMS. Mass fractionation corrected Rb isotope dilution analyses should result in: (1) improved Rb–Sr geochronology in examples where the Rb–Sr ratio provides the largest source of error; (2) application of this improved method to Rb–Sr geochronology on smaller samples such as single mica-flakes and micro-drill samples and; (3) by comparison with other geochronological techniques, more detailed cooling and crystallisation histories of igneous and metamorphic rocks. Taking advantage of these improvements requires a reevaluation of the Rb decay constant, which this technique should also permit.