This is an in-memoriam honoring Professor Darrell W. Pepper as an exceptional researcher, educator, and engineer.

The determination of the transverse tip deflection of an elastic, hollow, tapered, cantilever, box beam under a uniform loading applied over half the length of the beam presented in the V&V10.1 standard is used to compare the application of the validation procedures presented in the V&V10.1 and V&V20 standards. Both procedures aim to estimate the modeling error of the mathematical/computational model used in the simulations taking into account the variability of the modulus of elasticity of the material used in the beam and the rotational flexibility at the clamped end of the beam. The paper discusses the four steps of the two error quantification procedures: (1) characterization of the problem including all the assumptions and approximations made to obtain the experimental and simulation data; (2) selection of the validation variable; (3) determination of the different quantities required by the validation metrics in the two error quantification procedures; (4) outcome of the two validation procedures and its discussion. The paper also discusses the inclusion of experimental, input, and numerical uncertainties (assumed or demonstrated to be negligible in V&V10.1) in the two validation approaches. This simple exercise shows that different choices are made in the two alternative approaches, which lead to different ways of characterizing the modeling error. The topics of accuracy requirements and validation comparisons (model acceptance/rejection) for engineering applications are not addressed in this paper.

This paper discusses the application of the Area Metric to the quantification of modeling errors. The focus of the discussion is the effect of the shape of the two distributions on the result produced by the Area Metric. Two different examples that assume negligible experimental and numerical errors are presented: the first case has experimental and simulated quantities of interest defined by normal distributions that require the definition of a mean value and a standard deviation; the second example is taken from the V&V10.1 ASME Standard that applies the Area Metric to quantify the modeling error of the tip deflection of a loaded hollow tapered cantilever beam simulated with the static Bernoulli-Euler beam theory. The first example, shows that relatively small differences between the mean values are sufficient for the area metric to be insensitive to the standard deviation. Furthermore, the example of the V&V10.1 ASME Standard produces an Area Metric equal to the difference between the mean values of experiments and simulations. Therefore, the error quantification is reduced to a single number that is obtained from a simple difference of two mean values. This means that the Area Metric fails to reflect a dependence for the difference in the shape of the distributions representing variability. The paper also presents an alternative version of the Area Metric that does not filter the effect of the shape of the distributions by utilizing a reference simulation that has the same mean value of the experiments. This means that the quantification of the modeling error will have contributions from the difference in mean values and from the shape of the distributions.

The goal of this paper is to summarize and clarify the scope and interpretation of the validation procedure presented in the V&V20-2009 ASME Standard. In V&V20-2009, validation is an assessment of the model error, without regard to the assessment satisfying validation requirements. Therefore, validation is not considered as a pass/fail exercise. The purpose of the validation procedure is the estimation of the accuracy of a mathematical model for specified validation variables (also known as quantities of interest, system responses or figures of merit) at a specified validation point for cases in which the conditions of the actual experiment are simulated. The proposed procedure can be applied to variables defined by a scalar. For the sake of clarity, the paper reiterates the development and assumptions behind the V&V20-2009 procedure that requires the knowledge of the experimental values D and simulation values S at the set point and an estimate of the experimental, numerical and parameter uncertainties. The difference E between S and D is the centre of the interval that should contain the model error (with a certain degree of confidence) and the width of the interval is obtained from the validation uncertainty that is a consequence of the combination of the experimental, numerical and parameter uncertainties. The paper presents the alternatives to address parameter uncertainty and expands upon the interpretation of the final result. The paper also includes two examples demonstrating the application of the V&V20-2009 validation procedure including one problem from V&V10.1-2012 on solid mechanics.

Thermophysical properties including density, specific heat, and thermal diffusivity of a poly (diallyl phthalate) inert filler composite material were characterized over a wide temperature range from room temperature to 800 °C. Over this temperature range, the material decomposition was approximated by a one-step process with first-order kinetics. Thermal kinetics data were obtained by thermal gravimetric analysis with Fourier transform infrared spectroscopy (TGA-FTIR) and thermophysical properties were obtained from differential scanning calorimetry (DSC) and laser flash diffusivity experiments. The response of the material to radiant heating was simulated with a computational heat transfer, multidimensional, finite element code. Additionally, the experimental uncertainty in the measurements was quantified to estimate the uncertainty in the reaction parameters due to heating rate and variability in inert filler-polymer composition in large sample sizes. Hence, the thermal response and the uncertainty were quantified for a complex decomposing material in a practical geometry for technologically important applications.

The development of a manufactured solution for enclosure radiation in an infinitely long circular cylinder with a nonparticipating medium is presented. This solution is then used to verify the correct implementation of the commonly used discrete enclosure equations. The circular cross section is approximated by a faceted geometry; the numbers of facets used are 4, 8, 16, 32, 64, and 128. The crossed-string method, which is exact in this application, is used to compute the view factors. Computational results using six levels of grid refinement suggest that the error norm between the integral equation solution and the discrete equation solution behaves as h 2 where h is a characteristic mesh size.

This paper describes the thermal problem and presents the experimental data for validation. The thermal problem involves validating a model for heat conduction in a solid. The mathematical model is based on one-dimensional, linear heat conduction in a solid slab, with heat flux boundary conditions. Experimental data from a series of material characterization, validation, and accreditation experiments related to the mathematical model are provided. The objective is to use the series of experiments to assess the model, and then use the model to predict regulatory performance relative to a regulatory requirement. The regulatory requirement is defined in terms of the probability that a surface temperature not exceed a specified temperature at the regulatory conditions.

This paper presents an engineering approach to the thermal challenge problem defined by Dowding et al. (this issue). This approach to model validation is based on a multivariate validation metric that accounts for model parameter uncertainty and correlation between multiple measurement/prediction differences. The effect of model parameter uncertainty is accounted for through first-order sensitivity analysis for the ensemble/validation tests, and first-order sensitivity analysis and Monte-Carlo analysis for the regulatory prediction. While sensitivity based approaches are less computational expensive than Monte-Carlo approaches, they are less likely to capture the far tail behavior of even mildly nonlinear models.The application of the sensitivity based validation metric provided strong evidence that the tested model was not consistent with the experimental data. The use of a temperature dependent effective conductivity with the linear model resulted in model predictions that were consistent with the data. The correlation structure of the model was used to pool the prediction/measurement differences to evaluate the corresponding cumulative density function (CDF). Both the experimental CDF and the predicted CDFs indicated that the regulatory criterion was not met.

Coupled fire-environment/thermal-response models were validated using data for an object engulfed in a JP8 hydrocarbon fuel fire. Fire model predictions of heat flux were used as boundary conditions in the thermal response calculations of the object. Predictions of transient external shell temperatures as well as the surface temperatures of the embedded mass were averaged spatially and compared to data. The solution sensitivity to mesh size, time step, nonlinear iterations, and radiation rays were assessed and the uncertainties in the predictions were quantified using a Latin Hypercube Sampling (LHS) technique. The comparisons showed that the response variable was more sensitive to fire model parameters than to thermal model parameters. The observed relative difference in measurements and model predictions was also compared to the model uncertainty. The comparisons showed that the model plus uncertainty bounded the experimental data. I. Introduction Sandia National Laboratories has been engaged in testing weapon system safety in fire environments since the 1950s. Due to the high consequences involved, system safety has traditionally been demonstrated through full scale system tests, albeit with a limited number of tests. Historically developed standardized tests include the placement of a system in a fully engulfing fire for 1 hour. Systems are declared qualified and ready for production based on passage of these standardized tests and with reference to the testing and analysis during development. Beginning in the early to mid 1990’s, the DOE began a program of Science Based Stockpile Stewardship. A significant part of this program is the Advanced Simulation and Computing (ASC) program, in which modeling and simulation, through high performance computing has been applied to system development and qualification. As part of the ASC program, Sandia engaged in developing the capability to model fire environments coupled to system response in those environments. An important thrust area within the ASC program includes the advancement of the verification and validation (V&V) methodologies and uncertainty quantification techniques. Sandia National Laboratories has made strides in developing new capabilities in this area and applying them to current applications. A best estimate plus uncertainty approach has been fully adopted and incorporated into safety themes for system qualification. Providing uncertainty estimates along with deterministic results has provided value to Sandia programs and gives more insight into predictive capability. The direct contribution of this study to current and future systems is an understanding of the uncertainties in predicting internal system temperatures when an object is engulfed in a JP8 fire environment. The uncertainty in input parameters can be used with other scenarios and configurations to evaluate situations that challenge safety themes. Confidence gained in validation processes such as discussed in the current work is crucial when evaluating system qualification activities that include modeling and simulation. II. Numerical Modeling

This special issue presents the results of the Sandia organized Model Validation Challenge Workshop, held May 2006. The workshop brought together researchers from different fields to present various approaches to model validation, and focused on the methodological elements of model validation rather than on model building. Three problems were defined in the disciplines of structural statics, structural dynamics, and heat transfer, all with a uniform structure. The workshop was specifically designed to investigate the relative merits of different approaches to hierarchal model validation through application to these problems. This paper describes a hierarchal approach in the challenge problems, presents the uniform conceptual framework that was used for the challenge problem definitions, and provides an overview of the organization of this special issue.

Calore is the ASC code developed to model steady and transient thermal diffusion with chemistry and dynamic enclosure radiation. An integral part of the software development process is code verification, which addresses the question 'Are we correctly solving the model equations'? This process aids the developers in that it identifies potential software bugs and gives the thermal analyst confidence that a properly prepared input will produce satisfactory output. Grid refinement studies have been performed on problems for which we have analytical solutions. In this talk, the code verification process is overviewed and recent results are presented. Recent verification studies have focused on transient nonlinear heat conduction and verifying algorithms associated with (tied) contact and adaptive mesh refinement. In addition, an approach to measure the coverage of the verification test suite relative to intended code applications is discussed.