Gradient descent (GD) algorithm is the widely used optimisation method in training machine learning and deep learning models. In this paper, based on GD, Polyak's momentum (PM), and Nesterov accelerated gradient (NAG), we give the convergence of the algorithms from an initial value to the optimal value of an objective function in simple quadratic form. Based on the convergence property of the quadratic function, two sister sequences of NAG's iteration and parallel tangent methods in neural networks, the three-step accelerated gradient (TAG) algorithm is proposed, which has three sequences other than two sister sequences. To illustrate the performance of this algorithm, we compare the proposed algorithm with the three other algorithms in quadratic function, high-dimensional quadratic functions, and nonquadratic function. Then we consider to combine the TAG algorithm to the backpropagation algorithm and the stochastic gradient descent algorithm in deep learning. For conveniently facilitate the proposed algorithms, we rewite the R package 'neuralnet' and extend it to 'supneuralnet'. All kinds of deep learning algorithms in this paper are included in 'supneuralnet' package. Finally, we show our algorithms are superior to other algorithms in four case studies.

Random-effects Wiener degradation models have been widely used in the literature, to characterize unit-to-unit variability in a population. Bayesian inference for such models mainly relay on stochastic simulation techniques, such as Markov chain Monte Carlo (MCMC) method. However, MCMC approach will converge slowly or fail to converge for large volume data. In this paper, the variational Bayesian (VB) approach is proposed as an alternative tool for MCMC, which makes inference of Wiener degradation models with random effects more suitable for large inspection data sets. Despite of utilizing mean field methodology for VB approach, there are also certain limitations in model inference. Therefore, a three-step method is developed and embed into the VB approach, and statistical inference based on our VB approach is established. Numerical examples are provided to compare with MCMC method in terms of cost in time and accuracy in estimation for large inspection data problem. The proposed VB approach provides almost the same accuracy as MCMC, while its computational burden is much lower.

This chapter focuses on the study of early warning of failures or defects for a general class of highly reliable product testing data, which is accumulated in a time sequence. It proposes a Bayesian on-line method to make early warning of defects for highly reliable products. The test data are usually binary but with highly sparse defects or failures occurring in a long period of testing. Two ways are utilized to pre-process the test data: aggregating the test data by a fixed-time interval or by a fixed number of test specimens. The chapter presents a detailed early warning scheme based on the Bayesian predictive approach. A Bayesian on-line algorithm is presented to make early warning of defects based on the two models for the pre-processed the data from both schemes. The proposed algorithm is applied to a set of field mobile phone test data and turns out to be quite satisfactory.

Jelinski Moranda (JM) model is frequently used in software reliability. The objective Bayesian inference was proposed to estimate the parameters of JM model. Jeffreys prior and reference priors have been derived. Besides, the properties of corresponding posteriors were deduced and some modifications were made which made the posterior distributions proper. Then Gibbs sampling was utilized to obtain the Bayesian estimators, credible intervals and coverage probabilities of the parameters. Comparisons in the efficiency of the maximum likelihood estimators and Bayesian estimators under different priors for various sample sizes have been done by simulations and a real data set was analyzed for illustrative purpose.

Based on the angle of enterprise performance, the evaluation index system of S&T management personnel in the innovative enterprises is established, including the investment of innovative resources, the level of innovative techniques and the level of innovative proceeds. We adopt factor analysis to determine the weight of each index and the comprehensive score is obtained by weighted sum method. According to<The annual report data of 253 innovative enterprises of Zhejiang province in 2010>, we sort the innovative enterprises by their comprehensive scores and compare the average comprehensive scores of different cities in Zhejiang province. Finally, a number of improvement measures are proposed in this paper.

This paper establishes the model which describes hepatitis by ODE. It can calculate the equilibrium points of no disease and diseasedness and discuss the stability on the basis of classifying people into three groups: the susceptible, infectors, and people who are hepatitis-free. Eventually, we give numerical simulation in terms of the actual data of hepatitis and verify its practicability.

In this paper, we give algorithms to calculate multivariate wavelet coefficients approximately by sampled values of functions. Then we provide two formulae respectively to calculate the outlines and details of functions. Last, as an example, we draw figures of outline and detail of a function. 2000 AMS Classification: 42A65