SLiKER: Sparse loss induced kernel ensemble regression

•We develop a novel regression method based on kernel trick and ensemble principle. Its merit is that multi-kernel selection and parameter decision can be conducted automatically through a pool of kernels.•In our proposed method, we introduce sparsity to evaluate the quality of the model. With this sparsity model, well-behaved regressors are selected and the impacts of badly-behaved regressors are decreased.•Experimental results on UCI regression and computer vision datasets indicate that compared to other regression ensemble methods, such as random forest and XGBoost, our method has the advantages of best performances in keeping lowest regression loss and highest classification accuracy.