Reliable crop yield estimation is fundamental to food security and efficient agricultural management. However, current deep learning models still face limitations in selecting and integrating multi-source features, and their high predictive accuracy is often accompanied by limited interpretability. This study introduces a Bayesian Optimization–Temporal Convolutional Network–Bidirectional Long Short-Term Memory–Dual Attention (BO-TCBDA) deep learning framework for winter wheat yield estimation. Using Henan Province, China, as the study area, county-level winter wheat yield from 2013 to 2022 was estimated using the Enhanced Vegetation Index (EVI), Leaf Area Index (LAI), Solar-Induced Chlorophyll Fluorescence (SIF), and climate data. The proposed model was compared with five commonly used machine learning and deep learning models. BO-TCBDA achieved the best performance, with an R2 of 0.823 and an RMSE of 561.26 kg/ha. SIF improved the predictive performance of all models, with statistically significant gains observed in the deep learning models. The dual-attention mechanism provided interpretable insights by revealing relatively balanced contributions among the input features and highlighting the grain-filling stage through temporal attention. Furthermore, SHAP-based cross-validation analysis identified T12, corresponding to the latter part of the jointing stage, as the period with the highest contribution to yield prediction. The model also achieved an R2 of approximately 0.80 about 25 days before harvest. Overall, BO-TCBDA provides an accurate and interpretable approach for county-level winter wheat yield estimation and supports regional food security assessments and precision agriculture.