Adaptive Loop Filter (ALF) is a new in-loop filter techniques introduced in the Versatile Video Coding (VVC) standard. It aims at boosting the reconstructed video quality based on Wiener-based filtering approach. Despite of its superior coding performance, the abundant information generated from the compression process has not been fully explored in the ALF. To achieve better texture restoration, this paper proposes an enhanced in-loop filtering scheme by utilizing the reconstructed residuals at the ALF stage. Subsequently, both the reconstructed samples and residuals are deployed as the guidance of the luma filtering. Objective compression results on top of the Enhanced Compression Model (ECM) are provided to demonstrate the coding efficiency benefits of proposed algorithm with negligible encoding/decoding complexity impacts.