Haptic texture modeling is essential for immersive environments, yet accurate texture vibration synthesis remains challenging due to high-frequency nature and strong dependence on interaction parameters, such as speed and force. Traditional piecewise approaches handle this variability through algorithmic segmentation of signals into stationary components, followed by labeling across discrete contact conditions. However, the resulting segmentation overhead and postmodeling validation create a procedural bottleneck, limiting their ability to represent continuous user interactions. Recent deep learning methods eliminate explicit segmentation by learning continuous mappings, but their high computational cost hinders real-time deployment. To address these limitations, a lightweight Fourier-enhanced Transformer encoder network is proposed that eliminates segmentation and enables real-time texture rendering while maintaining high reconstruction fidelity. The model integrates a transformer encoder with the Fourier transform to process uniform sliding window inputs, separating spectral magnitude and phase in a Fourier block to capture texture-dependent frequency structure, while temporal features are extracted in a compact encoder for one-step prediction at approximately 1 kHz. The proposed approach was evaluated on a diverse set of textures, achieving strong reconstruction accuracy (time domain: mean absolute error 0.149; spectral domain: goodness-of-fit criterion 92.15%). A psychophysical experiment further confirms its perceptual effectiveness against existing methods when comparing real and synthesized textures.