Artifact reduction for image style transfer

An image processing system transforms content images into the style of another reference style image. For example, the system applies a noise mask to generate noisy versions of the content images. The system then recomposes a content image in the style of a reference image by applying computer models to the noisy version of the content image, which reduces artifacts in the stylized image compared to that of a stylized image generated by applying the computer models to the original content image. When the content images are part of a video sequence, the image processing system may adjust the noise mask applied in a subsequent frame such that it tracks the movement of the client device from the current frame to the subsequent frame. This allows the system to reduce artifacts while stylizing the frames of the video in a consistent manner.