Pixel-based object recognition in fashion images to generate colour palettes

There is growing interest in being able to automatically extract colours of garments from images. Automatic image analysis may allow the development of data-driven approaches to, for example, colour forecasting. A neural network (pix2pix) was trained on streetstyle fashion images to predict the semantic class of each pixel in the image. The trained network was able to correctly identify the class of each pixel in 93% of cases. A total of 10 participants were each asked to select three colours from each of 10 additional images to represent the clothes being worn. Colour palettes were extracted from the images using cluster analysis of those pixels identified by pix2pix as being clothes and compared with cluster analysis of the whole image. The work shows that pixel-based semantic analysis is effective for automatically generating colour palettes for clothes in digital images. This approach can provide effective software tools for colour designers.