We propose an unsupervised object category learning approach, where the output representations improve classification performance too. The contribution is threefold: we integrate a ’Network in Network’ (NIN) approach in unsupervised learning, improving the representational power of the network for local patches by replacing linear filters with micro convolutional networks. We learn receptive fields to connect layers of the network sparsely, and we propose a new encoding function that introduces sparsity in a natural way and avoids the necessity of parameter tuning. The learned model generates a feature representation of images, used for unsupervised category learning. Results demonstrate that the obtained image categories reflect true object categories well. In addition, experimental results on classification tasks show superiority of the proposed approach in comparison with an unsupervised stateof-the-art learning architecture.