In 2011, Senthilnath et al. proposed to utilize the Firefly Algorithm for K-means clustering. The algorithm has shown better results than the standard Kmeans algorithm or other combinations with bio-inspired optimization heuristics. In this study, we propose a further improvement of the method, based on an improved firefly algorithm. As a key aspect, the randomization parameter in our proposed algorithm is changed when the assignment does not change. We compare the standard K-means algorithm, K-means using the conventional Firefly Algorithm and our proposed algorithm on the basis of a simple data distribution. Numerical experiments show that our proposed algorithm is more efficient than the other algorithms.