This study presented a new classification method for single person’s motion, which is represented by Haar wavelet transform and classified by Hidden Markov Model. We tackle the challenge of detecting the feature points by Haar wavelet transform to improve classification accuracy. We extract binary silhouette and segment them by cycle after creating the background model. Then the low-level features are detected by Haar wavelet transform and principal vectors are determined by Principal Component Analysis. We utilize Hidden Markov Models to train and classify cycle sequences and demonstrate their usability. Compared with others, our approach is simple and effective in feature point detection, strength in scale-invariant and generalized in different motions. Therefore, the video surveillance based on our method is practicable in (but not limited to) many scenarios where the background is known.