State-of-art statistical models for object recognition are sensitive to large affine transformations in scale and rotation. This is due to the rather strict spacial correlation assumption made between model parts that cannot be solved using distance transform. In this work, to tackle these problems we take a different approach by utilizing topological properties of a graphical object model. Our model is a part-based generative model with each part representing a patch of an object image. The graph structure is learned by Boosting the graph nodes additively so that new parts are added in until a threshold is reached. Every time when a new part is inserted, all other parts are updated accordingly. The result of this process gives us a reliable and flexible graph structure that encodes local topological information of an object appearance. The recognition step is performed using two different methods: either by exhaustive search of a topological graph which best matches the model, or by fitting the model incrementally starting from selecting the most reliable (rather than most dominant from training) part candidates from the test image.