3D reconstruction is one of the hot researches in the field of computer vision, remote sensing, photogrammetry and so on. Since the topdown reconstruction approach starts from the overall structure of the building, it is not susceptible to noise disturbance. In order to reconstruct buildings from a single high resolution remote sensing image, a topdown building reconstruction method is proposed. In the process of optimization, special transition kernels are designed in order to achieve convergent extraction results and get rid of local minimum. Experimental results show that the proposed method can reconstruct building more accurately than method’s only using basic transition kernels by using IKONOS 1 m resolution on remote sensing images and almost 1 m resolution on simulated images. Moreover, it can get rid of local minimum and has strong robustness with respect to the disturbance of background noise.