Target tracking and detection technology is a hot and difficult problem in the field of computer vision research, which is widely used in guidance, navigation, monitoring and so on. Especially in the complex background, the accurate tracking of the target, in the occlusion of the stable tracking, is a hot research topic in recent years. By using a feature point cluster method, extracting feature points and estimating the target motion etc., solve the ground target under the light changes, scaling, rotation, and three variations of tracking and detection. This method has less limitation to the ground moving target, and the displacement of the moving object is smaller in the image sequence at the same time. By using the proposed algorithm, we can solve the problem of tracking and detecting the ground targets in the case of light variation, scaling and rotation.