Currently, there is color deviation between the image color collected by the image sensor and the real color of the object itself. This makes people to get information through the image inconvenience. Aiming at the problem of image color shift, a color restoration method is proposed. This method first confirms the ideal RGB value under the current illumination condition, then obtains the set of color correction matrix by the iterative method, and finally selects the optimal color correction matrix by calculating the average variance of each color reproduction matrix, and the original image is processed by the optimal correction matrix. The results show that the method can effectively correct the image color deviation, so that the image sensor to collect the image closer to the object itself color.