To improve the efficiency of surveying and mapping production, manual interpretation of polarimetric synthetic aperture radar (SAR) images have been widely used in the classification of surface features. Polarimetric SAR image quality assessment has become a significant study field. However, ordinary image quality assessment methods failing to take full account of the characteristics of pseudocolor and edge detection in polarimetric SAR images, neither takes account of the special needs of human visual system (HVS). Thus we advance a method of polarimetric SAR image quality assessment based on HVS and Structural Similarity (SSIM). It is improved by the ordinary image quality assessment and combines the feature of HVS and polarimetric SAR images. It is demonstrated by the experiments that this method could fit the features of HVS and provide an accurate image quality assessment focusing on the characteristics of polarimetric SAR images.