The goal of the objective image quality assessment is to quantitatively measure the image quality of a distorted image, which is close to the subjective image quality assessment such as the mean opinion score. The image quality assessment algorithms are generally classified into two methodologies: perceptual and structural. This paper proposes a structural information-based image quality assessment algorithm, in which LU factorization is used for representation of the structural information of an image. The proposed algorithm performs LU factorization of reference and distorted images, from which the distortion map is computed for measuring the quality of the distorted image. Finally, the proposed image quality metric is computed from the two-dimensional distortion map. Experimental results with the LIVE database images show the efficiency of the proposed method. In commercial systems, the proposed algorithm can be used for quality assessment of mobile contents and video coding, which effectively replaces the peak signal to noise ratio or the mean square error.