In this study, the authors present a visual sensitivity-based low-bit-rate image compression algorithm. The authors algorithm combines both visual sensitivity and compression techniques so that a higher compression rate, with satisfactory visual quality, can be achieved. In the coding process, the input image is divided into blocks, and each block is classified as an edge block (EB), a textural block (TB) or a flat block (FB). For EBs, which are most important to the subjective quality of decoded images, the standard Joint Photographic Experts Group (JPEG) coding scheme with a tolerant quantisation step is employed so as to restrict the blocking artefacts caused by the quantisation error to an acceptable level. For FBs, a skipping scheme is employed on blocks in the compression process so as to save the bits. The coding of the skip blocks, identified by the skipping scheme, will make reference to the reconstructed regions of the image in the encoding process. Owing to the masking effects of the human visual system on high-frequency textures, standard JPEG compression coding with a greater quantisation step is employed on the down-scaled version of non-skip blocks and TBs. Experimental results show the superior performance of our method in terms of both compression efficiency and visual quality.