The Hough transform (HT) is a commonly used technique for the identification of straight lines in an image. The Hough transform can be equivalently computed using the Radon transform (RT), by performing line detection in the frequency domain through use of central-slice theorem. In this research, an advanced Radon transform is developed using a multilayer fractional Fourier transform, a Cartesian-to-polar mapping, and 1-D inverse Fourier transforms, followed by peak detection in the sinogram. The multilayer fractional Fourier transform achieves a more accurate sampling in the frequency domain, and requires no zero padding at the stage of Cartesian-to-polar coordinate mapping. Our experiments were conducted on mix-shape images, noisy images, mixed-thickness lines and a large data set consisting of 751 000 handwritten Chinese characters. The experimental results have shown that our proposed method outperforms all known representative line detection methods based on the standard Hough transform or the Fourier transform.