We propose a method to accelerate Yang's real-time O(1) bilateral filtering algorithm, based on the observation that in the original algorithm, some of the computation can be strategically eliminated. To identify such computation, the algorithm steps are analyzed in conjunction with its recursive Gaussian filtering component. By block partitioning the image, the procedure to isolate these unnecessary computation is simplified, and the proposed algorithm only needs to skip some of the image blocks when performing recursive linear filtering. The resultant accelerated algorithm is able to achieve 1.5~5 times speedup, depending on the image statistics and the filtering parameters. The proposed algorithm only marginally degrades the accuracy of the filtering, and the simplicity and small memory footprint of Yang's original algorithm are largely maintained.