Stable and robust walking in various environments is one of the most important abilities for a humanoid robot. This paper addresses walking pattern synthesis and sensory feedback control for humanoid stair climbing. The proposed stair-climbing gait is formulated to satisfy the environmental constraint, the kinematic constraint, and the stability constraint; the selection of the gait parameters is formulated as a constrained nonlinear optimization problem. The sensory feedback controller is phase dependent and consists of the torso attitude controller, zero moment point compensator, and impact reducer. The online learning scheme of the proposed feedback controller is based on a policy gradient reinforcement learning method, and the learned controller is robust against external disturbance. The effectiveness of our proposed method was confirmed by walking experiments on a 32-degree-of-freedom humanoid robot.