It is well known that friction in haptic devices plays a key role in dissipating surplus energy to maintain passivity of the haptic system. This paper shows that the surplus energy can also be dissipated by careful exploitation of human operator's damping and analyzes the effects of the time-varying human arm impedance on the passivity of a haptic system. The human arm impedance is modeled as a second-order mass-damper-spring system. An impedance model is developed to describe the dynamic behavior of a haptic system that includes the human arm impedance. A new necessary and sufficient passivity condition of the haptic system is derived using an energy-based approach. The analytical results are experimentally validated using a one-degree-of-freedom haptic device. Experimental results show that the maximum achievable stiffness of the haptic system varies widely according to the degree of human arm impedance and is predictable more accurately using the newly derived passivity condition compared to the previous results in the literature.