In this brief, a power-based time domain passivity control is presented. The new passivity observer (PO) monitors the current power behavior and decides the activation of the passivity controller (PC). The PC output is distributed along the time index and sudden big force change is alleviated. Applications of the approach to haptic interfaces are simulated. Extensive comparisons with the two existing energy based approaches are made. Both simulation results and experimental results show that the performance of the proposed approach is promising in terms of stability and fidelity.