Contact dynamics are commonly formulated as a linear complementarity problem. While this approach is superior to earlier spring-damper models, it can be inaccurate due to pyramid approximations to the friction cone, and inefficient due to lack of convexity coupled with a large number of auxiliary variables. Here we propose a new approach: implicit complementarity. Instead of treating contact velocities and forces as independent variables subject to explicit complementarity constraints, we express them as functions of a minimal set of unconstrained variables, and design these functions so that the complementarity constraints are automatically satisfied. We then solve the equations of motion via a non-smooth Gauss-Newton method augmented with an original linesearch procedure which exploits the problem structure. This enables us to represent the friction cone exactly and to reduce the number of unknowns by about a factor of 3. Numerical tests suggest that, in usage scenarios typical for robotics, the solver takes only about 5 iterations even without warm starts. More extensive tests and side-by-side comparisons remain to be done, but nevertheless the potential of the new approach is clear.