Consider a communication system whereby multiple users share a common frequency band and must choose their transmit power spectra jointly in response to physical channel conditions including the effects of interference. The goal of the users is to maximize a system-wide utility function (e.g., weighted sum-rate of all users), subject to individual power constraints. A popular approach to solve the discretized version of this nonconvex problem is by Lagrangian dual relaxation. Unfortunately the discretized spectrum management problem is NP-hard and its Lagrangian dual is in general not equivalent to the primal formulation due to a positive duality gap. In this paper, we use a convexity result of Lyapunov to estimate the size of duality gap for the discretized spectrum management problem and show that the duality gap vanishes asymptotically at the rate O (1/radicN), where N is the size of the uniform discretization of the shared spectrum. If the channels are frequency flat, the duality gap estimate improves to O (1/N) . Moreover, when restricted to the FDMA spectrum sharing strategies, we show that the Lagrangian dual relaxation, combined with a linear programming scheme, can generate an epsiv-optimal solution for the continuous formulation of the spectrum management problem in polynomial time for any epsiv>0.