Optimization of signal sets for partial-response channels. I. Numerical techniques

Given a linear, time-invariant, discrete-time channel, the problem of constructing N input signals of finite length K that maximize minimum l₂ distance between pairs of outputs is considered. Two constraints on the input signals are considered: a power constraint on each of the N inputs (hard constraint) and an average power constraint over the entire set of inputs (soft constraint). The hard constraint, problem is equivalent to packing N points in an ellipsoid in min(K,N-1) dimensions to maximize the minimum Euclidean distance between pairs of points. Gradient-based numerical algorithms and a constructive technique based on dense lattices are used to find locally optimal solutions to the preceding signal design problems. Two numerical examples are shown for which the average spectrum of an optimized signal set resembles the water pouring spectrum that achieves Shannon capacity, assuming additive white Gaussian noise