A smart CCD image sensor with real-time programmable parallel convolution capabilities

A fully parallel focal plane charge-coupled device (CCD) array which performs image acquisition and convolution with arbitrary kernels is presented. The real-time programmable spatial convolution is generated for all pixels in parallel during the exposure. The 2-D convolution is performed by shifting a charge pattern in two dimensions and the exposure time is varied in proportion to the weight of each kernel coefficient. The problem of negative weights can be solved by taking the difference of two convolutions each with only positive weights. The CCD was fabricated using a standard CMOS/CCD process. Convolutions have been performed with a variety of linear filters that are commonly used in machine vision. Typical rms deviations from the ideal filter characteristics are between 1-2% of the largest kernel tap value. Results and practical applications of this work are discussed.