This paper focuses on quantized channel state information (CSI) feedback for downlink network MIMO systems. Specifically, we propose to quantize and feedback the CSI of a subset of BSs, namely the feedback set. Our analysis reveals the tradeoff between better interference mitigation with large feedback set and high CSI quantization precision with small feedback set. Given the number of feedback bits and instantaneous/long-term channel conditions, each user optimizes its feedback set distributively according to the expected SINR derived from our analysis. Simulation results show that the proposed feedback adaptation scheme provides substantial performance gain over non-adaptive schemes, and is able to effectively exploit the benefits of network MIMO under various feedback bit budgets.