Image processing has to deal with many ambiguous situations. Fuzzy set theory is a useful mathematical tool for handling the ambiguity or uncertainty. In order to apply the fuzzy theory, selecting the fuzzy region of membership function is a fundamental and important task. In this paper, a new method of membership function based on fuzzy theory by PSO algorithm optimized was proposed by analyzing the deficiencies of traditional enhancement algorithm. A new entropy definition of a fuzzy set was proposed. The new entropy definition of a fuzzy set was not only related to the membership (fuzzy domain) but also related to the probability distribution (space domain), it can respond to the variety of image input information. In addition, by quoting a novel particle swarm optimization (PSO) algorithm to find the optimization parameters for membership. We have employed the new proposed approach to perform image enhancement and thresholding and obtained satisfactory results.