Influences of pre-irradiation and thermal annealing on E' defect concentration are studied by measuring E' centers and optical absorptions using ESR and spectrophotometer, respectively. The results show that the E' defect concentration decreases significantly when silica glass is pre-irradiated by gamma rays and a subsequent thermal annealing. The effect of thermal annealing on the E' concentration has also been investigated for the temperature from 300degC to 1300degC. It is found from the ESR spectrum that the anti-irradiated property of the silica glass can be improved when the thermal annealing temperature is larger than 700degC. These results have been proved by optical absorption spectra.