This paper explores energy efficient building design for manufacturing plants. Many efforts have been directed into the field of building design optimization concerning building energy performance, but most of the studies focus on residential buildings or public buildings. Very limited research results studying plants buildings have been reported. However, plants buildings have certain unique features that make the design problem more challenging. Furthermore, the approaches presented in the current publications could not guarantee the performance of their designs if the computation capacity is limited. This paper attempts to address these two issues. First, an EnergyPlus-integrated overall energy consumption estimation framework is developed for a class of manufacturing plants, where the environmental conditions would not affect the energy consumption of the production processes. Based on that, the building design problem for this type of manufacturing plants is formulated as a stochastic programming problem concerning uncertainties arising from the future weather conditions and energy prices, where seasonal production scheduling optimizing is incorporated when estimating the performance of building designs. Second, Ordinal Optimization (OO) method is introduced to solve the problem so as to quantitatively guarantee a high probability of finding satisfactory designs while reducing the computation burden. A numerical example is provided, showing our solution method performs effectively in finding a satisfactory design.