Prompt Evolutionary Design Optimization with Generative Shape and Vision-Language models

Rapid advancements in text-to-3D shape synthesis using generative AI models have achieved remarkable general-ization performance, generating novel designs based on com-binations of different concepts. However, designs synthesized through automated optimization often result in ill-defined shapes, which render such designs impractical for engineering design applications. We present the first end-to-end prompt evolution design optimization (PREDO) framework contextualized in a vehicle design scenario that leverages a vision-language model to penalize impractical car designs synthesized by a generative model. The backbone of our framework is an evolutionary strategy coupled with an optimization objective function that comprises a physics-based solver and a vision-language model for practical or functional guidance in the generated car designs. In the prompt evolution search, the optimizer iteratively generates a population of text prompts, which embed user specifications on the aerodynamic performance and visual preferences of the 3D car designs. Then, in addition to the computational fluid dynamics simulations, the pre-trained vision-language model is used to penalize impractical designs and, thus, foster the evolutionary algorithm to seek more viable designs. Our investigations on a car design optimization problem show a wide spread of potential car designs generated at the early phase of the search, which indicates a good diversity of designs in the initial populations, and an increase of over 20 % in the probability of generating practical designs compared to a baseline framework without using a vision-language model. Visual inspection of the designs against the performance results demonstrates prompt evolution as a very promising paradigm for finding novel designs with good optimization performance while providing ease of use in specifying design specifications and preferences via a natural language interface.