An Early History of Optimization Technology for Automated Design of Microwave Circuits

This paper outlines the early history of optimization technology for the design of microwave circuits—a personal journey filled with aspirations, academic contributions, and commercial innovations. Microwave engineers have evolved from being consumers of mathematical optimization algorithms to originators of exciting concepts and technologies that have spread far beyond the boundaries of microwaves. From the early days of simple direct search algorithms based on heuristic methods through gradient-based electromagnetic optimization to space mapping technology we arrive at today's surrogate methodologies. Our path finally connects to today's multi-physics, system-level, and measurement-based optimization challenges exploiting confined and feature-based surrogates, cognition-driven space mapping, Bayesian approaches, and more. Our story recognizes visionaries such as William J. Getsinger of the 1960s and Robert Pucel of the 1980s, and highlights a seminal decades-long collaboration with mathematician Kaj Madsen. We address not only academic contributions that provide proof of concept, but also indicate early formative milestones in the development of commercially competitive software specifically featuring optimization technology.

were already in progress in the 1960s [1], [4], mostly based on heuristic search methods with no derivative information [5].
The first gradient-based optimization methods, mostly quasi-Newton approaches, were rapidly adopted in the 1970s [4]. By that time, automatic optimization was already seen as the most significant advance in microwave CAD [6].
During the 1970s and 1980s, mathematician Kaj Madsen (see Fig. 3) and his research group at the Technical University of Denmark [7] developed powerful minimax and related mathematically rigorous algorithms, consolidating quasi-Newton gradient methods for circuit design optimization. Co-author Bandler was fortunate to establish a decadeslong relationship with Madsen and his team.
Nominal, worst-case, statistical, and multi-objective circuit optimization was already in place [8], [9]. Microwave statistical design and worst-case tolerance optimization also started in the 1970s [10]. Initial optimization cases in highdimensional design spaces were developed in the 1980s for satellite waveguide multiplexers [11]. The first gradient-based direct electromagnetic design optimization method, tailored to microwave filters, appeared in the early 1990s [12].
In 1993, recognizing the urgent need for effective electromagnetic optimization, while puzzling over the mystery of the so-called engineer's "feel" for a problem and the cognitive concept of "model," and after visiting Madsen in Denmark, Bandler invented space mapping [13], [14], [15], [16].
The last two decades have confirmed the prediction made in 2001 [30], that knowledge-based techniques would dominate in addressing current and future microwave design optimization challenges [31]. Today's thinking about current and future trends in microwave design automation are summarized in [32].
This paper briefly tracks the early history of optimization methods and techniques for RF and microwave design. It highlights the main milestones in several overlapping chronological stages of microwave design optimization: 1) circuit-oriented optimization methods; 2) direct electromagnetic optimization techniques; 3) space mapping optimization and surrogate approaches; 4) surrogate methodologies and Bayesian approaches; and 5) cognition-driven design.
We also highlight the impact of the MIMIC (Microwave/Millimeter-Wave Monolithic Integrated Circuits) Program and a vibrant decade-mid 1980s to mid 1990s-during which intense theoretical contributions and commercial software innovations became widely accepted by microwave engineers. Co-author Bandler's optimization-focused company, Optimization Systems Associates, founded in 1983, acquired by Hewlett-Packard in 1997 [33], flourished center stage during that formative period in the history of microwave optimization technology.

II. CIRCUIT-ORIENTED MICROWAVE OPTIMIZATION
Successful numerical optimization methods for microwave design were first applied to equivalent lumped and distributed circuit models [4]. Multidimensional direct search strategies relying exclusively on objective function values, such as pattern search [4], Nelder-Mead's simplex [34], and razor search [35], were initially adopted. These were soon replaced by gradient-based methods when accurate and computationally efficient derivatives became available.

A. GRADIENT-BASED OPTIMIZATION WITH ADJOINT SENSITIVITIES
The breakthrough adjoint sensitivity technique allows the calculation of accurate response derivatives with respect to design parameters at a low computational cost. Director and Rohrer's milestone [36] for lumped circuit adjoint sensitivities in 1969 was extended in the 1970s to distributed circuits [37], and to first-and second-order sensitivities for wave variables [38]. The adjoint sensitivity technique requires at most two full circuit simulations, one of the original circuit and one of an appropriately excited adjoint circuit, regardless of the number of design parameters.
CAD vendors paid little attention to adjoint sensitivities in the subsequent two decades, mainly due to complications of implementation and unclear market interest [5]. In contrast, response surface modeling and interpolation were easier to implement. Nevertheless, powerful circuit-oriented adjoint network gradient-based nonlinear optimization methods were already available in the 1980s [40]. Renewed interest in adjoint sensitivities emerged as CAD moved towards electromagnetics-oriented design optimization [41], as described in Section III.

B. LEAST PTH AND MINIMAX OBJECTIVES AND ALGORITHMS
Another breakthrough during the 1970s and 1980s was the emergence of effective least pth and minimax algorithms for design optimization [39]. The generalized least pth objective (H p ) that played a significant role in microwave design [40], [42], and later paved the way to design centering as well as to active device modeling and parameter extraction [5] is given by where J = {j | e j ≥ 0} defines the set of indices for which the response error functions violate the specifications. In other words, if at least one error e j is nonnegative (at least one spec violation), H + p is used; if all specs are satisfied, H − p is used. These optimization methodologies found application in commercial software in the 1980s, 1990s and beyond.

C. YIELD-DRIVEN MICROWAVE DESIGN
Pioneering work on yield-driven design-also called statistical design, design-centering, or design with tolerances and uncertainties-emerged in the 1970s [8], [43], [44], including uncertainties not only in models but also in parasitic effects and basic environmental conditions [45]. The fundamental concept is illustrated in Fig. 4 [40], where we see a parameter space of design variables with a region of acceptable designs, and statistically generated outcomes around two nominal designs. The aim is to "center" a design with its possible outcomes to maximize yield or minimize cost.
Throughout the 1980s and 1990s, industrially implementable optimization algorithms were developed for yielddriven design using one-sided L 1 [40], [46], one-sided Huber formulations [47], and a family of advanced L 1 , L 2 , and minimax optimizers, with both exact and approximate gradients [48]. L 2 (least squares or Euclidean norm, p = 2) is very sensitive to gross errors (outliers affect its performance significantly), while L 1 (L 1 -norm, 1-norm, or Manhattan norm, p = 1) is more robust against "wild" data but is biased by small errors [46]. Huber is a hybrid of these two [49], treating large errors in the L 1 sense and small errors in the L 2 sense. Given its properties to deal with bad starting points, robustness and consistency in the presence of large and small errors, Huber proved extremely effective for statistical device modeling, analog fault location, and design centering [47], [50].

D. MINIMAX OPTIMIZATION
Based on prior work by mathematician Kaj Madsen [7], [51], [52], [53] and his team at the Technical University of Denmark, minimax (equal-ripple) optimization of microwave equivalent circuits [54] was successfully applied during the 1980s to diverse design application cases.
Representative minimax design optimization examples include: impedance transformers, interdigital filters, and microwave reflection amplifiers [51]; waveguide manifold multiplexers with up to 240 optimization variables following an automatic decomposition algorithm (see Fig. 5 [11]); and five-channel multiplexers optimized by weighted updates for gradient approximations (see Fig. 6 [55]).  In essence, a (so-called one-sided) minimax formulation to design optimization consists of solving where x * is the optimal design and U is the objective function consisting of the maximum error in a set of errors. For each response of interest (S-parameters, output power, etc.) and independent variable sample (frequency, time, temperature, etc.), an error function is defined such that a negative value in the kth error function, e k (x), implies that the corresponding design specification is satisfied, otherwise it is violated. If U(x * ) < 0, the optimal design satisfies all the specifications. if U(x * ) ≥ 0, at least one of the design specifications is being violated at the optimal design found.

E. NONLINEAR CIRCUIT DESIGN OPTIMIZATION
The 1980s saw significant progress in nonlinear microwave circuit simulation, intermodulation, frequency conversion, stability, noise analysis, sensitivity analysis, and optimization, exploiting the harmonic balance technique [56]. A fruitful Bandler-Rizzoli scientific synergy [57] was established for developing CAD tools with nonlinear design automation capabilities [58], [59]. Linking state-of-the-art optimization and efficient harmonic balance simulation was key to developing powerful nonlinear microwave circuit design optimization approaches in the early 1990s. The formidable problem of yield-driven optimization of nonlinear microwave circuits operating in the steady state under large signal periodic excitations was tackled by unifying FAST (Feasible Adjoint Sensitivity Technique: exact adjoints combined with perturbations) and DC, small-signal, and large-signal simulations [60]. Yield-driven microwave active device modeling and circuit design was achieved [61] by integrating physics-based MESFET models with harmonic balance simulation and optimization. Commercial CAD tools were developed for robust FET model measurement-based parameter extraction exploiting L 1 , L 2 , and minimax optimizers [62], later generalized to a nonlinear device characterization by statistical modeling enhanced by FAST and Huber optimizers [63].

III. DIRECT ELECTROMAGNETIC MICROWAVE OPTIMIZATION
Putting full-wave electromagnetic simulators into optimization loops was infeasible-laughable to many-in the 1980s and early 1990s given the computational cost implied. It simply seemed absurd to electromagnetics simulation experts [5], both academic and industrially oriented. They could not envisage HP's HFSS [64] and Ansoft's Maxwell Eminence [65] as ever being suited to direct optimization. Their vision proved wrong. The first direct, gradient-based electromagnetic optimization process applied to microwave filter design was published in 1993 [12]. It exploited response surface modeling, smooth gradient estimation, and data-base updates to overcome the computational challenges [66].
OSA's methodology for automated capture of structural, geometrical and material parameters in "external" simulators was key to enable direct electromagnetic optimization. In 1992, after a cordial meeting with Sonnet founder and CAD visionary Jim Rautio (see Fig. 7), Sonnet Software becomes the first commercial electromagnetic simulator vendor to offer its flagship software em TM to OSA specifically for design optimization purposes. Developed in 1992, OSA's Empipe TM [67] was the first commercially available software for such automated parameterization through its exclusive Datapipe [68] and Geometry Capture [69] technologies. Empipe empowered industrial optimization algorithms [59] with parametric encapsulation and control of Sonnet Software's planar tool em TM [70] in iterative optimization loops (see Fig. 8). For the first time also it became possible, using OSA90/hope with its one-sided L 1 algorithm [40], [46], to perform direct gradient-based yield optimization of circuits with components or subcircuits simulated by an electromagnetic simulator [71].
A parameterization scheme of 3D structures specifically tailored to Hoefer's TLM electromagnetic field simulation and optimization was achieved in 1993 [72], [73].
Later, in 1996, after Hewlett-Packard and Ansoft finally granted OSA access to their flagship electromagnetic simulators, OSA released Empipe3D TM [74], [75]. Empipe3D enabled automated parametric design encapsulation of arbitrary 3D structures by processing the HFSS project files in HP's HFSS TM [76] and Ansoft Corporation's Maxwell Eminence TM [77], allowing direct 3D electromagnetic optimization [78]. Once a structure was captured, Empipe3D automatically generated the appropriate projects for HFSS, invoked the field solver and processed the solution files to display and optimize various responses.
A method for unrestricted shape optimization tailored to 3D FDTD EM simulation became available in 1998 [79].
Subsequent developments for direct electromagnetic optimization include gradient-based methods using adjoint sensitivities. Initial "exact" sensitivities based on analytical derivatives of relevant FEM system matrices to compute S-parameter sensitivities [80] were implemented in a commercial FEM solver [81], [82]. Specific adjoint-style approaches are now commercially implemented [83], [84]. Promising exact analytical expressions for field-based sensitivities, not specific to any particular electromagnetic analysis technique or meshing strategy, are now available [85].
A more recent strategy to accelerate direct electromagnetic optimization consists of internally modifying the analysis method (FEM) to incorporate the optimization algorithm as part of the simulation execution [86], such that both, the electromagnetic simulation and design optimization, are finished at the same time [87], [88].

IV. COMMERCIAL OPTIMIZATION TOOLS: THE DECADE OF ACCEPTANCE
Until the early 1980's, iterative computer-oriented methods for microwave circuit optimization were regarded as curiosities, not taken seriously by either influential circuit theory "synthesis" purists or "nuts-and-bolts" engineers [16]. Moreover, because the then optimization test examples were often simple, frequency-domain, linear circuits with just a handful of optimization variables, proposed algorithms were looked upon as impractical, lacking in mathematical rigor, and incapable of solving industrially meaningful design problems. However, the use of digital computers to analyze complex circuits, linear and nonlinear, and to validate structures through electromagnetic simulation was never questioned no matter how daunting. In essence, computers were typically thought of as ultrafast electronic calculators. The prospect that a computer could somehow "create" an optimal design by repeated analysis driven by a simplistic, seemingly trial-and-error algorithm must have been frightening and, in the electromagnetics realm, quite unthinkable. The decade from the mid 1980s to the mid 1990s was a major turning point in design optimization technologies for RF and microwave engineering.

A. THE IMPACT OF THE MIMIC PROGRAM ON COMMERCIAL MICROWAVE CAD TOOLS
A key catalyzer for the accelerated evolution of commercial EDA tools for microwave optimization, and their wide acceptance in industry, was the 7-year, $0.5-billion U.S. Department of Defense MIMIC (Microwave/Millimeter-Wave Monolithic Integrated Circuits) Program, initiated in 1986 [89]. To reach its goals, the MIMIC program aimed at "substantially improving computer-aided design models and tools" [89], providing resources and structure to make possible the transition from design automation research to efficient and affordable MIMIC manufacturable production in gallium arsenide technology for a wide variety of applications [90], both military and commercial [91].
Key players in the MIMIC program participating in the industry consortia focused on CAD problems included Compact Software (in association with OSA [16], [92]), EEsof, and Sonnet Software, headed by joint ventures between the major contractors: Raytheon and Texas Instruments, and ITT and Martin Marietta [90], [91].
The MIMIC Program detonated fast developments for wafer/chip yield prediction and optimization, including stateof-the-art device statistics, measurement uncertainties, worst case design, design centering, and tolerance assignment for linear and nonlinear circuits.

B. TIMELINE OF MAJOR COMMERCIAL MICROWAVE OPTIMIZATION-ORIENTED EVENTS: EARLY 1980'S TO LATE 1990'S
The following is a brief, personal timeline of major commercial advances focusing on optimization technology in the microwave arena. Contributors in important, related areas of design automation are omitted in this brief sketch that specifically emphasizes optimization (see Fig. 9).
The period from the early 1980s to the late 1990s is marked by setbacks, surmounting barriers of disbelief, triumphs, corporate acquisitions, and the movement of experts and expertise between vendors, all ultimately benefitting advances in RF and microwave design and manufacturing technology.
CAD visionary Les Besser comes onto the scene early, puts his experience into practice, founds Compact Software in 1973 [93], and his commercial circuit design offering COM-PACT features optimization algorithms.
The next generation of COMPACT (Computerized Optimization of Microwave Passive and Active CircuiTs) is named SuperCompact (1981).
Optimization Systems Associates Inc. (OSA) is founded (1983). Canadian satellite telecommunications company ComDev commissions OSA (1983) to introduce advanced optimization technology to its in-house flagship waveguide manifold multiplexer design portfolio.
Former Compact Software developer Bill Childs co-founds EEsof in 1983 and creates Touchstone (1983) to compete with Compact Software's SuperCompact. Touchstone runs from the outset on the PC.
OSA contributes advanced optimization capabilities to EEsof's Touchstone (1985). But EEsof rejects Bandler's pitch for yield-driven design as a follow-up to OSA's upgrade of Touchstone's optimization options based on EEsof's conviction that their customers would never ask for such capability. Of course, after OSA later delivers this capability to Compact Software, EEsof scrambles to follow suit.
The imminent MIMIC program motivates Bob Pucel of Raytheon Research Division in 1985 to champion yield-driven design and brings Bandler and OSA together with Compact Software.
OSA is commissioned to re-engineer SuperCompact and introduces advanced optimization and yield-driven design (1986)(1987)(1988), also entirely new documentation to Compact Software's premier products, and then re-builds Microwave Harmonica (1987)(1988).
Sonnet Software starts shipping em in 1989 [96]. Ansoft creates 3D electromagnetic simulator HFSS for HP; HP starts shipping HFSS in 1990; Ansoft markets its own competing product Maxwell Eminence in 1992.
HP and Ansoft finally make their 3D simulators available to OSA and OSA creates Empipe3D (1996) to drive HP HFSS and Ansoft's Maxwell Eminence. Some backstory: after Dan Swanson (see Fig. 10) spoke highly of the Empipe technology during a meeting, a key member of Ansoft vowed to have Maxwell Eminence on OSA's doorstep ("Within two weeks or I'll eat my business card").

C. OPTIMIZATION SYSTEMS ASSOCIATES
The longest in-person professional associations with Bandler are those by Steve Chen (see Fig. 11) and Radek Biernacki (see Fig. 10). They were key creative members of Optimization Systems Associates until HP's acquisition and OSA's products were absorbed into HP EEsof in 1997. Biernacki and Chen relocated to HP EEsof in 1997 [102].
When faced with OSA's collapse in relations with Compact Software, Raytheon, Texas Instruments, and the MIMIC Program in 1989, Chen and Biernacki vowed to continue working with Bandler and OSA "to the bitter end." Without their creativity, dedication, initiative and intellectual skillset, the optimization-centric products, RoMPE, HarPE, OSA90, OSA90/hope, Empipe, Empipe3D, EmpipeExpress and empath would surely never exist. And successful commercial electromagnetic optimization and the space mapping concept itself might at best have been delayed by several years. None of the RF/microwave software vendors nor any practitioners in electromagnetic simulation had any thoughts or vision for electromagnetic optimization until OSA produced results.
Why did HP acquire OSA? Importantly, OSA had successfully leveraged the premier electromagnetic simulators of HP's competitors in addition to its own HFSS. " 'Integrating OSA's technology into our electromagnetic product line will effect a paradigm shift in how these tools are used,' [Jake] Egbert said. 'Our electromagnetic tools will move from being analysis tools to design tools."' [101].
But there may be a deeper reason. OSA's customers appreciated the full range of OSA optimization offerings. By 1997, OSA90 [68] included state-of-the-art optimization algorithms, both gradient-based (minimax, L 1 , L 2 , Huber, quasi-Newton, conjugate-gradient, yield-driven) and direct search-based (simplex, random, simulated annealing). Indeed, at the IEEE International Microwave Symposium Exhibitions, Bandler recalls the annual excitement of "what's new?" from academics and practitioners alike. Expectations were high. Competition stiff. Microwave Engineering Europe proposed CAD challenges regularly that OSA and other vendors rose to and OSA performed commendably, for example [103], [104]. The class B frequency doubler example [104] later demonstrated Geometry Capture with simultaneous electromagnetic/harmonic balance optimization [105]. See Fig. 12 (Geometry Capture) and Fig. 13 (doubler circuit).
Bandler recalls a visit with British Telecom in the U.K. and asked why OSA hadn't heard from them since British Telecom acquired OSA software. Their response, "We compare OSA device models with those of HP." "What do you do when the models disagree?" "Of course, we complain to HP."

V. SPACE MAPPING OPTIMIZATION AND SURROGATE APPROACHES
In 1994, space mapping [13], [14], [15] surprised the RF and microwave community. Space mapping is deeply rooted in engineering expertise and intuition [16]. In fact, as published everyday examples of space mapping demonstrate [20], [106], [107], it turns out that space mapping, in particular, aggressive space mapping, manifests itself as a mathematical expression of the engineer's mysterious "feel" or expertise [16].

A. INITIAL REACTIONS TO SPACE MAPPING
What was the immediate reaction of Bill Childs, EEsof Vice President, in 1994 during a poster presentation at the International Microwave Symposium in San Diego [111] to Bandler's first-ever public presentation of space mapping? Notable disbelief at its deceptive simplicity. Even outrage. On the other hand, at the same conference, when presented verbally with the gist of space mapping, filter synthesis giant Ralph Levy (early skeptic in the 1960's and 1970's of the utility of optimization and/or design with tolerances) grasped the significance of space mapping right away. "Yes," he said. "I did that the other day when I needed HFSS just to calibrate a single point." His excitement was palpable. As had been Vittorio Rizzoli's when, prepublication, Bander outlined space mapping through hand waving gestures in Rizzoli's office in Bologna, Italy, months earlier [16], [112].
Right from the start Bandler saw space mapping in action everywhere. He recalls a seminar by Wolfgang Hoefer during which Bandler suggested that Hoefer's design procedure constituted an example of space mapping. Hoefer, horrified, said, "Everything isn't space mapping, John." Hoefer's technique was intuitive, iterative, and yielded good results in a handful of steps, the hallmark of space mapping.
Experienced filter designers, for example, often arrived at good design solutions seemingly without a formal, widely understood procedure. How did they do it? Fast but approximate circuit models existed. High-fidelity electromagnetic validations were possible. But how to align the two simulation processes so that a designer could exploit the speed of the circuit model while frugally harnessing the accuracy of the electromagnetic model?
Until Bandler pioneered space mapping in the context of electromagnetic optimization, the microwave community largely adopted algorithms from outside, from mathematicians such Fletcher and Powell [113] and later Kaj Madsen, for example, [54], [55]. (At a conference on optimization circa 1970, Fletcher remarked to Bandler, perhaps facetiously, that he never read anyone else's papers.) From 1993 onward we began to reverse the trend, delving deeper and deeper into space mapping approaches, for example, [14], [15], [20], [24].
Bandler had struggled for 30 years to understand the so-called engineer's "feel" for a design problem until he discovered that it was not as mysterious as claimed by practitioners. Space mapping embodies this "feel"-its simplicity and effectiveness shocked both the microwave engineering and mathematical optimization communities.
Such a simple mathematical technique covered such a wide range of design optimization problems. This led to Bandler's conviction that the idea had been in widespread use already. Indeed, those with "expert" knowledge, like Hoefer and Levy, knowingly or unknowingly, harness the space mapping concept in activities ranging from everyday human experiences to expert tuning and design of complex systems with electromagnetic accuracy.

B. THE ESSENCE OF SPACE MAPPING
To avoid performing the direct optimization of computationally expensive models (also known as fine models, or highfidelity models), space mapping iteratively enhances or maps a coarse model (idealized or low-fidelity model) to the accuracy of the corresponding fine model. In space mapping, this coarse model is typically a physics-based quasi-global model (see Fig. 14). Space mapping mimics an expert's intuition exploiting available, fast, parameterized, and physics-based simplified models. Plain space mapping can fail (intuition often fails); remedies are widely discussed in the literature [20], [30], [107], [114].
Aggressive space mapping (ASM) arose from Madsen's realization that optimizing a fine model using a corresponding coarse model as a valid quasi-global approximation is equivalent to solving a system of nonlinear equations, and nonlinear equations benefit from Broyden's method of updating the Jacobian [14], [15], [107]. In essence, the ASM algorithm iteratively finds a solution to the following system of nonlinear equations: where P(x f ) represents the mapping from the fine model design space x f to the coarse model design space x c , and x * c is the optimal coarse model design. Any root of the above system of equations, x SM f , also known as a space mapped solution, implies that the fine model response R f (x SM f ) sufficiently matches the coarse model optimal response R c (x * c ), which is used as the target response [107]. The potential multiplicity in space-mapped solutions [107] as well as in coarse model local alignments is addressed in [20], [115], [116].

C. SPACE MAPPING GETS EXPANDED
Not surprisingly, in the early years following his announcement of space mapping, Bandler was frequently asked why his group seemed to be the only group applying space mapping. That trend was soon to revert by an explosion of diverse users with space mapping variations going well beyond the initial aggressive space mapping formulation.
Another key event that contributed to the wide spread of space mapping was the workshop entitled "Microwave Component Design Using Space Mapping Methodologies," organized by Bandler as part of the 2002 IEEE MTT-S International Microwave Symposium (IMS), in Seattle, WA. Vicente Boria, from the Polytechnical University of Valencia, Spain, an attendee of that workshop, immediately found innovative applications of space mapping to waveguide filter design [136], [137], [138], established a fruitful collaboration with Rayas-Sánchez [139], including reciprocal visits in 2006 in Guadalajara, Mexico (see Fig. 15), and Valencia, Spain,  and later contributed to an even wider dissemination of space mapping [140], [141], [142], [143], [144], [145], [146].
Synthesis or inverse modeling space mapping approaches have also been successfully employed for RF and microwave design. Here, an inverse mapping (from the coarse model to the fine model design space) is developed, either by a neural [168], [169] or linear [170], [171], [172], [173], [174] mapping, allowing the prediction of the next iterate by simply evaluating the current inverse mapping at the optimal coarse model design.
Even though space mapping emerged as an efficient solution for optimizing high-fidelity models, it was soon extended to develop space-mapped models for accurate and inexpensive statistical analysis and yield optimization [28,] [171], [175], [176], [178], [179], [180], [181]. Fig. 17 shows a photo of some of the main contributors to the utilization and expansion of space mapping.

D. SPACE MAPPING IS APPLIED BEYOND RF AND MICROWAVES
Although it started in the microwave engineering arena, space mapping optimization has found applications in a plethora of diverse disciplines. For instance, simple linear input as well as aggressive space mapping have been applied in areas such as materials design, environmental sciences, medical instrumentation, magnetic circuits, electric motors, chemical, civil, mechanical, biomedical, aerodynamic, aeronautical, and aerospace engineering, for example, [107], [108], [109], [110].

F. SPACE MAPPING MEETS SURROGATE MODELING
A key historical meeting organized by Madsen and Bandler brought together the mathematical optimization community with the engineering community working on microwave design optimization. This was the first International Workshop on Surrogate Modeling and Space Mapping for Engineering Optimization (SMSMEO), celebrated in 2000 at the Technical University of Denmark, in Lyngby, outside Copenhagen [198]. The second and third edition of this workshop took place in 2006 in Lyngby, Denmark [199], and 2012 in Reykjavik, Iceland [200], respectively. These events established a fruitful dialog bridge between the mathematics and engineering communities interested in optimizing computationally expensive functions.
Indeed, the first SMSMEO (see Fig. 19) in 2000 brought Bandler's and Madsen's groups (space mapping) into discussion with that of mathematician John Dennis of Rice University (surrogate modeling). Notably, Madsen initially hesitated in using "space mapping" as part of the name of the International Workshop on Surrogate Modelling and Space Mapping. He anticipated controversy, and he was right. The debate in Lyngby proved lively. The bottom line is that mathematicians then used Taylor-based (linear and/or quadratic) nonphysical localized coarse models as their underlying "surrogates" while space-mapping-oriented engineers underpinned their algorithms with quasi-global physics-based models based on engineering knowledge.
Direct electromagnetic optimization algorithms using abstract mathematical models is dramatically at odds with what a skilled engineer can achieve via an experience-tested "feel" for the problem. In hindsight, the effectiveness of space mapping utilizing relevant physics-based surrogates should not have been surprising.
Bayesian approaches [241] are suitable for dealing with computationally expensive models, especially those with noise-corrupted responses, including high-fidelity models based on physical measurements subject to statistical uncertainty and varying operating or environmental conditions [31].

VII. COGNITION-DRIVEN DESIGN
There are clear parallels between space mapping and human cognition (the engineer's traditional but mysterious "feel" for a problem): (1) the simple, intuitive examples of space mapping like the cheese-cutting problem [20]; (2) the popularity of aggressive space mapping as shown by Rayas-Sánchez, (3) the parallels with space mapping found in Kahneman [251]: his System 1, fast and intuitive, and his System 2 that is slow and effortful; (4) advances in space-mapping-based design using cognition-style markers like response features [233], [252].
Bandler created several simple illustrations of the space mapping concept in action [20]. They started with the socalled "cheese-cutting" illustration, then came the "wedge problem," "cake-cutting," "shoe-selection," [16] "parachute landing game" [5], [253], and other illustrations or games. These manifestations of aggressive space mapping date back to Bandler's shock the night before he was scheduled to address mathematician and collaborator Kaj Madsen's undergraduate optimization class at the Technical University of Denmark. Madsen abruptly questioned his own understanding of space mapping and indeed the concept of "model" as understood by engineers. Bandler scrambled overnight to find a common sense illustration suitable for engineer and mathematician alike and came up with the cheese-cutting illustration in time for Madsen's class the next day.
Similarly, Rayas-Sánchez was confused when Bandler presented him privately with the "parachute landing game" [253] during the IMS-2015 in Phoenix. What did this onedimensional task have to do with microwave design optimization? It was not until Rayas-Sánchez tried out the game on Bandler's laptop that he identified the relationship (calibration) between the inherent "fine model" and his own evolving mental "coarse model" of the game. There appeared to be three overlapping phases to the task required, (1) a familiarization phase (building knowledge and confirming a mental model), (2) a refinement phase (learning to trust the mental model while replaying the game on each reset), and (3) a mapping update phase (realizing that a "best" guess at the next iterate requires information from more than the immediately preceding iterate).
Bandler writes [16], "Aggressive space mapping efficiently invokes inner loops of conventional optimization-common sense at work-often yielding excellent results in a few iterations. The aggressive space mapping update/execution process is itself optimization on a higher level-meta-optimization?a process that uncannily mimics both common sense and the expert's 'feel'." Needless to say, Madsen was shocked, as many people within and outside our community still are, that space mapping can be reduced to common sense, like not too hot, not too cold, just right.
If we consider cognition-driven design methodologiesalong with machine learning and artificial intelligence-in terms of common sense, human experience, space mapping, etc., we might look for new avenues of mutual collaboration and dialogue with psychologists, neuroscientists, game developers, linguists, playwrights, etc., and work with human subjects and explore the emotional element. See, for example, [254].

VIII. CONCLUSION
The year 2023 marks 30 elapsed years since 1993 that engineering optimization, particularly microwave electromagnetic optimization, has exploited space mapping technology and related surrogate methodologies. It took 30 years of research in microwaves, computer-aided design, and optimization of circuits and systems for Bandler to reach that half-way milestone, 30 years during which the engineer's mysterious feel proved elusive, decades populated by visionaries and champions of numerical and iterative techniques to design optimization, and some inevitable skeptics, industrial and academic, many of whom would eventually be won over.
We touched on circuit-oriented optimization, adjoint sensitivities, yield-driven design, nonlinear circuit design, and gradient-based optimization algorithms such as Madsen's minimax and related formulations that have stood the test of time. We emphasized the origins and subsequent evolution of microwave electromagnetic optimization.
We devoted much attention to what we believe is a decade of acceptance, final acceptance by academia and industry, of iterative numerical optimization tools and corresponding commercial software offerings. We provided a timeline of major commercial optimization-oriented solutions and events, and noted the impact of the MIMIC Program.
We outlined the backstory to space mapping: reactions to it by the scientific and engineering communities, some success stories, some failures, and its ongoing reinvention as "cognition-driven design." We suggested the essence of space mapping [255], the expansion of space mapping into variations such as implicit and tuning space mapping, applications beyond RF and microwaves, measurement-based space mapping, and a synergistic period when engineers and mathematicians met to discuss space mapping vis-à-vis surrogate modeling. Finally, we touched on ever expanding surrogate methodologies.
A brief section sketched what Bandler has termed "cognition-driven design." We indicated its roots in space mapping, named simple illustrations and suggested a link of the concept to common sense.
We indicate but do not elaborate in our paper on the current state of the art of advanced surrogate technologies and their myriad applications, nor elaborate on our vision of the future.
Such details, we believe, are dealt with by Rayas-Sánchez et al. [31]. Instead, we provided a selected backstory and the likely early cornerstones of optimization technology in microwaves and RF.
In Bandler's words [5]: "Finally, the engineer's mysterious "feel" will most likely be automated, with accurate predictions of successes rather than explanations of failures being the ultimate goal." Returning to the first paragraph in our introduction: "Fully automated design and optimization is surely one of the ultimate goals of computer-aided design." After more than five decades, it remains one of our ultimate goals: we still have much to do! ACKNOWLEDGMENT The authors thank Melissa Skudra for her skillful assistance in researching, scanning, and archiving reports, manuals, and other documents.
Countless colleagues have contributed to our innovations in design automation in microwave engineering and electromagnetic optimization over the years or have championed our technical work. The additional contributions and support of Greg Atkinson, Beth Bandler, Stephen Elop, Farzad Foroutan, Cheryl Gies, Denys Shumakov, and Peter Thoma, are gratefully acknowledged.