On the Cybernetics of Crowdsourcing Innovation: A Process Model

For small and medium-sized enterprises and large enterprises alike, crowdsourcing innovation has become an important element of a product’s whole life cycle. It is the open call process of soliciting consumers to harvest and evaluate ideas or other intellectual assets. The previous proposed taxonomic framework for charactering this process is mainly for general crowdsourcing process and summarized by empirical study. The purpose of this paper is to propose a conceptual model for crowdsourcing innovation from a cybernetic and knowledge management perspective by normative research. The authors performed a normative study and deduced five systemic characteristics from the general laws of control system that guarantee ongoing efficiency for the innovation process. The normative research results provide two key contributions. Firstly, general control laws deduce five indispensable characteristics, and they reveal the intrinsic mechanism of crowdsourcing innovation: the knowledge flow controls, which are also the connotation of open innovation. Secondly, the authors have analyzed a five-characteristics system model and tested the model in several classical cases to show the design tricks of cases. This study provides a new conceptual framework that integrates the theory of open innovation and cybernetics to provide a new view of crowdsourcing innovation process design. In practice, this framework guides managers through the design criteria needed to implement a success crowdsourcing process.


I. INTRODUCTION
Over the last decade, innovation models have entered the new era of open-source intelligence. Large corporations may have been first to adopt the strategy of open innovation for attracting talent, harnessing ideas, and identifying customer requirements, but small-to-medium-sized corporations (SMEs) have been quick to get on board [1]- [3]. This is because what companies of all sizes are finding is that open innovation helps to maintain market share; it increases profitability; and it contributes to sustainability. In developed countries, like the United States, governments are building innovation ecosystems to not only guide major problem-solving efforts but also enlist external thinking in their creation processes and promote collaborative solutions to public management [4]. Initiatives like challenge.gov [5] The associate editor coordinating the review of this manuscript and approving it for publication was Laura Celentano . are unpacking traditional closed innovation processes with new open schemes that try to synchronize internal problems with external solvers. In large part, the result has been better solutions and a more efficient and more effective innovation system [6].
Open innovation can take several different modes, such as outsourcing, opensource, and crowdsourcing [7]. Among the most common styles, crowdsourcing innovation is to leverage knowledge of highly skilled groups of users to address innovation challenges concerning technologies, products, and services by an open call on a web-based platform. It is a collaborative process that involves the party (organization, institute, corporation, etc.) that posts the challenge, also known as innovation seeker, and the party (expert, engineer, student, etc.) that contributes possible solutions, also known as innovation solver [1], [7], [8], [50]. Thanks to Web 2.0 technology, innovating crowdsourcing is usually done online [11]. The internet is a cheap and VOLUME 10, 2022 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ expansive means for seekers to get in touch with solvers and to receive solutions. In this online process, seekers and solvers collaborate at problem level, while solvers collaborate with other solvers at solution level. Commonly there are three types of crowdsourcing innovation platforms: the first one is performed as an intermediary or two-sided platform or innovation marketplace that there are seekers on one side and solvers on the other, such as innocentive.com, Ideaconnection.com, openideo.com and 99designs.com; the second one is offered by a single company for their exclusive ideas, concepts, solutions solicitation, such as challenge.gov and HOPE.haier.com; The third one is used to stay in contact with innovation community such as Dell IdeaStorm and BMW community.
Although crowdsourcing innovation has its origin in research on co-creation [50], its final goal is to organize people to perform problem solving practice [29]. Over the past 20 years, numerous cases and platforms emerge, many of them have had great success in achieving extremely promising results [35], [44], [52]. However, crowdsourcing studies adopted many perspectives in different application scenario, which indicates the lack of a global and comprehensive view [50]. To better perform crowdsourcing activities, the underlying theory and the characteristics of crowdsourcing should be identified in conceptual level. Though some research tried to normalize the crowdsourcing dimensions [11], [22], [23], [50], it mostly is based on the empirical studies without deducing from a fundamental theory. Hence, a formal framework for crowdsourcing innovation should be established to bridge the gap between theory and practice, it will be a valuable guidance for innovation seekers.
Any innovation seeker that aims to adopt crowdsourcing as an open innovation way is necessary to consider the framework of the crowdsourcing process that will be used for the design of their crowdsourcing implementation [50]. Existing frameworks descript by characteristics that focus on the micro-task do not reveal the open innovation mechanisms that control inflows and outflows of knowledge to improve innovation efficiency. Our model is in the vein. It is a novel cybernetic framework that emphasizes feedback and control. Unlike a normal process model where the components and characteristics of the model are typically based on taxonomy, our framework is knowledge oriented. By its very nature, open innovation is a coordination process. It is, or in our opinion should be, a well-tuned system of transferring and combining knowledge across the boundaries of organizations [18], disciplines, and nations to generate new knowledge. From this perspective, appropriate methods of controlling knowledge and knowledge flows become a fundamental design goal when devising initiatives to crowdsource innovation. For instance, a contest, that open call design problem and receive solutions from contributor, is a process that control the problem to solver, and feedback the solution to seeker by incentives such as monetary reword, societal identity etc.
To this end, the principal contribution of the paper is to reveal the control principles behind the crowdsourcing innovation. Such insights not only provide a systematic of understanding framework, but they also provide some criteria to guide the evaluation and design process. Further, A conceptual process model is proposed to try to generalize all the cases of crowdsourcing innovation process, it also can be seen as an initial point of crowdsourcing innovation design methodology.
Following a brief literature review in the next section, we present our conceptual model. We then present five cases that illustrate how the model works in theory and in practice along with a discussion on its advantages and disadvantages. The paper concludes with a summary of the article content, the theoretic and practical implications of the constructs presented, limitations, and our intended future work on this nascent field of study.

II. REVIEW OF LITERATURE A. CROWDSOURCING IN OPEN INNOVATION
Last two decades, numerous organizations to have begun to engage in crowdsourcing to solve innovation problems. it has been successful in many areas such as data science, ideas brainstorming, service improvement, scientific research, products design, and graphic design [19], [33], [45], [52]. The term crowdsourcing was first coined in a 2006 issue of wired magazine [8]. With it, the writer intended to encapsulate the new frontier of cyber practices designed to harness the power of the people -from Amazon Mechanical Turk to the open-source communities developing Linux and Wikipedia to projects like SETI@home. Originally, crowdsourcing offered a new model of sourcing labor, with private enterprise, public institutions, and governments all outsourcing functions once performed by employees to an amorphous network of unknowns somewhere in the ether. However, while part of appeal of crowdsourcing might be rooted in the access it gives to cheap labor, some of the main motivating ideas behind crowdsourcing can be explained through theories like the ''wisdom of the crowd'' or ''collective intelligence'' [9]. These drivers derive from the notion that fully exploiting the different perspectives of a range of people can lead to better solutions to a given problem. For example, Woolley et al. [10] highlights that collective intelligence plays an important role in a range of problems, like moral judgments, brainstorming, visual puzzles, and more.
Leimeister [9] identifies five factors that determine whether collective intelligence will succeed at systematic solving a problem. These are: 1) Control: Opening up processes to collective intelligence leads to a loss of control and unpredictable outcomes. 2) Diversity & Depth: Diverse expertise controls the novelty of the solutions while in-depth expertise ensures the solutions are feasible. Therefore, both are required.
3) Engagement: People need motivation to participate. Such incentives can be monetary, such as cash and prizes, or non-monetary, such as altruism, selffulfillment, and reputational gain. 4) Policing: Misconduct and malicious behavior needs to be prevented through. 5) Intellectual property (IP): There need to be clear guidelines for how to define and divide the IP held within the ideas and solutions generated between the seekers and the solvers. Doan et al. [11] discusses similar factors as key challenges for crowdsourcing systems. They define two types of systems -explicit and implicit -with a list of considerations that includes: how to recruit and retain participants; what the crowd can do; how to combine their inputs; and how to evaluate them. Implicit systems functionalize or gamify the process of crowdsourcing -examples include Google's search engine and FoldIt. Explicit systems make sharing and evaluation functions in themselves, such as YouTube and Wikipedia. Estellés-Arolas and Gonzalez-Ladron-De-Guevara [12] are among the few to have proposed an exhaustive and cohesive definition of crowdsourcing. Derived from extensive research and a long history of practice in industry, the formulation combines definitions for eight characteristics found in (all) crowdsourcing initiatives being: the crowd, goals, compensation, the crowdsourcer, solutions, processes, type of call, medium. Although his definition does not accord with the associated concept of open innovation, crowdsourcing is definitely its most conduits [12].
A last and most notable article in this review concerns the generalization of crowdsourcing to the field of human computation [13], i.e., ''distributed systems that combine the strengths of humans and computers to accomplish tasks that neither can do alone'' [14]. In this piece, Michelucci and Dickinson [13] discuss integrating humans and machines in distributed problem-solving systems to tackle the world's wicked problems. They identify three levels of such architecture -micro-tasking, workflows, and the problem-solving ecosystem -and go on to explore the kinds of problems human computation might address -from data labeling to data generation, from knowledge sharing to knowledge creation.
The motivation for our efforts to conceptualize innovation crowdsourcing comes from Malone et al. [15] who conjecture that innovation problems can be naturally transformed into workflows or problem-solving ecosystems via systematic analysis.
To improve innovation efficiency, corporations today are increasingly turning toward unknown parties instead of simply collaborating with known experts [16]- [18]. For example, contests are becoming more commonly embraced as natural part of a company's R&D process when staff do not hold the relevant knowledge or a massive, but temporary, injection of intellectual resources is required [18], [19]. For instance, requirement identification or concept generation process is performed through idea generation contest for improving the maturity products from the external lead user and designer [19]. The same is to technical problem from industry, such as challenges in InnoCentive or Challenge.gov. The process is relatively standard. First, an open call for solutions to a challenge is released on the web. Would-be solvers then submit solutions, which are evaluated by the seeker, and usually a prize or some kind of reward is issued to the winner. In short, crowdsourcing innovation is to narrow the scope of the term crowdsourcing and better specify the goal of open innovation.

B. PROCESS TYPOLOGIES BASED IN THE CROWD
In 2010, Piller et al. [20] not only proposed a threedimensioned typology for customer co-creation, they also proffered ''customer co-creation'' and their typology as an open innovation strategy in which both customers and firms take on different and unconventional roles. Their framework asks firms to recognize the innovation process as an interactive relationship between producers and users. Through these interactions, customers provide information about their requirements and producers use that information to improve their products.
The first dimension of the framework is the stage at which customers are invited to become involved: the front end or the back end. The front end is the idea or conception phase, while the back end includes the design and/or testing phases. The next two dimensions are both continuums: one being the degree of collaboration from a single customer to an entire community; the other being the degree of freedom from a completely open task to a highly specified, very narrow task. These three dimensions lead to eight ideal types of cocreation, as depicted in Fig. 1. Each of these schemes has been practiced by firms as an open innovation method with valuable outcomes derived as a result [19]- [21].
Another notable typology, outlined in Fig. 2 was proposed by Geiger [22]. This model characterizes crowdsourcing processes for innovation in terms of four components: preselection, accessibility, aggregation, and remuneration. In preselection, two strategies for assembling a community of problem solvers are discussed: one is maximizing the size of the community by skipping any pre-selection or pre-filtering process, the other is to do the opposite thereby amassing a collection of participants with desirable skills, backgrounds, etc. Accessibility pertains to the crowd's contributions and the extent to which they can be viewed, commented on, or modified. In most contests, for example, a contributor's solution is not visible to the other participants, whereas, in some idea generation cases, crowds can view, rate, comment, and even modify the contributions of others. Aggregating the crowds' contributions can be either narrative or integrative, where the contributions are combined incrementally to form a collective solution, or selective, where the contributions are selected for inclusion according to some predefined metrics. The last element, remuneration, if given at all, can be fixed or successbased. In most innovation-driven scenarios, rewards are fixed with a bonus for success. However, what this typology does VOLUME 10, 2022  not mention is the evaluation process, nor how the goals of the innovation exercise are defined or communicated. From a design perspective, the process of crowdsourcing innovation should be problem-oriented and be a configuration to crowdsourcing system.
The most systematized analytical model, outlined in Fig. 3, was deduced from Input-Process-Output framework [50]. This model attempts to frame all the many multi-faceted contributions on crowdsourcing in Input-Process-Output (I-P-O) model which is a general process framework for studies in the field of management [51]. The advantage of this model is to help distinguish the main antecedents, components, and outcomes of the process under scrutiny. In crowdsourcing scenario, the model input is the innovation problem/task outflowing from the inbound of corporation or institute. Through operating on process by web technology, which includes session management, people management and knowledge management, the model outputs an innovation solution. On perspective of the characteristics in this framework, it nearly covers all the features proposed in previous literatures. However, no obvious logical relation of components is revealed in this process model, especially it separates people and knowledge management, in which people can be considered as carrier of knowledge for crowdsourcing scenario.
All characteristics of crowdsourcing in the references are listed in Appendix, Table 12. After reviewing the literatures, two hypotheses are made from investigating the crowdsourcing approaches: Hypotheses 1: in crowdsourcing innovation, the crowd plays two roles: one is the carrier of the knowledge; the other is the generator of the knowledge. The foundation of collaboration is to view others' knowledge contribution.
Hypotheses 2: crowdsourcing approach has a goal that the participants would try to achieve it by incentive.
The two hypotheses indicate that the nature of crowdsourcing innovation is the flow of knowledges (contribution). The solvers are informed by open call from the seeker and exert effects by incentive.

C. THE CHALLENGES WITH CROWDSOURCING INNOVATION CHARACTERIZATION
Although innovation crowdsourcing process characterization in [17], [20], [22], [50] have been used in analytical and operative perspective. The characteristics included in these typologies/architectures are all inductive from practices and process inherent properties, like incentives and task process [20], [22], [23]. Yet the cognitive dimensions of innovating are ignored, and lack of theoretical analysis framework. Thus, we propose two novel aspects for deducing the characteristics of crowdsourcing innovation process, they are: -Open innovation and knowledge management.
-The general principles of control systems. Crowdsourcing, as a key approach of inbound open innovation practice, should serve open innovation process [50]. Inbound open innovation, as an application scenario for crowdsourcing, provides the innovation pipeline to be opened up. Knowledge management defines the input, output and information flow of innovation crowdsourcing. General control principles provide a structural analysis for crowdsourcing process; it also brings a novel view to evaluate the process in conceptual level. In general, open innovation provides the business function to crowdsourcing innovation, knowledge management is the core connotation of crowdsourcing innovation, and control principle characterizes the structure of crowdsourcing process system.

III. A SYSTEMS MODEL FOR INNOVATION CROWDSOURCING A. A CYBERNETICS PERSPECTIVE ON CROWDSOURCING INNOVATION
Borrowing the term cybernetics from Plato, Noebert Wiener [25], [26], in 1961, was one among several that established this discipline of examining the phenomenon of circular causality, i.e., where the outcome of an action becomes input for the next action, or feedback as he dubbed it. The goal of cybernetics is to develop a language and various technologies that can truly solve the general problems of control and communications. The solving of general problem brings more general concepts which are independent from specific domains and guide the specific domain [27]. While these concepts have a quantitative component, modeled by information theory and feedback control, the primary idea is to form qualitative analysis tools -a philosophy of technology. According to W. R. Ashby [28], cybernetics tends to universalize the notion of feedback, generalizing it as the underlying principle of the technological world.
From a cybernetics perspective, crowdsourcing innovation is a distributed innovation/problem-solving system [13], [29], [30]. As shown in Fig 4, the control subject is the innovation seeker. This seeker makes an open call for solvers to help meet their innovation requirements. The crowd of solvers, i.e., the control object, produces solutions and submit them for evaluation to the seeker as feedback. Thus, crowdsourcing innovation can be seen as a cybernetic system following a cybernetic philosophy.
Through half a century of development, cybernetics has become the source of many generalized laws, regularities, and principles for the functioning and control of complex systems. Of these, five are highly pertinent to the nature of crowdsourcing innovation. As summarized by Novikov [31], these are: 1) Thelaw of goal-directedness: Any control has a goal. The general goal with a system to crowdsource innovation is to harvest innovative solutions from the crowd. Any specific goal depends on the requirements of the situation. Table 1 lists several example cases.
2) The law of requisite variety: The system or person with the most flexibility of behavior will control the system. This law, proposed by Ashby [28], is also sometimes referred to as the first law of cybernetics. Hence, to maintain adequate control over a system, the controller needs to prepare for every possible situation the controlled object might create. This notion highlights the central position risk management must play in any crowdsourcing innovation process. Moreover, in their three-phase risk management methodology, Souza et al. [32] note that the two most likely risks are intellectual property breaches, created by disseminating core technologies or processes, and a lack of motivation in people to participate.
3) The law of emergence (synergy): or Aristotle's the whole is greater than the sum of its parts. From systems theory, this law as it pertains to crowdsourcing innovation suggests that crowds can rise to better performance as a collective than that derived from many individuals. In other words, the wisdom of the crowd and collective intelligence has some scientific foundation. In some cases, a simple set of rules over interaction can make better solutions emerge from a crowd than from individuals. Platforms like 99designs and CROWDSPRING adopt this approach, making all contributions are visible to any potential contributor. Some great results have emerged at low cost as a consequence. Interaction and collaboration are also the purpose of synergy, which leads to increased orderliness and reduced system entropy. In crowdsourcing systems, orderliness tends to originate from incentive mechanisms. 4) The law of feedback: A system can be controlled by reinserting into it the results of its past performance.
Here, Weiner is referring to cause and effect. In an electrical system, this law translates to the control subject monitoring the consequences of an action to control and using that feedback to adjust the controls.
In crowdsourcing innovation, the seeker's feedback is the solutions by the solvers and how the seeker adopts, adapts, comments, or scores the solution is feedback to the solvers. Thus, we can add to our insights that communication between the seeker and solver is both two-way and indispensable, especially the seeker's feedback to the solvers, which guides them toward the most desirable solution. Z. Jiang et al. [24] validated the impact of performance feedback on the outcome of crowdsourcing contests. As such, Schenk and Guittard [23] highlight that designing crowdsourcing innovation schemes with multiple rounds of feedbacks, like PRESANS, is a logical strategy. PRESANS is a multistep dynamic expert sourcing process. It involves three stages of filtering, each at a different depth of solution to arrive at the optimal selection. 5) The law of optimality: A control action is ''best'' when it works to attain a goal given the current constraints. This law implies that a criterion or set of criteria to determine the best alternative is a wise strategy, both in theory and in practice. InnoCentive [33], it concludes that the criteria are from articulating an innovation problem, thus the judging criterion is defined with formulation of innovation goal, and solver must have a way to get the evaluation of their submission for optimal solution. Table 2 lists six factors deduced from these five general laws of control that should be considered in a framework for crowdsourcing innovation. Some are already in practice most of the time, such as IP management, innovation goal and incentive mechanism. However, others seldom receive attention, such as judging criteria and multiple feedback rounds. Hence, as a more comprehensive, and hopefully more successful, method of designing schemes for crowdsourcing innovation, our conceptual model tries to incorporate these factors.
Among six factors, multiple feedback rounds implied a non-linear innovation process; it emphasizes the importance of feedback when innovation seeker implements the crowdsourcing innovation process. Interactions and collaborations that no matter in seeker and solver or solver and solver is accompanied by knowledge flow among them, from knowledge perspective, it suggests the essentiality of knowledge schema which is a framework and should be elaborated by crowdsourcing innovation seeker. Judging criteria should be considered in dimension of innovation goal, it is the metric to evaluate the solution how well for achieving the innovation goal. In summary, the five dimensions are consisted of conceptual model of crowdsourcing innovation; they are innovation goal, innovation process, knowledge schema, incentive mechanisms and risk management. From design respective, they are sequential dependent, innovation goal is the starting point, and innovation process is configurated up to innovation goal. Knowledge schema is a framework of knowledge flow in the process. The dynamics of knowledge flow are from incentive mechanisms. Risk management would monitor and keep the success of the whole crowdsourcing process.

B. A CONCEPTUAL MODEL FOR CROWDSOURCING INNOVATION
Fig. 5 sets out the five conceptual dimensions of the crowdsourcing process, based on the five relevant laws of cybernetics, as a series of considerations for the design of an innovation crowdsourcing initiative. These can even be viewed as steps in the design process. First, the seeker must analyze the problem to form an innovation goal for their opening call. Next, decide on a process for selecting and guiding the optimal participants to provide desirable feedback [23], [24]. Preparing a knowledge schema means to define the knowledge flows between the call for help to receiving the solution through to commenting and scoring the solution and deciding the rules for interaction and collaboration. Note that knowledge schemas need to be defined for the knowledge flows between both seeker and solver and solver and solver.
Step 4 is devise the incentive mechanisms needed to drive potential solvers to participate and, last, is to consider appropriate strategies for managing risk. This sequence is in itself cybernetic in that the outputs of the last step, to some extent, inform the decisions made in the next step. For instance, the selection processes should be designed to suit the specific innovation goals, while any knowledge sharing must be able to operate within the guidelines set for devising solutions, and so on. The exact shape of the final crowdsourcing process will always hinge on the specific applications, constraints, and objectives of the problem to be solved. Each of these steps is covered in more detail in the following.

1) THE INNOVATION GOAL
Step 1, the innovation goal, must articulately formulate the objective of the innovation activity to the crowd. Thus, the internal manifestation of the problem and how that is described externally become the key content for describing the innovation goal. Prior literature has highlighted that describing a problem is, in fact, a process of constructing a shared problem space. In our case, this shared space is between the seeker and the solver. Thalemann & Strube [34] discuss the three parts of a shared problem: the initial situation, the goal(s) to solve, and the operators necessary to collaborate. In the case of an open innovation challenge, the goal and initial situation are commonly defined via a functional analysis, and the operator path from the initial situation to the goal form the solution. In practice, the innovation goal often derives from a product's whole lifecycle, and the challenges arise out of either R&D activities or market demand [18], [6]. At InnoCentive [35], for example, each innovation goal is accompanied by prompts for five types of challenges from idea generation to full prototyping, as Table 3 present. Hence, the potential challenges can span the full gamut of an organization's requirements.
Open calls commonly include a title, overview, background, prize, timeline, milestone, deliverables, and judging criteria. Therefore, while the conceptual model calls for goals to be established as a first step, the final draft of the call for innovation cannot be written until the process is complete. Following the shared problem space model, the overview would set out the innovation goal. The initial situation and constraints on the operators would be described in the background, and the judging criteria would explain how the proffered solution is to be measured against achieving the goal. The remaining items on the list are relatively straightforward and common to most proposals.
For interested readers, Dewey [36] details a general method and template for writing an open innovation challenge that includes tips for how to write titles, problem definitions, and backgrounds. He also suggests illustrating the call with Further, the true challenge with writing a call for many is how to describe the problem in such a way that others can see what truly needs to be solved. It is just like writing a query to problem solver for searching a solution. To this end, Sloane and Paul [6] discuss six rules for helping seekers write problem descriptions to ensure they are both as open as possible and as precise as necessary. Without going into the detail of all six rules here, the core idea is to step back to the initial scenario and describe the challenges that depend on the achieving the goal. Importantly, any previous attempts toward solving the problem, both successful and unsuccessful should be included in the description. This avoids wasting time vetting approaches that have already proven to fail.
Lastly, although the seeker determines the judging criteria, this point can be left open for the solver. After all, it is only logical that the person who crafted the solution should also have ideas about how to normalize the solution, guide its implementation, and/or generalize it to other applications. Some situations it is not revealed by the seeker with a purpose of diversifying the solution space. As a case in point, challenge.gov provides many examples of different types of judging criteria; on InnoCentive, judging criteria are seldom included in the call; and, on Threadless.com, contributions are openly evaluated by the crowd. So here, solvers not only create and submit the designs, they also determine the winner.

2) PROCESSES
The crowdsourcing process does not necessarily have to be one linear procedure. Rather, it can, and often is, a series of processes that working concurrently and/or in partitions that govern every aspect of the scheme -from who gets to participate to how submissions are lodged to how they are evaluated and taken up. Some processes are selective in that they only pertain to one aspect of the project, such as how the project is funded, or clearly specifying the problem. Another important set of processes governs whether and how the contributions are aggregated. Many innovation processes can be opened up and crowdsourced in selective or aggregated way, as listed in Table 4. Section 2.3 of the literature review covers several typologies for running a crowdsourcing project, beginning with collection right through to evaluation. There are also stepwise solutions for selecting participants and selecting the best implementations or solutions. For example, Sloane and VOLUME 10, 2022 Paul [6] present the Multistep Dynamic Expert Sourcing framework, which is a three-step process for selecting the most relevant experts to a task. As shown in Fig. 6, the submission in each step ranges from a half page on problem understanding to 50 pages of a full solution, and the seekers use these submissions to filter the experts. The Climate-Colab has similar multiple rounds of evaluation and revisions [15], which provides solvers with opportunities to refine their proposals and add contributors prior to the second round of judging. In fact, the more selective and aggregated divisions in the process, the more rounds of feedback there tend to be, which can help to overcome any feedback loop problems [23].
On a fundamental level, a scientific research proposals and innovation crowdsourcing schemes share a very similar structure. Developing a research proposal requires one to formulate research topics then the proposed research is outlined in three parts: the scientific problem, the research tasks to be undertaken, and the plan for undertaking them. From this perspective, seekers might select participants according to research topic at first, then collect and evaluate research plan. Along similar lines, Polyplexus.com guides its participants by following the scientific method, with an evidence phase followed by a conjecture phase. A scientific conjecture collection process is formed to help the seeker select the best scientific problem.
Although not necessarily specified to the crowd, processes for formulating actual technical solutions to problems have been well-covered in the literature. One model, the Vmodel [47], starts with a requirements analysis before moving into systems design, then prototyping, then engineering and implementation. Some parts of this process can be selected and crowdsourced with the goal of ideation being to crowdsource the requirements analysis and the system design.
Another key process worth mentioning is that of aggregating the offered solutions. As Geiger et al. [22] describe, this process can be integrative where contributions are combined to form outcomes, or, as with Climate-CoLab, the seeker can select any action deemed likely to meet the stated goal [15].

3) KNOWLEDGE SCHEMAS
The third dimension, knowledge schemas, determines how knowledge flows between the seeker and the solvers. Chesbrough [18] sees knowledge flows as the most fundamental aspect of open innovation, recommending ''the use of purposive inflows and outflows of knowledge to accelerate internal innovation, and expand the markets for [its] external use''. In this way, thoughtful knowledge schemas are imperative for a shared and common understanding of problem, solution and evaluation, and for providing structure to poorly structured or unstructured information. In innovation crowdsourcing, there are two main types of knowledge schemas. The first is between the seeker and each solver, which should be designed to convey information about the challenge to the solvers and feed back solutions/ideas to the seeker. The second is between solvers, which should be designed to guide desired levels of collaboration.
Many of the crowdsourcing innovation platforms, such as Challenge.gov and Polyplexus.com, provide a challenge schema template for seekers to include with their initial call. The solution schema templates ask solvers to respond to the challenge with information on their understanding of the requirements, their solution/proposed approach, budgets, and timelines, suggested technical paths, project plan, etc. Others, such as OpenIDEO.com and Yet2.com, use an idea schema, which is essentially a solution summary. These ask for the title, an overview of the key value proposition in the idea and supporting references to prove feasibility. In other scenarios still, such as 99design.com and threadless.com, the solution/idea schema is simply a design prototype.
Evaluation feedback schemas often take the form of a template that lists the judgement criteria for seeker (or solvers) to use to score one or more solutions. This raises the issue of who will be evaluating the solutions. In innovation crowdsourcing, there are two main levels of knowledge sharing. The first level is where the solver opens access to their solution for others to refer to or reuse. An excellent example of this type of knowledge sharing is the research papers we all write as academics for publication. Here, we hope and expect that other researchers will reference our ideas, take them up, build upon them, and, most importantly, critique them. The Climate-CoLab works this way, where every solution published by a solver is open to the public. The second level is where the crowd becomes directly involved in evaluation -akin to the peer review process as opposed to simply including critique in one's own contribution. At this level, solvers can not only access the solutions of others, but they can also evaluate them. The mode of evaluation says something about the kind of feedback [24] that the seeker desires. For example, a rating or thumbs up/down may be appropriate for quickly arriving at a shortlist or for gauging external acceptance of an idea, whereas a comment system would usually be aimed at identifying shortcomings with the idea through open interactive forums. In 2009, Riedl and Finzen [37] proposed the ''Idea Ontology'' for innovation management to provide a common language for idea sharing/reuse at the solution level. Through this ontology, they promote the interoperability of ideas among participants in open innovation and explicitly support feedback at the evaluation level via channels like comments and rating. Many idea portals have adopted schemas similar to the Idea Ontology, including Threadless.com, Polyplexus.com, and Climatecolab.com. Without a doubt, knowledge flow management is becoming an important and promising research direction in open innovation.

4) INCENTIVES
Incentive mechanisms determine how the solvers will be remunerated for their ideas [22], [23]. Although solvers may have both extrinsic and intrinsic motivations for participating in innovation challenges, extrinsic motivations typically play a more dominant role [38]. Hence, monetary rewards and job opportunities are two most common forms of compensation, with money being a prime motivator for most solvers [38].
From a game theory perspective, innovation crowdsourcing follows equilibria behavior. An analysis by Moldovanu and Sela [39] shows that, in most situations, a winner-takeall strategy is usually the best reward allocation. However, this does depend on some assumptions. Notably, Segev [40] demonstrates through empirical research that offering a higher reward does attract more solvers but does not tend to result in higher quality solutions being offered.
As the second main motivator, job opportunities can present a win-win situation for seekers and solvers. The winning solver receives technical and financial support, while, for the seeker, using innovation challenges as a method of recruiting talent has proven to be highly successful [41]. Rewards of opportunity are relatively common on InnoCentive.com, Ideaconnection.com, and Datacon.com.
Social capital and reputation are also important factors for extrinsic motivation. In practice, the platforms and seekers interview challenge winners to increase their reputation and social capital. This not only prompts other solvers to share their knowledge and skills, but also contributes to building a community based on the principles of serving a greater good [48]. For the solver, an announcement of their success also can engender a great sense of achievement and pride as an intrinsic motivator.
Common intrinsic motivators include efficacy and learning [6]. For example, the majority of teams who took part in the Netflix Prize data competition did so to exercise their skills, learn new ones, and receive feedback for improvement [6]. Again, however, although efficacy and learning tend to attract more participates, they are not a key factor for improving the quality of the winning solution. Social identity and a sense of achievement are other intrinsic motivators that should be considered in crowdsourcing innovation [6], [38]. Importantly, seekers should keep in mind that their controls over intrinsic motivation can be limited, while the impact of extrinsic motivators often hinge on both seekers and the contest platform/community.

5) RISK MANAGEMENT
Of course, the ultimate goal of the fifth component, risk management, is to ensure the innovation goal is met. Just as the name implies, risk management is central to dealing with the vulnerabilities of any open innovation strategy. In general, the greatest risks with innovation crowdsourcing surround the knowledge lifecycle and, more specifically, knowledge generation, knowledge sharing, and a lack of knowledge, as shown in Table 6. The most common risks related to knowledge generation and knowledge sharing both revolves around unwillingness. An unwillingness by solvers to generate knowledge usually indicates they lack motivation, while and unwillingness to share their knowledge usually indicates a lack of trust. What can be helpful in mitigating these two forms of risk is crowd monitoring. InnoCentive.com, for example, provides a crowd monitoring service where the seeker receives weekly reports on the number of participants and submissions. Detecting waning interest early should raise questions over why and, in turn, appropriate remedies. To combat trust issues, intellectual property policies are an obvious strategy, such as the ''Intellectual Property Infringement Policy'' on Threadless.com or InnoCentive's ''Challenge Specific Agreement''. In fact, intellectual property risks are among the most important to consider in open innovation [6], [18]. For crowdsourcing innovation, most IP protection is occurred on the internal information which is seeker revealed and the solution which is solver submitted. In a comprehensive review, Xiao et al. [42] run through several intellectual property protection strategies managers' use in the pharmaceuticals industry to protect their core technologies. Bogers [49] developed an inductive framework based on multiple intellectual property protection cases for knowledge sharing and protection in R&D collaborations. Many of these are applicable to innovation crowdsourcing.
Risks related to a lack of knowledge arise when expert intelligence needs to be developed before a problem can be solved. This is discovery in the truest sense and, as we all know, there is no magic formula for solving this problem.
The literature touts many other risk factors with crowdsourcing innovation along with corresponding mitigation strategies [1], [32], [43]. The factors include relationship complexity, control effectiveness, coopetition, loss of knowhow, and so on, and the mitigation strategies are often comprehensive and multi-factored. Souza, Ramos and colleagues [1], [32] have summarized many of these risk factors and developed a risk management methodology to increase success rates in crowdsourcing innovation. The methodology consists of three phases -risk assessment, risk mitigation, and continual evaluation assessment -along with activities to identify, mitigate, and monitor the risk during the entire crowdsourcing life cycle. One common feature of most studies on innovation risk is their attention to highlighting that risk events emerge as the ex-post result of uncertainty in unknown environments. Therefore, the old adage ''knowledge is power'' may be one of the best strategies for averting all risk.
The last risk factor that cannot avoid mention is one of cheating in the competition. Unfortunately, cheating is far too common, especially in data science competitions, with millions of dollars in losses suffered from leaked answers, sabotaged data samples, breached scoring systems, and other tactics. These events underline the importance of both protecting the data assets of the competition and taking a holistic view of risk management for crowdsourcing.

C. SEVERAL CROWDOUSRCING INNOVATION WEB PLATFORM CASES
Five characteristics in the proposed framework are sequentially dependent. When we are instantiating the framework in specific field, innovation goal is the formulation of origin problem, it determines the process of innovation that we can partition for feedback. Partitioning the processes establishes the knowledge schema and its possible modes of sharing. Incentive mechanism is established for the knowledge generation and sharing. To ensure the high-quality of crowdsourcing outcomes, a risk management strategy should be considered throughout the whole process.
Overall, the conceptual model presented contributes to a more systematic understanding of the generic processes involved in approaches to crowdsourcing innovation, which is useful when designing such initiatives. To further assist managers in these endeavors, the following demonstrates five different use-cases of the framework and shows how it might be used to analyze the pros and cons of these online web platforms, each typical of a particular application type of crowdsourcing innovation.

1) NETFLIX
In October 2006, Netflix [44] released a large movie rating dataset and challenged the data mining, machine learning communities to develop a recommendation system that could beat the accuracy of Cinematch. The competition came with the promise of $1 million grand prize to the team who could improve the accuracy of the system by an additional 10% RMSE. However, the real merit of Netflix's competition design was to award progress prize to incentive knowledge sharing at the solution level. Contestants were required to make predictions for 3 million withheld ratings in the qualifying set. RMSEs were computed automatically from a fixed, but unknown, qualifying set called the quiz subset. The submitted result values were reported to contestants and posted to the leader board, but the test subsets were not made available. Rather, they were used by Netflix to identify the potential prize winners. Participants were able to discuss the competition in a provided forum. After almost three years, the BellKor's Pragmatic Chaos team was ultimately announced as final winners on 21 September 2009.

2) INNOCENTIVE
InnoCentive [33], [35] is an open innovation market founded in 2000 from Eli Lilly's new business model incubator. It connects ''seeker'' firms posing scientific challenge with external ''solvers'' who submit solutions. Solvers who succeed in providing solutions are rewarded with cash prizes in exchange for associated intellectual property. Additionally, solvers receive access to a collaboration database to find potential partners. Through 20 years development, InnoCentive has formed five types of intermediary services for open innovation. One prominent advantage of the InnoCentive model is the maturity of its business processes, and particularly the range of pipelines offered to help its clients meet their innovation objectives. Its team-building and collaboration introduction tools are also another significant merit.

3) CLIMATE-COLAB
The MIT Climate-CoLab [15] platform is a contest web dedicated to addressing complex climate problems. It provides a pool of contests covering different aspects of climate change, plus an integrated contest designed to combine the proposals for each of the various pool contests into one unified plan for solving a more complex climate problem. In each contest, the goal is described on a provided template, and each proposal in response is visible to any member for comment and rating. The proposals are evaluated and respondents are given two  opportunities to revise their final proposals before the finalists are selected. A panel of invited expert judges then selects the winners of the Judges' Choice Awards. There are also Popular Choice Awards, voted on by the community.
Interestingly, the Climate-CoLab also creates a taxonomy of different parts of the problem so as to facilitate the reuse of basic problem proposals. Through these means, complex problems are divided into smaller subproblems, and complex problem proposals are built by combining responses to the subproblems together. A virtual pricing system motivates members to participate in the integrated contests. The basic idea is to motivate the members to contribute to both the integrated and basic proposals through a point allocation system. Most members are keen to receive recognition for accumulating a large number of points.

4) POLYPLEXUS
Ployplexus [45] is an interdisciplinary scientific research platform initially funded by the Defense Advanced Projects Research Agency (DARPA), which is designed to:

1) Increase researcher efficiency by encouraging interdis-
ciplinary collaboration by conversation on the scientific evidence. 2) Strengthen the quality of proposals by building a diversity background community. 3) Surface more funding opportunities by facilitating realtime dialogue with researchers and sponsors. At the design level, it opens up the procedure of scientific research process [46] and crowdsources the formulation of research problem steps. It provides two basic knowledge schemas called MicroPublicaitons that includes microevidence and micro-conjecture. The micro-evidence is a scientific fact that summarized from literature, which can be shared and communicated as knowledge flow in the platform for knowledge synergy and emergency. The microconjecture is generated based on two micro-evidences, it is private and can be collected in portfolio which is a basis of constructing research hypothesis and selective to share with other users. In order to incentive users to create more micro-evidence, Ployplexus hold an evidence competition, and award the most featured evidence. It also provides incubators for specific research field to connect the sponsors and researchers, in incubator research can directly obtain feedbacks from sponsors. Till June 2021, Polyplexus community has accumulated registered user more than 3500, generates more than 10000 MicroPublications, and open 51 public incubators.

5) 99DESIGNS
99designs is a crowdsourced graphic design platform where users create design contests for other users to submit their ideas for evaluation, competing for monetary amount offered [52]. The contest is mostly composed a short title, a design brief and a package as innovation goal description. The title and the brief are description of what is needed. A package specifies how much will be paid to the winner. For the contest process, it commonly will run for at least 7 days. Through three rounds of filtering, one designer is chosen [52]. For collaboration, at the filter stage, contest holder could give feedback through comments. at the final stage, contest holder works closely with winner to finalize the design and obtains full copyright. For incentive, it also provides package service to keep design crowdsourcing quality. Since 2008, 99designs freelance community has brought more than one million creative projects to life for thousands of genius entrepreneurs, savvy business owners, and brands with big ideas.

D. DISCUSSION
Crowdsourcing has become a popular and effective approach for open innovation. Many successful cases indicate that the design of crowdsourcing innovation is a decisive role for a successive crowdsourcing activity [15], [35], [44], [52]. However, the existing frameworks are empirically induced from different operational perspectives, and thus lack theoretical interpretation for the structure of framework. By applying our proposed framework to analyze five typical cases of existing crowdsourcing platforms listed previously, their merits and design tricks can be profoundly revealed. Further, the five dimensions of the proposed model have logical interdependencies, which can be the initial point of crowdsourcing design methodology. The proposed conceptual model is still not detailed enough for other platform cases mentioned in this paper, such as OPENIDEO, CROWSPRING, Threadless.com, Chanllenge.gov, etc. In other literatures, several crowdsourcing design methodologies were proposed for a specific field. A gamified crowdsourcing design framework for higher education was proposed and instantiated to a real digital setup case [53]. It emphasized ''gamification'' for intrinsically motivation which is within the dimension of incentive and ''control mechanism'' for quality control of contribution which is considered by risk management in our model. In the generation stage of industrial product design concept, thanks to data technics, relevant unstructured customer reviews (CRs) can be firsthand concept generation resources for product designers [54]. In this framework, there are two stages for the process dimension, like Climate-Colab, the first is CRs collection stage, and the second is designer generation/combining/trial stage with CRs data processing, which has the same function with the What-Where-Who-How taxonomy in Climate-Colab. On the one hand, it makes full use of knowledge schema and controls the knowledge flow by data technics at the second stage. On the other hand, obvious shortcoming which can create design bias is exposed because there is no direct feedback loop from customers. The current framework is a conceptual model; although it covers nearly all the cases mentioned in this paper, we believe a more systematic approach for detailed design is highly desirable. We also believe that many key problems, especially for crowd collaboration, need to be addressed [50].

IV. CONCLUSION
In this paper, crowdsourcing innovation is defined as a control system, in which the innovation seeker controls the crowd to solve innovation problems. From a cybernetic perspective, the control laws are used for system identification, i.e., deducing the system components, relations, and information flows. From an open innovation perspective, the information flow is identified as the knowledge flow. According to these two perspectives, a five-dimension conceptual model for innovation crowdsourcing system is proposed, which emphasizes on the nature of innovation crowdsourcing and provides an initial point for innovation crowdsourcing design. In a normative aspect, the proposed model better interprets the existing typical cases and covers their design tricks. In aspect of design framework, it demonstrates more generality. Under the proposed framework, the typical cases can be resolved as a configuration of the five dimensions.

A. PRACTICAL IMPLICATIONS
In many instances, crowdsourcing innovation activities fail to deliver the promised benefits. Therefore, it is highly important for seekers to understand the mechanisms that enable these activities to successfully generate valuable solutions. The findings provide guidance to crowdsourcing innovation organizers to design an initiative from the primary innovation goal to risk management strategies, through mechanical processes, knowledge sharing, participant motivation, and risk management to ensure that the goal is realized. It provides suggestions to managers what information is necessary to share with the participants, how to promote the innovation process in several phases, how to incent the participates, and what is the root of the problem if the results are not ideal. Second, the principles of control incorporated into the framework stress the importance of collaboration and feedback, which should be elaborated by managers for a successful crowdsourcing innovation activity. The findings indicate that knowledge sharing is the key design element, and thus how to harmonize the collaboration and IP right is the key to success.

B. THEORETICAL IMPLICATIONS
This article contributes to the extant research in two ways. Firstly, most of the present studies of crowdsourcing framework are at operational process level and are concrete on a specific area [15], [33], [35], [52]. Our findings advance crowdsourcing framework by normative research with two novel perspectives. The five dimensions uncovered in this study provide solid evidence for how to control the crowd to solve innovation problems, which not only promote the in-depth understanding of the multifaceted nature of innovation crowdsourcing, but also provide a system for crowdsourcing innovation subject. The innovation goal how to formulate a problem is an important research direction in problem solving. The knowledge schema, i.e. the organization of knowledge, is a hot topic in knowledge management. The dimension of incentive is the fundamentals of economics. The dimension of processes is the subject of methodology, and risk management is an important direction of management science. Secondly, the cybernetics contributes to a cooperation synergic understanding of the generic processes in crowdsourcing innovation approach. It indicates the emergence phenomenon of crowd coming from the collaboration of innovation solver. How to overcome the contradiction between knowledge sharing and reward allocation will generate many dilemmas for research.

C. SOCIETAL IMPLICATIONS
In the coming decades, crowdsourcing will become an important approach for inventive problem solving. According to our findings, the researchers and practitioners will find it convenient to apply crowdsourcing innovation, and some typical innovation methodology would be crowdsourced. For example, John Gero's Function-Behavior-Structure model and ontology, the design model could be developed with three stages of Function-Behavior-Structure, and the Function-Behavior-Structure ontology is naturally knowledge schema. With the age of knowledge fragmentation, our proposed model provides guidance for knowledge management to accelerate convergence of innovation. It also identifies a new application area for ''cybernetics and society'', i.e., control of crowdsourcing problem solving. This proposed conceptual model is undoubtedly promising in society for both management and cybernetics.

D. LIMITATION AND FURTURE RESEARCH
Our proposed model is only validation in conceptual level, it lacks detailed design when applied in a concrete area in practice. Especially in the aspect of how to overcome the collaboration and IP right protection during the crowd innovation process, it will be the key direction of our future research. In addition we will make a trail to address the design and development of a crowdsourcing platform that can be applied for any problem solving.

APPENDIX
The list of characteristics of crowdsourcing approaches (see Table 12). He is currently an Associate Professor with the Institute of Computing Technology, Chinese Academy of Sciences. His current research interests include big graph mining, time series mining, and designing intelligent & scalable algorithms for massive data.
LI XU received the B.S. degree from Peking University and the Ph.D. degree from The University of Hong Kong. He is currently an Associate Professor with the Institute of Computing Technology, Chinese Academy of Sciences. His research interests include financial big data analytics, statistical learning, and information theory.
XUEQI CHENG (Senior Member, IEEE) is currently a Professor with the Institute of Computing Technology, Chinese Academy of Sciences. He has published more than 100 publications in prestigious journals and conferences. His research interests include network science, web search and data mining, big data processing, and distributed computing architecture. VOLUME 10, 2022