Proximity Metrics for Selecting R&D Partners in International Open Innovation Processes

The implementation of open innovation models for R&D activities has been extensively tested by many public and private institutions by using several models; however, both lessons learned and rationale for internal decision-making have not been systematically analyzed to serve as a basis for future institutional process management improvements, which increase the success rate. The large diversity of cases, firms, sectors and open activities, and the multiple factors affecting success has made it difficult to elaborate a general framework where to extract from some policy design rules adapted to specific contexts; thus, involved firms do not have a consolidated framework to measure the effectiveness of the open innovation practices in use. Open innovation for R&D activities conducted by multinational firms is usually carried out at the international level looking for specific knowledge and complementarity of partners. One of the critical factors for success in open innovation activities is the “right selection of adequate partners”. This paper postulates that the multi-variable selection based on the previous characterization of the relationship between the firm and one potential international partner constitutes a key factor for success. In addition, the article presents a semi-quantitative approach to measure the “differences” (a metrics based on “proximity” is defined) to analyze and compare between different partners from several dimensions. The key goal is to select external partners with the “minimum distance” to the firm. The proposed selection metrics were applied to various open research projects launched with university partners in the three Latin American countries (Mexico, Chile, and Peru).


I. INTRODUCTION A. LOOKING FOR A RATIONALE
The implementation of open innovation models for research and development activities by a technology-based firm has been extensively tested by many institutions through the implementation of several open model variations [1]- [3]; however, both rationale and lessons learned for internal decision-making have limited analysis to serve as a basis for future institutional process management improvements in such models [4].
The associate editor coordinating the review of this manuscript and approving it for publication was Nicola Andriolli.
There has been an impediment to elaborate a general framework to extract a set of common policy design rules in specific contexts [9] for the large diversity of possible cases where open innovation [5]- [8] is applied and where the multiple factors affecting open innovation success exists.
Even when firms are institutionally committed in implementing an open innovation strategy (termed here as leading firms) they face a difficulty to measure the effectiveness of the open innovation practices in use or to set-up a continuous improvement of open innovation processes and, then, to learn from the experience [10]- [12].
The use of open innovation (OI) models for R&D activities [13] conducted by multinational firms are usually carried out at the international level looking for specific knowledge VOLUME 7, 2019 This work is licensed under a Creative Commons Attribution 3.0 License. For more information, see http://creativecommons.org/licenses/by/3.0/ and complementarity of potential partners in some priority geographical areas. One of the critical factors for success is the ''right selection of adequate partners'' for specific open innovation activities [14]; both for conducting bilateral R&D activities between the firm and the external partner, or as a part of consortia where the R&D activity is performed by several entities playing different roles and tasks. It is important to take notice that this case of open innovation should be clearly distinguished from the more conventional and well-known case of ''subcontracting'' some activities by following detailed specifications. In the OI cases, both parties are assuming uncertainty and sharing higher risks in the open activity [11]. The concept of open activity is defined when the OI leading firm gives partners freedom and flexibility to find disruptive solutions for the research challenges which were less defined (detailed specifications are not possible for not conventional contract research).
To implement open innovation models, ''open innovation leading firms'' (i.e., those companies promoting open innovation activities) should identify and select the most effective external partners over the world based on their adequacy to the technical challenge to solve and their readiness in using open innovation schemes [14], [15].
Sometimes, the OI leading firm should focus on geographical areas where the interest of the open project is higher (e.g., closer to the target market or where there is a concentration of potential partners) or, on the contrary, it focusses in very specialized partners when specific technical knowledge is needed regardless of contextual constraints.
Potential partners differ significantly in missions and legal structure, and they should be carefully compared to select the right one. As an example, in the public sector, OI leading firms are partnering with universities and research centers, while in the private sector typically partners are start-ups, SMEs or even large companies (jointly participating in joint ventures).
The rationale to embrace in OI cooperation differs from one type of partner to another. In the case of (public) universities, OI cooperation in R&D constitutes a well-known mechanism to align the research and innovation agenda with the interest of larger firms and to increase external funding by keeping some freedom and Intellectual Property (IP) ownership (even if the Intellectual property rights are transferred to the firm). Furthermore, OI cooperation is a pre-condition to access to public research and innovation funds in open calls issued by funding agencies. Specifically, this is the situation in the European Union Framework Program [16] and in many national research and innovation programs where open calls require to submit a proposal submission by a ''consortium'' of public and private entities.
Companies implementing OI schemes with universities or research centers [17] are well-aware that a wrong selection of external partners can jeopardize the whole open R&D activity [18]; furthermore, wrong selection introduces a highrisk factor in the efficiency of OI models where it is necessary to identify, characterize (in terms of primary/secondary causes, impact and probability of occurrence) and mitigate, if necessary, with specific measures. This situation calls for the careful definition of specific and efficient management procedures adapted to the type of OI leading firm, external partners and activities.
This situation also differs from the cases when the firm launching the OI activity with external partners is a small and medium enterprise (SME). As an example, SMEs require R&D collaboration with large corporations despite risks on misappropriation of technology secrets according to resource-based theory. In order to achieve this collaboration, protection of their own critical technology by strong Intellectual Property Policy (IPP) is critical [19]. It is important to mention that the well-known problem of paradox of openness-the creation of innovations often requires openness, but the commercialization of innovations requires protection. Pommerening and Al Wawi [20] have pointed out that ''the main goal they try to achieve when choosing partners is to build collaborations with: the highest quality of outcomes, most cost-effective activities, and most time-effective processes.'' The risk behind comes from the internal cost of identifying and selecting one external partner by an SME; high costs act as a barrier, and the firm needs to optimize it by cooperating with the same external partner in the future. Thus, for SMEs, the selection process is even more crucial than for larger firms with higher internal resources available, since these larger firms can repeat the selection process more frequently. Hossain and Kauranen [21] have identified some lessons in the SMEs cooperation revealing that SMEs are more active in open innovation than generally perceived although not necessarily benefiting from cooperation in R&D: • SMEs collaborate more in the commercialization stage than in innovation activities.
• Careful trade-off between revealing and protection of IP in collaboration.
• OI is valuable for new product development but risky for incremental innovation.
• Proactive openness results in important strategic innovations.
• Mixed results on whether external innovation benefits in innovation performance.
• New entrants adopt opens system more promptly than incumbents do. More relevant than the legal structure of the partner entity is the ''role'' it plays in the OI activity. Roles can differ from ''suppliers'' (both for knowledge or components), ''competitors'' (working in pre-competitive technology development-one example is the cooperation in the development of 5G standards for mobile communications regardless the competition in specific mobile products and services), ''venture capital'' (for start-ups), ''governmental or regulatory bodies'' (to support standardization activities which are open by nature), ''freelances'' (highly skilled people), ''users/consumers'' (to accelerate product development or market introduction), etc.
For some authors [22] the number and type of partners with which a company collaborates is something that determines the level of openness of the innovation process of a company: the more partners the company has, the more ''open'' its innovation process is.
The degree of openness in a collaboration network for innovation depends on the extent to which membership is open to anyone who wants to join [23], [24]. In totally open collaboration (i.e., crowdsourcing schemes), everyone (suppliers, customers, universities, governmental institutions, private research centers, and even competitors) can participate in providing solutions to pre-defined open technical challenges by following pre-defined rules to enforce quality of contributions.
This article is focused on the case where the OI leading company is opening the R&D cooperation only to a limited number of external partners. In these cases, the company shares the technical challenge to address with few previously selected parties because it believes they have the crucial capabilities and assets to provide innovative solutions on time and with high quality. For that reason, pre-selection becomes a key issue to increase the success rate. As the number of participants increases, the likelihood that a participant's solution will be selected decreases. In such situations, transaction cost economics [25] suggests that the best potential partners can be discouraged from participating because they do not want to make transaction-specific investments that cause resourcewasting, if they are not sure of being selected. Thus, the best parties prefer to participate in fewer relationships (i.e., closed modes), to be maintained in the long run [24].
Within this context, the relatively high ''transactional costs'' related to the selection of an external partner are compensated by long-term partnerships where some open activities can be framed around some research and innovation lines for months or years; in those cases, mutual trust between parties is easier to get and consolidate.

B. THE RISKY PROCESS FROM AN ENTREPRENEURIAL ACTIVITY
Problems reported in the implementation of OI activities reflect the existence of difficulties (e.g., delays in launching activities or cultural biases or barriers to transfer results) to execute OI activities. Increasing transactional costs jeopardizes, in practice, the expected advantages of OI. In other cases, the management and coordination costs are higher than expected to limit the scalability of OI use by a firm. For that reason, firms tend to rely on well-known subcontracting practices to engage external partners (e.g., specific contract models borrowed from contract research practice). These practices are far from OI principles in IP, risk-sharing and freedom dimensions.
Effectuation and causation theory [26], [27] has largely been developed and employed to analyze individual startups or relatively small firms and their management teams. It is important to mention that ''effectuation'' is a way of thinking that serves entrepreneurs in the processes of opportunity identification and new venture creation. Effectuation includes a set of decision-making principles expert entrepreneurs are observed to employ in situations of uncertainty. Situations of uncertainty are situations in which the future is unpredictable, goals are not clearly known and there is no independent environment that serves as the ultimate selection mechanism. Experienced entrepreneurs tend to apply effectuation rather than causation, while uncertainty does not have a systematic influence. Entrepreneurs using causation-based international new venture creation processes tend to engage in exporttype entry modes, while effectuation-based international new venture creation processes do not predetermine the entry mode [28].
We assume that such a firm will be involved in more partnerships and these partnerships will have taken on an institutionalized character (even promoted or supported by governmental agencies to get more competitive firms). The interpretation in the context of OI means that partner selection probably more often takes preexisting formal or informal networks as a starting point, corresponding to an ''effectuation strategy,'' even though the firm may have the resources to pursue a more formal causation approach.
In the case study analyzed by Solesvik and Gulbrandsen [29], two conclusions were extracted: 1) Firms that had mutually beneficial relationships with certain firms in past open-innovation projects would tend to engage the same partners in new openinnovation projects. 2) Firms that prefer to keep sensitive information related to a product to be created in an open innovation project tend to select partners from those firms that they know from the past and have established trustful relationships, rather than select partners in the market. Similar conclusions were extracted from the European Commission (EC) in analyzing the cooperation in consortiabased projects in the European Union Framework Programme [30]. Groups of partners repeat the participation in several consortia over time (even in different topics).
The search for partners in open innovation settings often consumes substantial time and managerial attention. To optimize time and cost organizations tend to get trapped in local searches, which typically leads to collaboration with partners already known to them [31]. This behavior eliminates the cooperation with non-local partners, even when those nonlocal partners were potentially better.
The relationship between one specific firm promoting and funding one OI project and one external partner can be characterized by using a set of factors. The right combination of factors affects the decision to select one partner from a group of potential ones, it affects the way that OI is conducted and controlled, as well as how it influences on the results and success rate of the OI activity.
This article postulates that multi-variable selection based on matching the characterization of the relationship between a firm and their potential international partners and their behaviors with the technical challenge to address constitutes VOLUME 7, 2019 a key factor for success (understanding success as the capability of fulfilling the declared objectives within time and budget). We postulate that the success in running OI R&D activities is strongly dependent on the adequacy and performance of the external partners involved in the proposed activity (one or more depending on the approach used); The management procedures related to partners' identification, partners' pre-qualification, partners' selection (for one specific research activity), and partners' monitoring during the development of open activities constitute a basic ingredient for the definition and implementation of the appropriate OI R&D model.
To address that goal, this article presents a semiquantitative approach to measure the ''gap'' between firms and external partners (a metric based on logical ''proximity'' is defined). This article analyzes and compares between different partners from several dimensions: previous common experience in open activities, technical proximity to the open challenge, cultural compatibility, regulatory facility to commit resources for international activities, and the type of management support needed. The key goal is to identify those external partners with the ''maximum proximity'' (or ''minimum distance'') concerning the firm and to engage them in open activities with minimum resources.
The proposed proximity selection metrics were applied to open pilot research projects launched with university partners in three Latin American (LATAM) countries-Mexico, Chile, and Peru. The Universidad Politécnica de Madrid (UPM) was conducting open innovation activities with these countries framed in a larger subcontract agreement with an international technology company. The intention with these projects was to learn from the experience on the implementation of OI models in international networks as well as to extrapolate lessons to other geographical areas and larger number of partners. In this context, the UPM is serving as an intermediate hub with full responsibility in selecting and contracting partners.
The paper is structured as follows: after an introductory section, in Section II describes the theory development. The article introduces the basic concepts on which the article is developed. This is followed by an outline of the methodology in Section III, where the metrics to select the partners is shown. In Section IV, the application of the methodology to select the LATAM partners is presented. In Section V, some lessons learned are extracted and discussed to serve as a guide for sustainable and scalable OI processes at the international level. Finally, some conclusions are shown in Section VI.  presented as one of the drivers to ensure the success of open innovation in R&D in technology-based firms.

II. THEORY DEVELOPMENT
More formally (see Table 1), partners' relevance could be understood as a function of the role played by the leading OI firm's required partner and their legal structure (which condition cultural bias and extrapolation from previous experiences). Both aspects will condition the type of desired relationships which are necessary to create, maintain and optimize open cooperation.
Five roles were identified to cover specific phases of the innovation activity (which were taken as a basis as a linear model from idea generation to commercialization). As Table 1 suggests not all types of partners are well adapted to play specific roles, but it is common to be able to play more than one (probably in different open activities). This article will not consider the last role in the table (marketing, sales and users' support) which is outside the scope of this article's focus on OI R&D. Furthermore, the emphasis is placed on public partners (see the colored area in Table 1).
Firms involved in OI activities should cooperate with a diverse set of external partners playing different roles in activities open to external cooperation. Even when many of them could be identified as potential partners, not all possible partners can participate in all phases of cooperation for several reasons. A complementary view of Table 1 is assigning partners with their current involvement in OI activities; this information provides the basis of the structure of a firm's OI relationship framework. It is based on three main categories of partners related to their situation in the network:  Impact of open innovation cooperation in organic growth. Note: Size represents the volume of resources involved. Notice that the same external partner could appear in more than one program.
• ''Dormant'': contacted and well-qualified partners (see Section V) as part of the firm's ecosystem anticipating OI activities in the (near) future.
• ''Non-active'': identified partners who could potentially participate in future open activities if they were selected for them (even when they are not aware of the identification made by the leading OI firm and they still have not been accepted to become part of the open ecosystem). The intended goal for the OI leading firm in the selection process of external partners is to optimize its management resources (i.e., highly skilled human resources and time devoted to management issues) needed to ensure the scalability of the use of OI schemes. The way to address this barrier is to increase the number of open activities running simultaneously with the same or different external partners managed with limited internal resources and not to rely on organic growth. Fig. 1 depicts this situation where two snapshots of the firm's activity are described. On the left part of Fig. 1 all activities (''operation'' and ''projects'' framed in ''programs'') are developed by using internal resources; on the right side of Fig. 1, both internal units and external partners (EP) are used to cope with a growing number of projects and operations.
Notice that in the example, only programs 1 and 3 are affected by the implementation of OI schemes and many of external partners are related to new ''projects'' (only one is linked to a new ''open operation'' related to program 1).
Many of the identified external partners (EP) in any of the three categories mentioned above could be located in countries different than the tech-based firm headquarters' country, which would pushing the open network to a true international context. 1 If properly managed, firms should be able to build up and maintain their open innovation networks at an international level with a fast-growing and dynamic number of potential partners in several countries (by assuming dynamic changes (new entrants and exits) in the network). This approach allows the firm's expansion without relying on organic growth based on recruitment and investments in new equipment (if partners can have it used in the cooperation project).
Within the open process mentioned above, specific techniques are needed to identify adequate partners for a given OI activity. The use of the concept of ''distance (proximity)'' to the firm as a proxy of engagement capabilities (e.g., higher/ lower success) will provide a basis for managing those OI activities at international level.
The selection process of external partners runs in parallel to the strategy followed by large firms towards the internationalization of research capabilities based on the creation of their own corporate research centers abroad (if necessary, complemented with several partnerships with local R&D entities). Large tech-based firms have pushed it to access to skilled talent ''closer to local markets'' (taken as an internal decision of the firm but with the pressure and support of governments abroad to increase added-value of foreign investments) and to be closer to local customers' peculiarities.
In the context of the implementation of OI models, partners' selection techniques are used as key management tools to facilitate the identification of adequate local partners and to reduce the ''distance'' between those partners and the firm who seeks better positioning in future engagements of the firm in other countries.

2) FROM INDIVIDUAL PARTNERS TO BUILDING UP AN OPEN INNOVATION COMMUNITY
The identification process mentioned in the previous section does not imply any relationship between pre-identified partners. The usual approach relies on bilateral agreements between one partner and the OI leading firm.
Nevertheless, by taking advantage of the knowledge, capabilities, and complementarity of external partners, firms VOLUME 7, 2019 implementing OI models could be interested in creating a ''community of OI partners'' ready to participate in future open innovation R&D activities and to redirect to the specific requests for proposals to address technical challenges. They could be ready to exchange detailed contextual and technical information to participate in R&D projects jointly.
The building up of the ''community'' is a key strategic task. The OI leading firm identifies that some external entities could have the required potential knowledge potential to become partners in one specific domain open to external cooperation; then, the firm should approach external partners and convince them to participate in the OI community.
Not necessarily all identified potential external partners for one firm are valid for conducting one specific R&D activity; reasons to reject (or not select) a given partner for one activity, even if it was interested in participating, range from the lack of experience in open R&D cooperation with industrial firms to the lack of knowledge or practical experience on the technical challenge to solve, or due to very different cultural roots which makes a successful cooperation difficult.
Furthermore, there is an implicit competition within the ''community'' because other pre-identified partners participating in the above mentioned ''OI community'' could propose better R&D proposals (according to various evaluation criteria) and they could be selected instead. In addition, the possible complementarity of partners can provide the basis for developing more complex activities if the growing management complexity is well controlled.
This preliminary identification could begin by reviewing highly cited scientific publications in the field or a number of related patents where key researchers of the external partner appear as authors or inventors, apart for public information uploaded on websites on relevant R&D projects. Unfortunately, in these public sources no information is easy to obtain in regard to academia-industry cooperation culture, the performance in meeting deadlines or budgets, the quality and efficiency of their internal management procedures, etc. Information can be collected however when indirect references are obtained other external units or firms (within the company or from other well-known entities) with previous joint activity. Furthermore, this process is even more difficult to obtain, because within a single partner entity striking differences appear from one firm's unit to another.
Even if the partners' selection is restricted to a subset of those entities with the adequate knowledge and capabilities to address the technical aspects of the intended activity, and direct knowledge about their performance is well established (usually from previous joint activities), they are not necessarily ready to operate in an OI context at an international level in a given moment. Cultural aspects, legal frameworks, timing to start the activity, and pre-assumptions about the open innovation IP context should be taken into account to guarantee success.
Thus, all tech-based firms involved in the implementation of OI approaches for R&D activities should manage a set of potential risks in the selection of partners, which imply the need to implement a set of actions and procedures to mitigate those risks through the right selection and monitoring of potential partners.
The article will set-up the adequate framework and concepts for cooperating with external partners, as well as the selection criteria and the management procedures to guide companies in the best selection of international external partners for OI R&D activities.
More specifically, the next section will set-up the partners' landscape for a systematic selection of external partners; then, a simple ''distance-based'' metric to understand the closeness of a potential partner to the OI activity proposed by the leading firm is presented. This will be the basis for describing the proposed management procedures to qualify an external entity as a potential partner. Next, the concept and advantages of building up a community of solutions providers is discussed and, finally, the article addresses the applicability of the proposed system in selecting technology innovation partners in LATAM.

B. PROCESSES REQUIRED FOR QUALIFYING EXTERNAL PARTNERS
A key element to creating the international network of OI partners is to define a set of well-rounded criteria for the selection of potential nodes and to set-up some mechanisms to build-up the sense of belonging to a given community of ''solutions providers''. These are pre-requirements in order to create a stable and trustable international network of committed partners.
Usually, in critical industrial sectors, this process is addressed through the implementation of pre-qualification techniques. To our knowledge, pre-qualification of potential entities (companies, universities or research labs) is a typical procedure in cases of strategic large projects in high-tech sectors such as aerospace implemented by public institutions in the space sector like ESA, NASA, or ministerial departments (e.g., for Defense and Security projects, or large scientific research facilities). The approach followed by the Department of Defense in the USA to require a minimum Capability Maturity Model (CMM) level 3 to obtain a contract with the Department of Defense is an example of pre-qualification to participate in tenders.
In short, only those partners satisfying the various requirements could be considered as potential partners, and then, they would become ''qualified'' by the firm to have access to future open technical challenges and to elaborate proposals for funding. To be qualified entitles presenting a proposal but it does not imply to receiving funds; it will depend on the evaluation process and competition of other potential proposals.
This pre-qualification approach is commonly used by large companies or institutions (in this section will focus on the interest of large multinational companies usage) for four main reasons which globally contribute to the firm's risk reduction strategy.
• The complexity of the challenge requires a long and detailed exchange of technical information which cannot be offered and discussed to any partner because it implies the allocation of expensive internal firm's resources. It is important to mention that complexity also refers in this context to the use of immature emerging technologies (low TRLs), the size and interdisciplinary skills of the project team, the need to buy/adapt preexistent equipment, and other aspects found by one entity.
• Some challenges will imply the access to confidential information and for security reasons NDAs should be provided by all participants before accessing that information (even when there is no commitment for final for funding selection).
• Risks' exposures are high and one of the techniques to avoid them or to reduce the likelihood of occurrence is to request some previous ''certification'' of adequacy which ensures that the partner uses some management control procedures (this is the basis of requesting one threshold of CMM 2 levels or ISO 9000 to be eligible).
• Qualified members of the network can share some information to promote cooperation between them, and to offer better solutions to complex challenges by joining forces. The combination of these reasons has advised many companies to define and approve ad hoc pre-qualification systems in order to reduce risks when implementing OI models at the international level. Some companies have developed a sophisticated multilevel ''certification of knowledge on some products or services'' system as a pragmatic proxy for the wider qualification concept described here. Certification is used to ensure that persons (at the individual level) or entities (at the institutional level) get a certification (its/his/her knowledge is certified by an external unit at some pre-defined level) on proprietary products or services. Very well-known companies (CISCO, IBM, Microsoft, Salesforce, etc.) have well-established knowledge certification systems at several levels with professionals participating over the world. The same certification could be delivered to an entity if both the management procedures and key professionals are certified.
Formally, the qualification system is applied to any potential partner interested (or previously invited) in participating in an open innovation activity proposed by a company. A generic pre-qualification system consists of the three steps depicted in Fig. 2.
The stage 1 is aimed at informing potential partners about the possibility to become a pre-qualified partner for the firm. Some information is widely distributed (or more focused as we will discuss later) to explain goals and methods and to motivate external entities.
Usually, this process is launched through an ''invitation'' to those entities (or units within entities-For instance, departments or research groups of a university if more granu-2 CMM: Capability Maturity Model larity is needed or convenient-) which have attracted the interest of the firm based on the review of public information like scientific publications, patents, participation in relevant projects, etc. In addition, these entities should satisfy some legal conditions to be eligible (e.g., legal personality as a public entity, headquarters located in some country/ region, etc.).
Invitations are sent at any moment to one entity (it is not necessary to synchronize them as a part of an open call because next stages are conducted bilaterally also). This process should not be confused with the stage 3 described later.
As a result of this process, invited entities should (formally) answer the invitation received and, if they were accepted, the stage 2 qualification system could start with them.
The stage 2 is aimed at obtaining the ''pre-qualification'' status of all the accepted entities in stage 1. This prequalification will allow them to participate in open innovation activities (stage 3). The pre-qualification procedures could imply three different types of activities: • Training processes. It includes information about the qualification process, documents to present, details on experiences with firm's technologies, presentation of some procedures to be used compulsory by partners, etc.
• Visits to partner's premises and facilities. The intention is to get a better idea about capacities, working procedures, cooperation culture, etc. This activity could be substituted by VCs if costs or time make it difficult to implement, however face-to-face visits are strongly suggested.
• Formal assessment of the entity as a qualified partner based on documents presented by the partner, and the decision about its adequacy. Usually, this qualification process affects all units of the entity and not only to individual researchers or project teams. Nevertheless, from our point of view, it should be relevant to distinguish between three different levels of qualification: individual, research group, entity. VOLUME 7, 2019 • Individual: Based in researcher's performance and skills; usually a given academic level as a tenure position could be enough; in other cases, specific personal accreditations are needed (e.g., personal NATO qualification for security reasons).
• Research group: Based on the demonstrated experience and performance of the research group (e.g., research group accreditation).
• Entity: Based on the management capabilities and the acceptance of confidentiality, IP conditions and legal responsibilities (e.g., administrative procedures to contract with external multinational entities, or type of facilities).  Table 2 summarizes the main issues related to each of these levels. Notice that the qualification of one (large) entity does not imply that all its research groups or employees are qualified too. Even at the entity scope, in case of universities, the qualification of one ''school'' or ''faculty'' or ''department'' does not imply the qualification of the whole university because differences in knowledge and experience to cooperate in OI could be very high.
The recognition of the pre-qualification status at the research group or entity level could be granted to one external partner for some time (e.g., for two or three years) but it will be necessary to extend the renewal of qualifications in the future because the context and capacities of the partners could change (and the key personnel involved too). In some cases, a short formal ''re-assessment process'' should be developed-it is a common practice in many academic processes like education degrees accreditation where some external (public) entities perform this reassessment task.
For product/service development the renewal of the ''qualification stamp'' is like a ''quality control assurance'' process: it is based on the analysis of the management procedures used to conduct the activity. For research cooperation, an analysis of the research performance linked to key personnel knowledge and attitude is crucial also.
The stage 3 is aimed at facilitating the qualified partners with access to challenge information (contained in a database) and then, to be able to submit a proposal within the set deadline. In this case, the access to that information and the will to participate can require the signature of specific NDAs because the firm could be interested in sending requests for strategic/confidential challenges; in other cases, challenges are very generic, and NDAs aren't necessary (they could even be counterproductive because they could reduce the number of possible proposers).
As the number of pre-qualified partners participating in a ''challenge competition'' becomes smaller (only accessible for qualified partners), the OI leading firm can devote some specialized human resources to interact with proposers and to provide them some feedback on initial drafts of proposals to guide the refinement process. In the end, the intention is to receive the best possible well-focused proposals to the open challenge. Reducing misunderstandings in the interpretation of the challenge by potential proposers is a key factor for success.
Detailed management procedures at stage 3 will require the leading firm to answer some open questions: • Should all challenge-based projects be framed in open calls, or is it better to negotiate bilaterally in some cases? In what cases?
• Are all partners competing for one challenge problem statement? Alternatively, is the OI leading firm ''inviting'' only to a reduced subset of partners within the community?
• When should a partner abandon the community? What should the procedure be to withdrawal one specific prequalified partner?
• Are invitations to participate (only for pre-qualified partners) known by the rest of the partners?
• Should partners sign specific NDAs or is the signature of generic ones enogh? Are NDAs signed at the individual or entity levels? There isn't a single answer to these questions and firms implementing open innovation schemes should define them according to their strategic interests and cultural roots. In some large organizations, implementation rules can change from one unit to another.
In the case of sophisticated OI networks where some external partners could act as ''intermediate hubs'' to coordinate a subnetwork, it is feasible to use the same approach for the partner acting as hub.

A. METRICS OF LOGICAL DISTANCE AS A PROXY FOR SUCCESS 1) INFORMATION AVAILABLE FOR PARTNER SELECTION
As mentioned in the previous section, partner selection became a key issue for success in the implementation of OI activities. Then, to extend the scheme over the world, companies should provide potential partners with some information on the technical challenge on which to react and, in parallel, to obtain enough information on the partner's performance, readiness and organization from direct and indirect sources.
In practice, there is always a ''gap of adequacy'' between the leading firm and one external partner to address one specific technical challenge. Usually, OI implementation is an asymmetric situation where the firm will have much more information than the partner with respect to the open activity to address, and that situation tends to be preserved during the cooperation process. The rationale for this behavior comes from the need of preserving sensible information in-house and to reduce the well-known risk of information leakage found and reported in open innovation schemes [32].
Thus, firms opening R&D activities prefer to assess the adequacy of the partner regarding its readiness to be involved in one open activity in two steps: 1) Closeness to the firm. Use of public information on the potential partner (obtained without a bilateral formal contact to get it). The leading firm will interpret results in their own context without sharing information with external partners. 2) Technical adequacy to the challenge. Readiness to address one activity by transferring the minimum information to accomplish the task to partners identified in phase 1 (if necessary, by signing specific NDAs to share information). In other terms, instead of selecting all partners in a single step, some filtering is performed to reduce the number of the most promising partners and to focus the effort on them. Nevertheless, the selection process is not finished and to move forward for a final decision, specific quantitative techniques are needed as presented in the following section.

2) METRICS FOR PARTNER SELECTION
The task of selecting the best partners (i.e., to choose the most adequate external entity to find a solution of a given technical challenge open to the community of external partners) with previous knowledge and experience ''closer'' to the stated technical problem, and with the right set of skills (event the cultural ones) is conceptually defined as a framework for guiding the selection process.
To support this process, the concept of ''distance-based metric'' between the firm leading the OI and the potential partner is proposed in this document as a semi-quantitative technique to test the adequacy of a partner for a specific type of open R&D activity.
The selection process is performed by measuring at what extent some target entities are ready to participate in the open technical activity proposed by the firm. The selection approach postulates that the adequate partner should be the entity with the smallest ''distance'' to the OI leading firm for one specific challenge.
''Logical distance'' is informally defined as the ''solution gap'' existing between the proposed technical challenge and the technical knowledge and abilities of the targeted partner to solve the challenge on time and within budgetary constraints. Technical aspects are completed with factors in relation to the closeness to the firm. Conceptually, there is always a ''distance'' between the firm's defined challenge to solve and the potential partner providing a solution. The shorter the distance, the more adequate the partner is.  The computed value of the distance between the firm and one partner is not applicable for all types of open activities: it should refer to one specific technical challenge (external partners and relative distances could be different for other challenges). Based on the experience, the distance between a firm and a partner to solve one specific challenge is interpreted in this document as a variable depending on several factors. The factors related to the closeness to the firm (step 1) are the following: • (G) General level of technical knowledge of the partner on the interests or activities developed by the firm. It includes the technologies that are well known by the partner related to the activities or interests of the firm. Value assignment: Maximum value (5) when the technical knowledge of the partner with respect to the interests or activities developed by the firm is low or very low; Minimum value (1) when the partner's knowledge fully covers the interests or activities developed by the firm.
• (R) Relationship with the partner, based on previous experiences by the partner with the firm and the Hub (an organization that acts as intermediary between the firm and the partner) and cultural diversity. The intention with respect to the cultural diversity is to capture at what extent the diversity in working procedures, values for cooperation, etc. between the firm and the partner could condition the success. Cultural diversity is very relevant when partners could come from emerging countries where it isn't possible to assume that cooperation and working values are fully shared. It is also higher in cases of public entities with respect to private VOLUME 7, 2019 ones (e.g., SMEs) when they participate in open innovation activities. Value assignment: Minimum value (5) when the relationship is practically nil or ill-formed and cultural diversity is not relevant at all; Maximum value (1) when the relationship is very close and cultural diversity is key. Table 3 describes the values of P1 to P4 based on the above identified factors for the closeness to the firm.  available resources put on the table by the partner, and it also depends on the available budget and facilities. Value assignment: Since this value is expressed in terms of percentages of ''improvement'' with respect to deadline, 1 (minimum value) means that the development time corresponds to the deadline (or even lower), 2 between 100-90%, 3 between 90-80%, 4 between 80-70% and 5 (maximum value) less than 70% (it is far from the firm's expected time).
• (B) Budgetary resources allocated to the partner by the firm (both in kind and in cash) and the difference with respect to the budget requested by the partner to develop the proposed solution. To simplify the discussion, we assume that some maximum budget to address the challenge is known by potential partners-Indeed, one firm's strategy is to hide this figure (keep it unknown for potential partners, at least, at the beginning of the negotiation process). Value assignment: In the same way that the factor time/deadline, due this value, is expressed in terms of percentages of ''improvement'' with respect to the maximum budget, 1 (minimum value) means that the request budget corresponds to the maximum allowed budget (or even lower), 2 between 100-90%, 3 between 90-80%, 4 between 80-70% and 5 (maximum value) less than 70% (it is well above the estimated budget).
• (E) Engagement with stakeholder of the possible partner with respect to the target of the challenge. This factor tries to capture at what extent the engagement with the stakeholder is important to have success of the challenge. To specify, this factor will be based not only on the knowledge of the target (geographical area, people, regulation, etc.), but also on contacts with public entities, e.g. local government, and private organizations (providers, support, etc.). Value assignment: Maximum value (5) when the engagement with stakeholder is not relevant for the success of the project; Minimum value (1) when the engagement with stakeholder is key for the success of the project. Table 4 describes the framework to define the relative distances of partners P1 to P4 based on the above identified main factors for the identified challenge.
The values allocated to the first two columns (specific level of technical knowledge and the challenge-solution complexity) are arbitrarily measured in the range 1-5 for minimum (1) to maximum (5) levels. More precise values are difficult to allocate in semi-quantitative approaches and users should follow specific guidelines based on the experience to allocate them.
For specific challenges and context conditions, the challenge solution complexity is similar for all partners when the OI leading firm publicly makes one technical challenge for intended proposers, but the ''perceived complexity'' could vary depending on the proposed solution elaborated by individual partners and execution contexts. The next two columns (time vs deadline and resources requested vs maximum budget) reflect how the proposal elaborated and submitted by one invited partner offers some advantages with respect to other proposers. The OI leading firm can indicate the estimated deadline or the maximum level of resources (e.g., maximum budget) for the action. Partners should prepare and answer based on their capacities and possibilities to meet budget and deadline-Companies prefer not to publish these constraints to give additional freedom to proposers. If these aspects are unknown, the risk of wasting management time is very high.
The fifth column refers to the ''engagement with stakeholder'' of the chosen partner. The weight of the column indicates the relevance of this aspect in the evaluation of the distance.
Finally, columns are weighted. Weights are provided by the OI leading firm considering the relative importance of each of the factors (Specific Level of technical knowledge, Budgetary resources, etc.) for the firm's decision; the sum of the weights of all factors must be equal to 100%. They reflect the relevance of the factor for that specific challenge; other values could be set for other challenges or in another moment depending on internal or external pressures and accumulated experience. After weighting columns, a simple sum provides the final value of the distance.
Then, the logical distance between one firm and one partner for one specific technical challenge is expressed as: di (firm, challenge, Pi)= Square root ((''Closeness to the firm factors'' x Weights) 2 +(''Technical adequacy to the challenge'' x Weights) 2 In Figs. 4 and Table 5 are depicted an example of the application of this method to six partners.
This method provides a semi-quantitative approach that reduces the arbitrary selection of one partner for one specific open activity (e.g., based on acquaintance between the firm's  responsible and key researchers in the partner entity as it happens in usual cases).
The weakness of the approach comes from the fact that it relies on self-assessment by partners and leading firm and subjective figures in columns that can introduce a bias and require subtle analysis, which is not easy to do. As it happens in other semi-quantitative methods, valuation improves if the process in the firm is carried out by a group of people with different profiles. A panel of diverse demographic evaluators that have no inter-discussion or coordination to arrive to their individual conclusions can diversify and increase the VOLUME 7, 2019 TABLE 5. Detail of the evaluation data for the example of ''distance'' between the leading firm and external partners with respect to a given challenge. efficiency of the valuation of the firm and how valuable of a partner they are. Too few or narrow evaluations can stove pipe or incorporate non-objective evaluations. Below is the calculation of the ''Technical adequacy to the challenge'' for partner 1 of Table 5, for better understanding: Furthermore, the value obtained for the distance is only valid for one specific challenge; however, extrapolation to other types of technical challenges involving the same partner will dramatically modify the logical distances (except for very similar technical challenges). Computation of logical distances should be done rapidly (even by automating part of the process) to reduce the time needed to decide.
The metric of ''distance'' proposed and described in this section relies on a previous and necessary work to be conducted by the firm: the construction and characterization of a limited number of potential partners (from geographical, size or legal criteria). In these cases, it is necessary to find a good balance between simplicity and the capture of relevant issues to make decisions on a filtered group of potential partners.
Another group of complementary internal factors to select one specific partner (not linked to the concept of distance) refers to the trade-off between the cost of partners' selection and the time necessary to start a new open activity. These are two internal factors in the firm, which can constraint the selection process in specific cases. Usually, private entities can make decisions (e.g., to sign and start an activity) faster than public ones We assume that the cost of partner's selection will depend on the type of partner (less expensive would be for active partners because they were previously selected indeed). The second factor, the time necessary to start the activity, could be very relevant in the case of time pressure to start the development of one open R&D project with external partners.
To visualize this complementary perspective, Fig. 5 shows three zones in relation to the concept of logical distance defined above in this document. Zone A corresponds to active partners with time to start below some threshold and a short distance. Zone B corresponds to active or dormant partners with time below the average and medium distance. To simplify and speed-up the selection process, the qualification processes of entities to be accepted as partners was historically applied in many firms and sectors; the usefulness and applicability of pre-qualification techniques will be discussed in the next section.

B. BUILDING A COMMUNITY OF SOLUTIONS PROVIDERS TO TECHNICAL CHALLENGES 1) CONCEPT OF OPEN COMMUNITY
The set of pre-qualified partners represented in Fig. 4 are, at the beginning, totally independent. They have the prequalification recognition through independent processes and, probably, in different periods, contacts. Exchange of information and working knowledge between these partners is not evident.
Even more, when they participate in proposing a solution to an open challenge, it is probable that they are unaware that they are elaborating a proposal (competing) to provide a solution to the same challenge in parallel with other prequalified partners. Only the leading firm launching the technical challenge could know the identity of all partners invited to present a proposal and, who the partners submitting a proposal are.
The positive consequence of this ''anonymity'' feature is to keep the selection of one specific entity to provide a solution as a pure ''competitive process'' (at least in the first phase of stage 3) where each pre-qualified partner decides to prepare a proposal or not within the framework and conditions setup by the firm. In this method, innovative proposals are encouraged.
There are negative consequences as well. One of these consequences is the difficulty to offer all pre-qualified partners the ''sense of community'', because bilateral interaction between the firm and each external partner dominates the scene. Thus, cooperation between partners is very limited (not explicitly promoted or even forbidden) and the construction of a rich OI ecosystem is not supported. What occurs is that once the open innovation approach is implemented, it can become a set of independent bilateral R&D projects. From a topological perspective, it resembles a star network and not a meshed one.
Nevertheless, there are some other possibilities where cooperation between partners is encouraged. Once one partner has been qualified, it belongs to a ''community'' with some rights to receive information, to react to open challenges or to participate in some events. Why not to use it as an instrument to promote partners' interaction?
A ''community'' visible for members (noted that it is only for pre-qualified partners) could increase the potentiality to address more complex challenges where individual knowledge is not enough and cooperation between partners is a condition to elaborate better proposals. In the end, this is the essence of any OI movement. In those cases, the proposal to answer a challenge is elaborated by two or more partners even if one of them acts as coordinator of the ''consortium''specific conditions for eligible consortia can satisfy other policy goals. In the EU Framework Programmes for Research and Innovation (e.g., the current H2020) consortia should be composed by a minimum of three entities from two EU Member States. The intended goal is contributing to the creation of the European Research Area with stronger links between entities located in several countries. Usually, firms, universities, research centers and administrations could be members of the consortium.
Due to the wide range of products/services offered by large firms over the world, the corporations can be naturally derived of a network of open communities focused on specific domains. The next section will address the creation, support and evolution of these open innovation communities over the world.

2) CREATION, SUPPORT AND EVOLUTION OF OPEN COMMUNITIES
By using the procedures described in the previous section, companies implementing open innovation models have qualified a set of international partners in each stage and, in principle, all of them belong to the same open community. In addition, learning from experiences will contribute to the diversity of approaches, which could be aligned to the objectives of the ecosystem, thus creating opportunities for innovation, networking and visibility [33].
In practice, this is not necessarily true unless a set of activities were actively defined and carried out to create and support that open community where both the firm and its qualified partners benefit from the activities.
The intention in this section is to discuss some suggestions to establish a very active and rich community of prequalified partners as well as to propose specific management procedures to ensure a win-win scenario for firms and external partners.
First, the firm should decide if it is better to create a single community of qualified partners for the whole range of open activities in all domains of interests or, on the contrary, if it is better to distribute it in several sub-communities focused on specific technical domains. Pros and cons exist in both cases.
In the first case of a single community, multi-disciplinary approaches will be easier to implement although not all of them will be related to specific technical challenges. In the end, every partner in the community is free to analyze the proposed challenge and to decide by itself if it deserves or not the effort to prepare a successful proposal. Notice that this approach is exactly the method followed by public administrations when they publish an open call for research proposals where all eligible entities can participate.
In the second case of multiple communities, management becomes more complex because borders between domains are blurred and one partner could belong to several communities; then, the identification of communities as isolated repositories has some artificial nature.
For the rest of the section we assume the existence of a single OI community where all qualified partners belong. In addition, we focus on academic partners (no other private companies or administrative bodies).

VOLUME 7, 2019
The following is a list of open-ended questions that are necessary to address in order to create stable networks with committed communities of pre-qualified partners: • What are the generic and specific rights and obligations for the members of the community? Is there a margin of flexibility depending on the type of activities in the network? Community partners expect to obtain some rights after qualifying and they are ready to accept some obligations too. Nevertheless, both rights and obligations should be transparent for the members to avoid hyper or infra expectations. Then, the list of rights and obligations should be selected and implemented, keeping in mind the need to increase trust and future commitment. As an example, the initial list of rights could be based on: The right to be invited periodically to present a proposal (even when there are no rights to be funded!) The right to receive early information about future research lines of interest to facilitate its alignment.
The right to participate in specific events to facilitate contacts with key people in the firm and other members of the community. The right to present their research activities/ results in public events. The right to publish results (with the conditions set-up by the firm). In the same way, some possible obligations for a community member could be: The obligation to participate (by invitation) in some corporate events (usually, one per year organized in parallel to other type of technical events). The obligation to periodically inform on advances in various S&T domains (e.g., by uploading relevant papers, doctoral theses, grants for other research projects, patents filed, etc. in a common community repository-These obligations only affect nonconfidential information of the partner). The obligation to submit at least a proposal a year to one of the open technical challenges issued by the firm. This period could be even shorter (e.g. one semester). The obligation of fulfilling the confidentiality clauses in the general partnership agreement, and necessary updates when other people are involved in activities.
• Is it feasible to specialize various groups of entities for specific technologies or markets or, on the contrary, is the emphasis on integration between all entities a key element to maintain and promote? This question is closely related to the single-multiple community discussion above but interpreted in a more flexible way. In fact, the very broad range of technical themes for possible open challenges for cooperation makes it impossible for a single partner to address all of them. Usually, only a subset would be relevant to prepare a proposal. To organize partners on categories by potential areas of interest stated, the members themselves can facilitate the preparation of messages and exchange of information with the maximum relevance. The negative effect of receiving invitations to participate in areas outside of the background and experience of a community member is another factor of frustration. The use of AI algorithms based on information provided by the member (or captured from public sources) to be prequalified as well as the behavior as a member of the community can help in preparing automated messages to the right destinations. This process can aid in the development of a multilevel structure of partners where several partners (acting as ''hubs'') can control or coordinate other local (or remote) nodes with a minimum interaction of the leading firm-The development of the three pilot projects in LATAM coordinated by the UPM is an example of this multi-level structure.

3) PROPOSED MANAGEMENT PROCEDURES FOR SELECTION OF PARTNERS
We postulate that a moderate version of the proposed procedures can be applied within a technology firm division on the following phases: • Phase 1. To build up a data base of potential partners with the use of ''distance'' to leading firm as a useful metric for partners' classification approach. It is convenient to track record of the dynamic changes in the table as new partners are identified and others could change their position over time.
One table can be built based on ''type of similar challenges''. Technical challenges sharing the same type of technologies and applied in the same domain (e.g., assuming not more than a couple of dozen ''types of technical challenges''. This information should be extracted from the (confidential) list of potential projects, which is not available for partners). Specific partners could appear on several tables if they have various experience and multiple capabilities to address several types of challenges. This is the case when the finite method chosen to identify the partner is the whole entity (i.e., the university) because new teams could progressively participate. Distances with respect to the leading firm can be computed for each active and dormant partner based on the collected information, the position in the table and external constraints.
Some non-active partners could be analyzed but it requires a longer process with some visits and face-to-face meetings.
• Phase 2. The set-up of assessment processes of preselected partners based on visits and small pilot projects to check cultural biases, reactions, and performance.
Visit(s) to the selected partner as an essential procedure to obtain tacit knowledge about the partner and its capabilities and technical facilities. Exploratory visits should be well-prepared to have time to visit facilities but, above all, to meet key people which could be potentially involved in future open innovation projects. Pilot projects (running on several months and reduced budget) constitutes the easiest way to compensate for the lack of direct knowledge of the partner's capability.
The obvious difficulty is the effort needed and the time necessary to set-up an explicit relationship which sometimes could delay the launching of the actual challenge-based project. One possibility is to delegate that responsibility in another per-existent active partner (the launching process of three pilot projects in LATAM is an example of this type of ''delegation'').
• Phase 3. To define a clear and transparent system to invite partners to propose solutions to specific open technical challenges. The intention is to increase the likelihood that an external (invited) partner be attracted by the published challenge, and then, it decides to devote efforts and money to prepare a proposal, even when there is not 100% guarantee of getting funded.
The number of partners invited to participate in one specific open challenge should be controlled to avoid the oversubscription problem mentioned above and to reduce the volume of effort allocated to exchange information, as well as to evaluate the proposals during the period where the challenge is open. Specific visits to some partners in relation to complex technical challenges could be necessary to clarify proposals. We understand this issue is easier to implement with partners located in the same country but more difficult to do in countries located elsewhere. The use of VCs could overcome some difficulties for interacting with remote partners.
• Phase 4. To select the most adequate partner for the open challenge and to set-up the contract between the partner and technology firm division. Based on the most adequate proposal, and the previous qualification of the partner, the firm selects one partner. In some cases when two proposals address a key challenge from different technical approaches, two proposals could be selected (if allocated funding allows it) in order to evaluate the best approach.
A specific contract is signed between the technology firm division and the chosen partner. The template of the contract model to use should be known by partners from the assessment phase in order to accelerate the signature. A kick-off meeting is scheduled to agree on the development of the project and to establish the necessary control via deliverables and periodic reporting.

4) MANAGEMENT OF CULTURAL COMPLEXITY
If the creation of a corporate culture is a problem for a given multinational firm with teams distributed over the world, the case of OI introduces an extra cultural complexity because partners will have their own cultural roots and they will not change them because of the agreements of a few research projects with a large firm. This issue has clear relevance in the case of academic partners because the cultural framework in a university will impose a set of constraints in the negotiation and execution of projects. These constraints should be understood and accepted if long-term partnerships are highly valued.
We postulate the need to embed the OI procedures as depicted in Fig. 6 in a corporate cultural framework interacting with cultural framework of (academic) partners. From the firm's perspective the scheme suggests that units involved in OI activities should derive a specific culture focused on the cooperation with academic partners. Fig. 7 also suggests that cultural gaps could change from one partner to another based on their institutional culture and the influences on national cultural values (more relevant when cooperation will be implemented with public partners located over the world because the influence of administrations is higher).

IV. RESULTS
The objective of the project ''international open networks and models'' was to test the way that open innovation could be implemented in networks at several levels. To test it, a simple model was used during the development of this project with three universities located respectively in Peru, Mexico and Chile as depicted in Fig. 8.
Three pilot OI projects were defined and implemented with universities located in Mexico, Chile, and Peru respectively, in order to test the adequacy of the OI model discussed between the Contracting technology firm and the UPM.
In this case, the responsibility of the selection of the participating universities in those countries relied on the UPM knowledge and procedures and not in any management structures supported by the technology firm (even if firm's legal representative had the final responsibility of signing up the contract with the UPM to launch pilots with LATAM universities and transfer the funds according to the invoices received by those entities once the technical deliverables had been accepted by the UPM).  One striking difference in this case with respect to the general model of open innovation is that the participation in any of the pilot research projects did not imply the need to obtain any pre-qualification recognition for those LATAM universities (It implied the ability to establish some form of Sponsored Research Agreement or a Sponsored Academic Research Agreement) to participate directly in a challengebased competition. Those LATAM universities could sign the agreements if both parties agree on it, but this possibility was not part of the pilot projects' outcomes.). Nevertheless, the global context remains similar with an added complexity because three entities intervene (technology firm, UPM and the LATAM entity) to the conventional case between a leading firm and a potential partner. Fig. 9 expresses the context of selection of LATAM partners in this case-The number of potential partners (universities) located in the three selected countries (Mexico, Chile and Peru) was very large with a large disparity in sizes, research capabilities, legal structures, etc. as international rankings show, and it was not possible to interact (and even to check off of them in a very short period of time). The schedule of the project gave only three months to identify, connect, interact, convince, propose and sign the participation in the pilot projects.  Time constraints to select and sign (three months), and the UPM knowledge that pilot projects, even when they were not defined yet, were focused on a specific technical domain (linked to main research projects under development), motivated the decision to focus on technical universities with demonstrated R&D experience and capabilities in each of the countries. This decision dramatically reduced the number of possible candidates.
As an average, these pilot projects were running for fourfive months. Obviously, technical challenges were limited to be addressed in a few months for a small group of people.
With all pre-selected LATAM universities (initial list obtained from the UPM contacts in the region), four types of activities were conducted: LATAM universities in each country in a proximitybased matrix by using public information collected from databases and analysis of public web sites on each university. The intention was to obtain information from public sources on their capabilities to address the proposed technical challenges and types of cooperation with multinational firms. In a second phase, the initial matrix was refined to incorporate further details of some ''research groups'' where the right behavior and attitude of individual researchers (as local selectees) were essential to start detailed negotiations.
• Analysis of distances and elaboration of the OI profile. This activity required the interaction with all of the distances of partners to know additional details on cultural biases and capabilities to participate in the pilot projects. The intention was to ensure that they are prepared to enter in highly demanding open innovation activities. In the addressed cases, there was no previous experience of R&D cooperation and substantial effort was needed to explain the context and meta-objectives of the pilot OI projects apart from the obvious technical activity to perform.
• Adequacy to technical challenges. This phase was strongly dependent on the technical specification of pilot projects to be sure that the selected university had experience in the field, and they had the required facilities.Difficulties came from time and budget constraints, which made it difficult to accept the technical challenge unless the research groups were ready to start the activity in very short periods of time (it was substantially easier to accept with for studies than for technology-trials). Fig. 10 depicts the result of applica-tion of partner's selection tool to the twelve potential partners pre-selected from Chile, Mexico and Peru, in order to assess the main factors: closeness to the firm and technical adequacy to the challenge.
• Negotiation of pilot projects. Even when the selected LATAM university satisfied the rest of the factors, they needed to accept the conditions set-up by the UPM (based on previous negotiation with the technology firm) to sign-up the research contracts (e.g., decision on budget, deadlines for deliverables, IPRs, confidentiality rules, quality control for payments, etc.). In this negotiation phase, the UPM faced a binary challenge: 1) to deal with the limited margin for negotiation in time and budget accepted by technology firm simultaneously, and 2) the need to act as a last resource for the negotiation, shielding the technology firm division from direct interaction with LATAM partners.

V. DISCUSSION
This article has contributed to the implementation of OI models and networks at an international level by focusing on one specific issue: the selection of external partners as a key element to increase success rate. Conceptual analysis, metrics definition and support methodology to identify right partners were applied to various pilot initiatives where a primary node of OI network (the UPM) was able to create a secondary network in other countries (Mexico, Peru and Chile) to scale up the use of OI by a larger multinational firm.
From the experience running the three pilot projects, six main lessons on OI implementation were learned, and are as follows: • Lesson 1. The creation of the adequate framework between the leading multinational firm launching OI R&D activities and the primary node entity to expand open innovation networks at international level requires substantial effort, time and a high level of mutual trust.
Here, the UPM and the technology firm discussed (internally and amongst them) finding the right general framework to cooperate with external partners in multilevel OI networks where the UPM had the freedom to select the right partners. This effort was addressed before selecting and contacting with any potential LATAM universities to agree on contract models, indirect payments' models, liabilities (of the UPM with respect to the technology firm and LATAM partners with respect to the UPM), monitoring and reporting schemes, IPRs, confidentiality issues, etc.
• Lesson 2. The identification of the right individual interface is a key factor for success. It requires identification of an internal empowered person (the open innovation leader) to remove institutional obstacles and to accelerate agreements from the technical and managerial perspectives. As it is clearly stated in multiple cases of OI implementation, if leaders were well-known (e.g., they had previous cooperation and scientific mutual recognition), time and effort is substantially reduced. In the pilot cases described in this article, the UPM explored a more general case where there was no previous knowledge of the UPM with the working group leader in the LATAM universities. Then, mutual confidence building was embedded in the process after a blind pre-selected process based on data. A relevant issue in this case was the visibility of the technology firm in the selection process. Risks are to be considered included in UPM as the firm's proxy and not a fully empowered entity to negotiate. The UPM acted in this way (it was the single interface in the negotiation process) but motivation was higher if LATAM partners knew that the specific technology firm was behind the implementation and that they were part of controlled experiments to implement OI models at international level.
• Lesson 3. The selection of an entity as an intermediate node implies a double qualification process: 1) to be able to conduct some specific open R&D activities bilaterally with the OI leading firm and, 2) to control the execution of projects to other entities in the next level. These two qualification processes cannot be simultaneous. The second one requires a good performance in the first one and high credibility to delegate responsibilities. The case of the UPM coordinating pilot projects in three countries was initiated two years after signing agreements and with the experience of running three projects at the UPM with the technology firm.
• Lesson 4.Extrapolation of proximity with respect to one specific research group to the whole entity is not immediate. Mutual trust and confidence are firmly anchored in personal acquaintances between two entities' research groups, but it does not necessarily apply to other research groups in institutions with considerable autonomy degree for research and innovation cooperation. This lesson is particularly evident when public academic institutions (like universities) are involved. In these cases, the academic freedom to negotiate and obtain agreements with external entities for specific R&D projects becomes a personal decision of the groups' leader (framed in some general rules defined by legislation and institutional bylaws).
To extrapolate it to other group's leaders by assuming that the second occurance both effort and time to launch a new open activity will be dramatically reduced is not necessarily true.
• Lesson 5. Benefits of the proposed proximity-based metrics approach would mature over time if abundant data were collected during OI experiences. Thus, it is necessary to embed a set of pilot projects to monitor management procedures as part of the OI methodology definition effort. To address this issue, the monitoring effort carried out by the UPM was decomposed of two components: 1) technical monitoring to ensure the smooth development of the contracted tasks and 2) OI monitoring to extract enough lessons to share with the technology firm's research division. This second component had as primary goal to extrapolate the approach used to other LATAM partners in the same countries, in other LATAM countries and in other geographies where international OI networks were needed.
Firstly, some limitations of our study should be noted. The restriction regarding to a specific region, which is LATAM, might not would replicate in different regions (e.g., Sub-Saharan Africa, Southeast Asian). Second, cultural variables should be considered, since they could affect the performance of a project. In addition, this must be recognized with regard to emphasizing with a teamwork in a multicultural environment, and even, the different time zones can be a challenge to arrange meetings or videoconferences to guarantee the flow of the project. Third, not knowing the partners directly could imply a certain loss of information regarding aspects that could lead to a reduction in commitment, or a certain ''distance'' with respect to the project's objectives.

VI. CONCLUSIONS AND FUTURE WORK
The implementation of scalable OI R&D models in a technology-based firm implies efficient and smooth cooperation with external partners to address some open technical challenges proposed by the industrial firm. To be successful, the identification of the right international partners with whom to cooperate and to define a set of criteria to decide what are the best R&D activities to conduct in open innovation scenarios became key issues to increase the efficiency in the implementation of OI models.
All the proposed instruments and management procedures (and many others emerging from a more detailed analysis of individual entities' cases) should be focused to increase the attractiveness of the open community to become potential external (academic) partners. Furthermore, the intermediator role of UPM in this case study, created an efficient point of contact facilitating a hub to enhance nodes between the innovation seekers and providers to manage the dissemination of OI for the achievement of common objectives on both sides [34].
Successful implementation of open innovation models for a given firm was described in terms of efficiency (in management effort and time to launch an activity) and scalability (to expand the open community of partners to cope with wider range of possible challenges internationally).
The three pilot experiences described in previous sections run in LATAM are, by nature, limited in scope, geography and time to derive full scalability analysis for the whole range of potential OI activities. Nevertheless, the usefulness of proximity metrics was made clear to guide the pre-selection process where only a limited number of universities were identified in each of the three LATAM countries.
Further pilot cases with other partners and countries will be necessary to have a sound partners' selection methodology based on semi-quantitative proximity-based metrics. More specifically, the use of the approach for selecting start-ups seems feasible by adapting some of the elements to small groups of entrepreneurs.
Furthermore, one of the key selection factors for the implementation of one specific OI model for research activities refers to the capability to apply the chosen model to a growing number of external partners and R&D projects over the world by creating complex multi-level OI cooperation networks in pre-defined areas of interest. This goal is termed as scalability of OI models. The intended goal by using scalable OI models is to optimize the allocation of management resources (i.e. highly skilled human resources, set-up ad hoc teams and allocation of management time and cost) to ensure the appropriateness of the selected open innovation model after dramatically increasing the number of open activities running simultaneously with the same or different external partners. This will be another of our future lines of work as a continuation of this paper.
In Future works other approaches (e.g. Analytic Hierarchy Process) can be used even in more complex decision-making processes, to balance the additional efforts concerning the number of criteria and alternatives to take into consideration. Whereas that these criteria should not require additional management efforts with a view of scaling these approaches to other business models.