A Transparency Maturity Model for Government Software Tenders

Over the last two decades, governments have increased their investment in information technology to improve the use of public resources, using public electronic procurement systems to obtain better prices, better solutions and to show transparency in the procurement process. Public procurement of software development projects is specific acquisitions having specific technical, methodological, and management constraints that make transparency an elusive target. This article proposes a maturity model as a tool to measure tendering transparency when government agencies procure software development. We have used a procedural model to support the design of maturity models along four dimensions: Institutionalization, Software procurement process, Communication, and Accountability. We have defined a five-step model, and we have tested it with real government buyers. The model is supported by an appraisal tool that helps to guide the next steps in the transparency of software acquisitions.


I. INTRODUCTION
Transparency is a concept that emerges in the 90's and is established mainly in opposition to corruption. It is recognized as a term hosting several ideas behind the visibility metaphor, such as honest decision-making processes, proper behaviors of officials, public disclosure, integrity, accountability, and even democratic values [1]. However, in spite of all these features being possible values of social systems, it is recognized that transparency is today a computer-mediated goal whatever be its analysis perspectives [2].
In the last two decades, governments have made significant advances in the use of information technology in electronic government (e-government) to improve the quality of the public services and increase the transparency of government processes [3], [4], which implies to improve citizens' perception and trust in government. In various countries, these initiatives began with laws for the public sector on The associate editor coordinating the review of this manuscript and approving it for publication was Khursheed Aurangzeb .
Information Technology as an enabler, support, and streamlining mechanism [4].
Due to public resources involved in governmental purchasing of goods or services, governments endeavor to use efficient and effective electronic procurement systems. In some countries, all public sector organizations manage their public services tenders through procurement web portals in order to show a transparent procurement process [5], [6].
In order to increase the efficiency and effectiveness of purchase processes, governments need to face several challenges, for example: fully integrating technology and developing e-government systems [7], defining guidelines and policies to secure widely available high-quality public services; promoting legal frameworks; promoting acceptable management practices; and creating an adequate institutional structure while training operators and public officials [8].
In particular, public procurement controlled by e-government systems has a crucial role in national governmental programs. The procurement controlled purpose is to secure active transparency and efficiently improve relations with suppliers [5]. Therefore, these procurement systems should reduce corrupt practices, increasing confidence in the procurement system, increasing procurement productivity, and attracting new suppliers hoping to do business with the government [9].
Despite these expectations, corruption can be found in public sector procurement [10], [11]. Therefore, increasing the transparency of procurement processes, in any of their steps, is one of the main challenges faced by governments and a way to enable the citizens to monitor it. For that purpose, each government provides its policies and laws concerning how government procurement is published, and the indices by which it tries to measure transparency or corruption in its institutions and organizations [12], [13]. Some examples are the International Country Risk Guide index [14], Business International [15], the Global Competitiveness Report Index [16], and the Corruption Perceptions Index [17]. However, laws by themselves do not ensure genuinely transparent behavior [18].
This article proposes a maturity model as a tool to measure transparency in software development biddings. A maturity model is a map that guides the organization in the implementation of good practices, offering a starting point [19], [20]. It describes a path of evolutionary improvement, from inconsistent processes to the most mature of the organization [21]. It allows to evaluate the development status of an organization or business process, clearly outline improvement strategies to achieve the planned objectives and identify the areas where the organization should focus to improve [19]. The proposed maturity model aims at providing a framework that guarantees a transparent software acquisition process in public institutions. In this sense, it represents the first study designed for conferring transparency to software tenders via maturity models, to the best of our knowledge. Our main contribution to the state of the art is the adaptation of existing models to this purpose, combining a variety of approaches from the e-government and software development literature. Our research question is: how can a maturity model successfully confer transparency to the software acquisition process in public institutions?
Our work is structured as follows: In Section 2, we described Public procurement of information technology. In Section 3, we present the methodology used to create our proposal. In section 4, we explore the context of transparency and accountability as applied to the Information and Communications Technology which governments have made available to the public. In Section 5, we indicate the importance and uses of maturity models and a brief review of models related to our proposal. In Section 6, we describe the selection criteria for our proposal. In Section 7, we present maturity levels as a tool to measure transparency. In Section 8, we describe the criteria used to create the measuring tool, which we then tested. Finally, in Section 9, we present our conclusions and future work.

II. PUBLIC PROCUREMENT OF INFORMATION TECHNOLOGY
Public procurement of information technology is a frequent and widespread tender item; it includes software development projects that move large amounts of money between governments and the software industry [22].
Public tender processes associated with a software product usually include a set of practices related to Software Engineering, which have been explored mainly from Requirements Engineering. Indeed, Software Engineering addresses various problems and new challenges that include improving the quality of output procedures and Software Requirements Specifications (SRSs) [23], [24].
Next, we will explain the tender process. To do this, in Figure 1, we have represented the process by using BPMN notations. In this notation, the actors (roles) are represented by swimlanes, in this case Supplier and Customer. Empty circles represent the initial and possible ends of the procedure, clocks represent deadlines or time constraints, diamonds represent making decisions activities and rectangles represent sub-processes, finally, flows represent information messages and/or sequence controls. These multiple interactions are managed by electronic platforms. First, the customer or acquirer (usually a public administration) publishes the call for tenders and classifies it according to topics and areas (upload C4T document). All public calls for tenders managed through electronic platforms are by definition open, i.e., anyone may submit an offer. The platform supports a suppliers' database, which is also classified by areas and topics of expertise. The platform then sends an email to any supplier whose area of expertise matches that of the new offer (flow from customer to supplier). Furthermore, suppliers can perform a manual search of public calls for tenders at any time according to a set of predefined criteria (search bases for proposals). Once the supplier has decided to enter a public tender process, the acquirer prepares the proposal according to the call for tender document specifications (prepare offers). During this stage, some public tender processes allow suppliers to send questions to the customer, who may answer publicly, to all suppliers at once through the platform, or privately, via email. Also, the customer may organize a public interest manifestation meeting, where the VOLUME 9, 2021 customer and the tenderers can gather to further clarify the actual problem the project aims to solve. Once the public tendering process has been prepared, the tenderer submits its proposal through the platform (Post offer as bases). At a given date (clock), the customer opens all the tenders received and evaluates them according to the set of criteria that was published in the call for tenders (Opening of offers). The evaluation result is finally communicated to all the suppliers on an electronic platform. The customer must publish not only the selected supplier, but also a chart presenting the scores of every tender submitted (Proposal award).
This type of procurement suffers from the same lack of transparency, which is a contradiction to the fact that transparency is a vital aspect of software systems quality [25]- [27], but also a market symmetry indicator, and one of the values most appreciated by the public [28]. Therefore, transparency is a crucial indicator for potential providers that must choose what software projects to bid and for governments to demonstrate, to the citizens, that rational decisions take place under its command. Unfortunately, current transparency parameters cannot address all the situations encountered in software development projects, and there is no transparency benchmarking for software procurement processes. Moreover, transparency is widely recognized as an organizational-culture issue having a soft-side to observe as part of progressive improvements [29].
In Information Technology, socio-technical progress is assessed by Maturity models. Maturity models provide a systematic framework of reference for measuring organizational performance in specific areas, focusing on technological areas [30]. Besides, there is evidence that the soft-side of particular organizational issues requires particular maturity models [26], [27]. Therefore, measuring the maturity of the transparency in the public procurement process implies e-government agencies to increase transparency and to identify and disseminate best practices for electronic procurement [6].

III. METHODOLOGY
To develop the proposed maturity model, we have followed Becker's procedural methodology [31]. The selected method to create Maturity Models (MM) for Information Systems (IS) presents a process proposal to design and validate maturity models. It has the aim of avoiding the variability in the ways in which MM are motivated, produced, and evaluated. The methodology of [31] begins by establishing eight sub-phases for the development of maturity models that are grouped into two main phases: ''Generation'' and ''Transfer''.
The Generation phase has four sub-phases: the first, called ''Problem definition'', determines the scope of the domain of the maturity model, as well as the conditions for its application and the intended benefits.
The second sub-phase is ''Comparison of existing maturity models'', the aim of which is to analyze existing models in order to propose the new one. As [32] points out, a new model may also be an improved version of an already existing one.
The next sub-phase is ''Determination of a development strategy'', in which we suggest a design strategy that combines extracts from a set of maturity models into a new model. We recommend using the identified maturity models as a starting point for the design process.
The fourth sub-phase is ''Iterative maturity model development'', in which the maturity model is submitted to revisions. The steps of this sub-phase are: ''select design level'', ''select approach'', ''designing the model section'', and ''testing the results'', which should be iterated by each maturity level.
These four sub-phases constitute the first phase. The second and final phase is the transfer phase, which also consists of four sub-phases. It starts with academic publications and public web sites enabling companies to evaluate their degree of maturity, providing additional support for an empirical basis.
The interpretation of the general terms of the Becker methodology is a methodological design to achieve our objectives, for this, we have focused our proposal on following the procedures of the generation phase of the maturity model. Therefore, in order to generate our Transparency Maturity Model for Software Acquisitions, we have followed the corresponding sub-phases: i) Problem definition, ii) Comparison of existing maturity models, iii) Determination of development strategy and iv) Iterative maturity model development.
In Figure 2, we summarize the phases of this procedure as it has been proposed by [31], in addition to the original representation, we have added checks on reached sub-phases. The following sections detail the development inside of each of these sub-phases.

IV. SUB-PHASE I: PROBLEM DEFINITION
Transparency refers to information that has not necessarily been available to the public in the past. For a process to be transparent, information concerning it must be accessible, high quality, and reliable, complete, relevant, accurate, timely [33].
Transparency is regarded by governments as an essential mechanism in order to meet public expectations and their responsibility to society. It has been even claimed as a democratic right of every citizen and it has become essential for the exercise of good government [34]. However, it has not only been associated with the quality of a particular government but as a crucial target as economic growth is [35].
Transparency is also used by governments to publish certain information or to open up to the public certain decision-making processes [1].
A lack of transparency and accountability in the public sector tends to contribute to macroeconomic destabilization, distortions in the allocation of resources, and economic inequalities. These effects cannot be ignored [18]. Therefore, over the last decade governments have worked to increase their transparency, using Information and Communication Technology (ICT), which is expected to bring about a reduction in corruption [36].
Government procurement portals can reduce corruption, for example, by enabling people to compare prices [36], offers, types of bidder and delivery times of bids; to understand the time required to execute the awarded project and the payment methods proposed; and by permitting an interaction between bidders and customers through the inquiries resolved during each tender process.
This commitment by governments has generated enthusiasm and concern about the potential for ICT to create a substantial social and cultural change in attitudes towards transparency [37].
Many governments encourage the use of ICT to promote efficiency and transparency at the same time [38]. However, evidence suggests that incorporating technology is not enough to produce this change, as several other issues need to be addressed at a cultural level [37], [39]. For example, the public in some places still has a strong preference for personal or telephone contact when making inquiries or requesting an external service, which can be interpreted as an operational obstacle and even a lack of transparency in some cases [40]. Furthermore, socio-cultural factors determine that making information available is not enough to ensure transparency, as indicated by [41] and the World Bank report [42].
In the software industry, transparency is regarded as an aspect of quality [31], and an important feature of quality is accountability [43], which we describe below.

A. ACCOUNTABILITY
Accountability is a process attribute that concerns us when discussing transparency, as it is among those attributes required when governments prepare Software Product Tenders (SPT). Accountability is a characteristic of representative government [42]- [44], [45], [46], and includes control processes exercised by the public over their elected representatives through elections or vertical electoral accountability, and public participation mechanisms or vertical social accountability [44], [45]. It also includes forms of control which operate within public administration, i.e. within the bureaucracy that implements public policies (vertical internal accountability in the control of internal management), and between this bureaucracy and government suppliers (contractors, consultants, etc.), who are considered a specific interest group [46].
Accountability is usually perceived as reporting or accounting for how authority has been used. It is also understood in specialized literature as responsibility [47], [48], defining who is responsible to whom, who is reviewed, evaluated, sanctioned, and as a result liable and controllable. Thus, accountability as responsibility contains three successive levels [49], [50]. The first is the informative level (account: a description), in which whoever is being held accountable should report their performance and decisions. The second is the explanatory level (answerability or responsibility, account: an explanation), in which whoever is being held accountable must also justify their performance or decisions (explain what has been done and why) and compare it with previous commitments or performance standards established in advance. The third is the enforceable level, in which whoever is being held accountable is subject to proportionate sanctions, as a result of their performance and justifications compared to their commitments. In this way, their commitment and performance are made binding.
Accountability within a theoretical principal-agent relationship implies an obligation by the agent to the principal. In the relationship between bureaucracy and its suppliers, accountability is unidirectional, i.e. from the provider as an agent to the bureaucracy, who directs the agent within the framework of a contractual relationship. Duties are bidirectional as a bureaucracy needs to meet the necessary conditions to secure transparency and efficiency. We propose that it should be held accountable to its suppliers, by punctually fulfilling its payment commitments and clearly communicating its requirements, such as the goods required and their condition, which should be clearly set out in tender processes and subsequently in contractual documentation, and include budgetary conditions, deadlines, evaluation criteria, etc. [26], [27]. It must justify its requirements and be evaluated and sanctioned if it fails to comply with its commitments to its counterparty.
An example of the foregoing, within the context of preparing software development requirements by governments, is the proposal from [27] regarding the importance of transparency in the software development process. The authors argue that accountability creates auditability, which in turn creates transparency. Whoever requires software to be developed must be held accountable, as they must be able to explain their requirements clearly and intelligibly. The functionality required must be articulated in the software procurement process, which should identify how to achieve the targets, which processes cause certain events, which processes end at certain events, the software requirements for activities, and the information used during the process. In addition, they need to justify their initial requirements, the decisions taken during the process, the results, the sequence of activities, and the activities and players taking part in the process.
This ensures that the information required to design software is available, and results in the necessary transparency. Thus governments as principals identify and justify -or to use the terms established above, inform and explain -as part of their accountability processes. According to [27], this makes them auditable, since their counterparties, who are not only the software designers but also the public as citizens, and eventually as users, can examine them to verify the quality of their decisions, performance and compliance, and the attributes of the finished products made available to the public. Therefore, we can say that the process improves transparency because the required information is made available and thus the process that affects the counterparties is open to evaluation. This makes the third level of accountability viable, which is the enforceable level, provided we believe that public evaluation is a positive or negative sanction that can make governments responsible and hold them accountable in a comprehensive manner.
Accountability requires articulated activities relating to participation, openness, and frequent, symmetrical, proactive, synergistic and long-term interaction, which is ultimately governance. This enables feedback, learning and adaptation to occur at the interface between the players involved. Cultural differences can arise within governments between the officials or bureaucrats who work there, which can affect comparison of information transparency, in this case the transparency of Software Product Tenders. Socio-cultural factors exist, which do not immediately change.

B. NEED TO ASSESS TRANSPARENCY IN SOFTWARE ACQUISITION PROCESSES
The legitimacy of governments and stability of democracy as contrary values of corruption, as mentioned above, is fought with transparency, this is the justification for protransparency actions of greater value [51].
With the idea of increasing the transparency of public contracting, as well as identifying and disseminating the best practices of electronic contracting, the measurement of transparency in government agencies in public bidding processes could be part of the solution or mitigate part of the problem [5]. For this reason, some governments, as an initiative, have established different regulations in terms of transparency when it comes to contracts for software development, such as in the European Union, Australia and Brazil [52].
In [53], we noted the need to assess transparency in government software tenders. As methodological support, two qualitative studies have been carried out, on the one hand, a semi-structured interview with experts, both software providers and public officials who carry out bids, and analysis of regulations of some countries on software development bids. Below are the aspects that support the proposal to have a maturity model for software development tenders: i) Regulations for Software Tenders: The existing regulations can be considered a response to the need to distinguish that general transparency regulations require an extension for the case of software development tenders. This is because the public tender processes for software products have aspects such as requirement creation and analysis, which are a set of practices commonly related to software engineering. The scenario of software development tenders includes the specification of requirements, which considers the main functionalities of the system requested as well as non-functional requirements, which makes it a difficult process for civil servants.
The Commission of the European Union, Communications Networks, Content & Technology Directorate-General, development a Guidelines for Public Procurement of ICT Goods and Services SMART 2011/0044: One section of the report describes the difficulties that civil servants in government organizations face when putting out a tender in the ICT area, and therefore they raise the need for political reform with respect to ICT tenders. The focus is on developing market competition among ICT suppliers through the correct use of standards, thereby avoiding dependency on suppliers and salespeople. On the other hand, the focus is on the practices in the tender process terms, so restrictive or discriminatory specifications can be avoided, and competition promoted.
For example, in Italy on July 1, 2006, the Italian Council of State rendered a decision (Consiglio di Stato, sez. V, n. 5181/2005) focusing on the rules governing the public e-procurement of software and ways to limit the risks of software products becoming out of date during an e-procurement.
In Brazil, the Office of the Chief of Staff published Decree 7174 on May 12, 2010, which regulated the procurement procedures for information technology by government agencies in Brazil. Note that this decree contains further details on subjects related to the acquisition of hardware and software, such as requiring the development approach and a survey of all the project features while still in the planning phase; these remained obligatory by virtue of this decree as seen in the following article: ''Art. 2 • Acquisition of goods and services for information technology and automation must be preceded by contract planning, including the basic project or terms of reference containing the specifications for the object to be contracted, Decree 7174, Brazil, 2010 ''.
The government of Australia established a working group that drafted the Report of the ICT Procurement Taskforce, which recommends the development and implementation of a new framework for government procurement of ICT.
We can see that there are initiatives by different governments regarding how to approach the behavior of software acquisition processes with their respective legislations. With this background, we can state that software acquisition processes as they are currently being conducted are in need of a legislative change. Therefore, we have proposed the need to assess transparency in software tenders.
ii) We design an electronic survey: using short questions to know the role of stakeholders within an organization, to know opinions and visions regarding the evaluation of transparency in general and specifically in software tenders, we use variables that we have identified of the behavior of the software bidding process, and we also added an open question to gather additional variables.
The questionnaire was answered by 15 stakeholders. They were selected for convenience, as they all belonged only to the companies participating in the bidding processes, as well as government officials, who could be contacted by email. The form was applied through the Google Forms platform. In this research, we have identified that evaluating transparency in the public bidding process is recognized as a necessity among stakeholders, and also specifically when it comes to software development bids. The details of this research can be reviewed in the attached article: Need for Evaluate Transparency in Software Procurement Processes.
Therefore, in order to have a general measure of transparency, a maturity model is proposed to evaluate transparency in software development tenders, due to its observable variables, such as software functions, suitability, learning capacity, security, among others, and also its very different condition of being. An intangible product seems to justify a differentiated approach.

V. SUB-PHASE II: COMPARISON OF EXISTING MODELS
Maturity models aim to provide systematic frames of reference for measuring the performance of organizations in certain areas of work, with greater emphasis on the technological area [30]. There are also academic disciplines that use the term maturity and develop maturity models as classification schemes. Authors such as [54], [55]- [58] use them to develop diagnoses and define progress measures using maturity models.
The most widespread software maturity models are those belonging to CMM/CMMI (Capability Maturity Model and CMM Integration) of the Software Engineering Institute (SEI) 1 in the USA. They are focused on the development, maintenance and acquisition of software products and services. Their capacity and maturity structure and the mechanism used for determining this structure have been replicated by many other models in other areas [59]. The original CMM model was the Software Capability Maturity Model [60], which was used to identify the software development capabilities of an organization. Its success resulted in CMM-based models being used to improve engineering processes. These were quickly adopted in 1998 by the US government and the SEI to formulate their Capability Maturity Model Integration (CMMI), which improves an organization's processes using a unique structure and a multidisciplinary approach [61], [62].
In [59] the authors, focus on human resources and describe how People-CMM can be applied as a standard for evaluating work-place practices, and as a guide when planning and implementing improvements. The authors indicate that 1 http://www.sei.cmu.edu/cmmi. People-CMM was designed to increase the capability of the labor force, and that SW-CMM [62] was designed to increase the capability of software development processes in organizations.
For example, in [63] applied a capability and maturity model [64] to the supply chain and developed a maturity model for supply chain management processes to improve the performance of the supply chain. In [65]  In the technological area, for example in software process improvement, the Capability Maturity Model is a measure of maturity which defines a structure for software development [66]. The term maturity is also familiar in the area of information systems, for example, the stages in a growth model illustrate the stage an organization has reached in the development process where the use of Information Technology in that organization is being measured [67].
Maturity models are used to measure IT management in the public sphere, as reflected in Uruguay, Brazil, Chile, Colombia and Mexico, among other Latin American and Caribbean countries [52]. We will now briefly describe maturity models contextualized in e-government and open government, which we have used as an important reference in our proposal.

A. MATURITY MODELS FOR ELECTRONIC/OPEN GOVERNMENT
Previous studies have proposed several contextualized maturity models to develop e-government [55]; for example, [68] proposed a growth model consisting of four stages for e-government: cataloguing, transaction, vertical integration and horizontal integration. This model is focused on the functionality, technical capacity and integration of e-government. [30] proposed a maturity model that extended the Layne and Lee model incorporating a customer-centered approach.
In [69], the authors proposed a maturity model consisting of five levels, focusing on the interoperability of digital government: ICT interoperability, process interoperability, knowledge interoperability, value interoperability, and goal interoperability. In [70] the authors developed a progressive maturity model with five stages for joint government, based on dynamic capability. These maturity models can be usefully applied to electronic governments in general, but were not designed to guide open government initiatives focusing on transparency, interactivity, and participation [71].
In [72], a model for e-government was proposed, which integrates technological, organizational, operational and human capabilities, with a multi-dimensional integrated and evolutionary approach. The model includes adjustment mechanisms to align it with national guidelines on e-government. It is based on international experience in maturity models, VOLUME 9, 2021 together with specific e-government models that measure the capabilities of public agencies. The proposed model consists of the three axes of a cube: the information criteria that support business processes, IT resources, and leverage domains. The authors changed the traditional IT approach and defined four leverage domains: e-strategy, IT governance, process and people management, and organizational capabilities. Furthermore, the maturity model produced by [73] incorporated some open government initiatives within its proposed five levels: Casual, Transparent, Participatory, Collaborative, and Engaged. The authors in [71], developed a model specifically to evaluate and guide open government initiatives that focus on transparency, interactivity, participation, and public commitments to collaborate that have become possible largely due to emerging technologies, such as social networks. The model consists of five levels: initial condition, information transparency, open participation, open collaboration, and ubiquitous engagement.
However, the authors [27], complement our proposal by indicating the importance of software system transparency. They propose that to ensure transparency, it should be addressed within the context of the requirements, i.e. transparency should be treated as a non-functional requirement. In [74], the authors investigated further the characteristics of transparency. The authors stressed the importance of ensuring that transparency is accessible, as other quality attributes cannot be achieved without access to a software system. Therefore, our proposal implies a learning discipline within the organization, and we shall now address this concept.

B. SYSTEMATIC MAPPING OF LITERATURE
Systematic mapping allows us to have an overview of a research area through classification [75]. It is a method used particularly to answer, methodically, one or a set of research questions. To do this, the main studies that may contain relevant information are identified (search), the relevant studies are selected after further review (inclusion / exclusion) and, where appropriate, a quality assessment of the selected studies is carried out (bias / validity ) [76]. To compare with other maturity models, it is necessary to know globally the related maturity model works. For this, the protocol for conducting systematic mappings defined by [77] has been followed, adapted to the subject as described in continuation: General guidelines of the work: the contribution of this part of the research is to propose a Maturity Model as an instrument for measuring the level of transparency in LPS's, under the assumption that this model would help government agencies increase the transparency of public bidding and identify and disseminate the best practices for the preparation of LPS's.
Research questions: the IPs that arise from the objectives of the research and the motivation that exist to apply systematic mapping are indicated below.
PI-1: Of the selected studies, how many propose maturity models (or theoretical proposals)? How many propose to use maturity models for LPS's? PI-2: For the maturity models, how many studies address transparency? PI-3: For the maturity models, how many studies do LPS's address?
Key concepts and search strings: to build the search string, a set of keywords were extracted from the IPs and the research objectives, which were concatenated using logical connectors. This search string was iteratively tested in search engines and validated among researchers. When performing the search, this string was applied only to the abstract of the works [85], The resulting string was: (('' methodology '' OR developing'' ''OR implementation'' ''OR software'' ''OR egovernment''' ''OR e-government'' ''OR open government'') AND ''Maturity Model'').
Search and data extraction: the sources that were considered were databases and also websites where digital libraries could be accessed with search engines implemented in order to choose sources with a large number of articles related to the search objects and that allow search for jobs using search strings and logical operators. The selected data sources were Google Scholar, IEEE XPLORE, Association for Computing Machinery (ACM), SPRINGER LINK, and SCOPUS data sources, recognized among the most reputable within the IS community [78].
Inclusion and exclusion criteria: the option with at least one of the words found only within the titles was used as search parameters, excluding patents and citations.
Inclusion criteria: studies from the year 2000 onwards. Studies that address maturity models.
Exclusion criteria: studies prior to 2000. Studies without an author. Documents that do not include the software as a deliverable product. -Duplicate studies in different databases. Studies whose title is not related to the research object. Documents that do not come from traceable journals or procedures.
Search execution: The search string was applied to the selected sources and an initial quantity of 905 jobs was obtained. The information was extracted using the export tools of each of the digital libraries. After eliminating those jobs that were doubly indexed, it was reduced to 898 jobs. Then, the inclusion / exclusion criterion was applied, which was done by reading the titles, abstracts, keywords of each work and labeling them, accordingly, obtaining a final set of 230 works.
Classification scheme: publications are classified in three dimensions: time, categories, and type of proposal. The temporal dimension classifies the works according to the year of publication, taking into account the last 19 years, period 2000-2019.
The dimensions in which the publications are categorized are E-government, Information Technology, E procurement, E service and Business. Although there are jobs that can be classified in more than one category, categories were built from the combination of characteristics.
The dimension type of proposal classifies the works in: -Review: perform reviews of the literature on maturity models or make comparisons between one model and another. -Use: they use maturity models applying them to organizations and evaluating their results.
-Implementation: they propose maturity models related to the proposed classification, that is, the use of maturity models reviews/comparisons with other models, or that address other functions within the electoral process.
Map construction: the final product of the systematic mapping was a map to facilitate its representation and analysis. Figure 3 on the left side shows those that were classified into: implementation, use and review. On the right-hand side, the rankings of the publications by year range are presented.
68 papers, published between 2000 and 2020, were identified that raised a proposed model or implementation. In the same range of years, 14 documents were identified that use maturity models and have been applied to organizations evaluating their results and levels.
Given the search and selection of works carried out, a single work was identified that proposed a maturity model addressing transparency and accountability between 2013 and 2015.
It is observed that efforts are put into formulating and implementing a maturity model for software and process improvements, with 137 jobs (60%). Assuming that transparency is important to governments, the question that immediately arises is: Why are there so few papers that address transparency? For example, only two papers identify key elements of transparency and how to evaluate it. In turn, there is a low number of jobs that focus on open and electronic government, only 32 jobs (13%). One possible explanation would be that establishing a maturity model is a procedure that requires methodological guidance, which becomes a complex process to follow. Therefore, incorporating the measurement of transparency in open and electronic government seems to be a challenging area. According to the extent to which we have been able to review in the technical literature, incorporating the measurement of transparency is an immature area, especially if we intend to measure transparency in LPS's.

C. ORGANIZATIONAL LEARNING
The staff that participate in processes recognize the importance of developing specific procedures. They establish VOLUME 9, 2021 standards to guide and evaluate their performance, practice, and strategies, to trigger adjustments, changes, and innovations that drive improvements in procedures, techniques, technologies, skills, and effectiveness. This requires processes to be based on learning, and that the nature of learning for stakeholders in general, the staff within an organization and the staff at various organizations is well understood. These stakeholders must adapt to their environment, which is a key challenge for which learning is essential, as they can update their knowledge of the environment and thus improve their skills in areas of work where specific processes operate [79].
Staff or adaptive agents can learn and configure shared mental models, which include strategies and rules governing their strategic behavior, on the basis of repeated, stable interactions. These strategies and rules are interaction patterns that are modified dynamically. They enable agents to use the information to anticipate situations, to behave in certain ways, and ultimately to adapt to each other in an endogenous and self-organized manner [80]. Agents make decisions based on typical environmental structures, which are interpreted using heuristics and which operate as cognitive rules of information processing [81].
Thus, agents learn to cooperate among themselves in an uncertain environment and develop a culture, a tacit, shared mechanism for how things are done, based on past experiences and mutual expectations about the future [82]. The dissemination of this culture is not only driven by learning around the strategic benefits that result but also is adopted or imitated and then replicated as a way of doing things, through the mutual influence that agents have on each other [83].
Based on the above and on [84], we can define learning as mutual and dynamic adjustments, endogenous, incremental interactions, founded on experience and new information, which make it possible to revise beliefs, objectives, and how problems are understood, managed and resolved. This organizational learning should be expressed systemically, i.e., using work-place formality. We relied on the following section in the guideline ''Scientific Documentation'' [31] to establish reasonable criteria for our proposed maturity model, and we now detail the steps in the maturity model selection process.

VI. SUB-PHASE III: DETERMINATION OF DEVELOPMENT STRATEGY
In this section we define the development strategy for the Transparency Maturity Model for Software Acquisitions. The different strategies come from designing a completely new model, to enhance an existing model, to mix existing models by transferring structures from selected models. In order to make a decision about the strategy, we first establish a set of concepts to include as part of the target model.

A. RATIONALITY FOR SELECTING THE DEVELOPMENT STRATEGY
A maturity model enables a government agency to evaluate its current maturity level [55]. If adopted in various agencies, it can be used as a standardized tool for benchmarking, as well as providing a common language and a framework for planning and implementing open government. The model informs government agencies of the approaches, capabilities, processes, outcomes, challenges, best practices and metrics for each maturity level [71].
Within government agencies, e-government portals play an essential role in national e-government programs, which includes creating greater efficiency and transparency. Maturity models have been proposed to measure public procurement portals, as in [5], [55].
Measuring the maturity levels of public procurement portals allows e-government agencies to increase the transparency of public procurement [5], identify areas for joint action, encourage knowledge exchange between agencies, and identify and disseminate electronic procurement best practices [85].
Public tender processes associated with a software product are formed from practices commonly related to Software Engineering, which have been explored mainly from Requirements Engineering.
Our proposal requires a specific maturity model that combines e-government and open government measurements, plus the particular attributes of a software tender process, whose practice is commonly related to Software Engineering and whose classification requires specific measurement [23], [24].
Software transparency is a concept that relates to the dissemination of information; it has been used in various environments, mostly relating to the empowerment of the public with regard to their rights [27]. The authors argue that in order to apply transparency, society will have to deal with the way software is offered under this concept.
Furthermore, they suggest that to provide transparency, it must be addressed when specifying software requirements. They describe how requirements engineering should play an important role in software transparency and propose treating transparency as a Non-Functional Requirement (NFR), using the following classification for NFR quality factors: Usability, Auditability, Accessibility and Informativeness, suggesting that these factors contribute to achieving transparency.
Accordingly, accountability is an attribute that should appear in transparency processes. Specifically, it contributes to auditability, which then contributes to transparency [27]. Accountability is perceived as responsibility, and state bureaucrats must report their decisions, requirements, and performance, by informing (informational level), providing justifications (explanatory level), and being subject to sanctions (enforcement level). The recipients of this accountability are software designers and the public as users; they are counterparties with the right to demand information, justification and to punish public managers, who are therefore auditable.
Therefore, we have a set of core concepts related to the problem definition which we need to consider in the target model, they are software process, process improvement, electronic government, open government, accountability, and transparency. From the list in Figure 4, we can see that different models concern different topics, therefore, to take only one model to enhance will imply to miss relevant features of other models, hence, the selected strategy will be to combine different models into the new one.

B. MAIN CONTRIBUTIONS TO THE TARGET MODEL
Given the close relationship to the defined problem we have exhaustively studied different proposals, for those contributing to our model we comment on some aspects to include and main differences with our proposal.
The Open Government Maturity Model (OGMM) [71] was specifically developed to evaluate open government principles, i.e. transparency, participation, and collaboration. Basic ideas contributing to our model are: (i) transparency can be appraised, and (ii) data management practices and citizen-government ways of interaction are useful to describe capabilities. However, we have a great difference with this approach, transparency is a level in the model, and moreover, it appears as a precondition for participation. On the contrary, our model adopts the transparency perspective discussed in [86], where by using some mutual reinforcing patterns, a synergistic relationship between transparency and participation is claimed. This way, we consider some citizengovernment types of interaction and different data management practices as observable transparency capabilities.
The SW-CMM [64] was designed to increase the capabilities of software development processes within organizations. It provides five progressing steps for describing the adoption of a disciplined, quantifiable, systematic and continuing improvement approach for software development. Ideas contributing to our model are: (i) the focus on evidence-based practices and artifacts, (ii) the difference between discourse, formal definitions, and actual disciplined practices, and (iii) the statistical control of the improvements as highest capabilities. The main difference with our model is the domain and the nature of the domain, while the final product in CMM is a technological product, in the case of transparency the product is social perception.
The maturity model proposed in [55] and [72], includes e-government best practice. Its basic structure was based on a comparative analysis of the two main international trends in maturity models as applied to software engineering [72]: CMMI [64] from the USA, and the ISO/IEC standard 15504 from Europe. Main ideas contributing to our model is that CMMI's foundational progressing steps can be moved to the public sector. Another relevant idea is to identify the axes of progression for proposing a multi-dimensional approach. On the other hand, although transparency is recognized as a desired outcome, it is completely out of the universe of discourse in this proposal, trying to adapt it as a transparency model for public procurements of technical systems results impossible because acquisitions is also out of the model. Moreover, the way of getting the maturity level in this proposal is by assigning different weights to variables losing the principle of reaching high maturity levels due to baselines practices.
Software transparency is a concept that is related to the dissemination of information; It has been used in various settings, mainly related to the empowerment of the public regarding their rights [27]. In [27], the authors argue that to apply transparency, society will have to deal with the way the software is offered under this concept. Furthermore, they suggest that to provide transparency, this should be addressed when specifying the software requirements [27]. They describe how Requirements Engineering should play an important role in software transparency and propose to treat transparency as a non-functional requirement (RNF), using the following classification for RNF quality factors: usability, auditability, accessibility, and informativeness, which suggests that these factors contribute especially significantly to improving transparency. The need for auditability supposes giving weight in transparency processes to the attribute of responsibility as a factor that contributes to said auditability, which in turn contributes to transparency [25]. The responsibility implies that the managers of the state bureaucracy must, regarding their decisions, requirements, and actions, inform (informative level), give explanations and justifications (explanatory level), and submit to sanctions (demanding level). Recipients of responsibility, which are software providers and citizens as users, as counterparts, have the right to demand information, justifications and also sanction public managers, who, therefore, are auditable.
Our proposal establishes the characteristics of Software Product Tenders (SPT), defining the SPT, the budget, the bid deadlines and the criteria for evaluating the bid, together with an analysis of the maturity models, and removing some dimensions in our context, as shown in Figure 4.
Our proposal is based on the ''Multi-methodological procedure'' guideline, as we propose an ontological approach to transparency maturity models for software tenders. We have also reviewed the technical literature relating to maturity models for e-government, for open government and for software, and investigated how transparency and accountability contribute to our model. VOLUME 9, 2021

VII. SUB-PHASE IV: ITERATIVE MATURITY MODEL DEVELOPMENT
We propose a maturity model for procuring ad-hoc software, i.e. where a specific government agency requires a development or customization project. We understand that a government agency is a division of the State. It is responsible for a specific state role that requires independent strategic planning, for example, a hospital, a school, a municipality, a university or a branch of the armed forces. Software procurement in such agencies is centralized, and the maturity model can therefore be applied to a government agency with a single procurement process. The model is defined with five levels, as shown in Figure 5. The model measures transparency in four dimensions: Institutionalization, Software procurement process, Communication and Accountability. Each level is defined in Table 1.

VIII. IMPLEMENTING THE MODEL
All maturity models that have reached a certain degree of consolidation are subject to auditing [59], [64], [87] which includes questionnaires and evidence collection. The maturity process developed in this work consists of five levels: 1: Initial, 2: Developing, 3: Coordinating, 4: Managing, and 5: Systematic, to measure a set of dimensions: Institutionalization, Software procurement process, Communication and Accountability.
In order to assess the procurement process in reference to the proposed transparency framework, we need a method for mapping different observable organizational elements and behaviors to one of these five maturity levels. In order to describe what exists in an organization and, hence, to distinguish what elements from them may be observable ones, we account with different proposals of organizational ontologies [88]- [91]. According to this goal, we have used the organizational ontology described by [92] given that they provide basic organizational elements such as goals, tasks, and resources (as inputs and outputs of tasks) which are part of a basic understanding to members of an organization and may constitute part of a questionnaire for organization members or auditors. However, a more complex concept to assess is maturity itself. To solve this question, we assume that to certain degree of maturity should exist specific goals, tasks, and resources, but it does not solve the problem of assessing the progress of the target organization on the defined levels. To do that we use general system theory and, in particular, learning models of organizations. In [93] shows a very good simplification, using wide-common vocabulary, of an organizational learning model passing from some intuition-based behavior to institutionalized practices. Therefore, in order to have elaborate an appraisal method based on observable organizational elements we have considered: Objects, resources: The objects identified for the software procurement process are: administrative documentation, technical documentation, price range, exact budget, questions made by potential participants and answers, award documentation, quality reports on technical bids, supplier evaluation table, sanctions reports, list of bidders, and evaluation mechanism. Objects have six states: ''none'', ''private'', ''private but reactively reported'', ''private but proactively reported within the organization'', ''public but reactively reported upon request'' and ''public and proactively reported''.
Formalized practices: We associate formalized practices with how the discourse regarding software procurement transparency is aligned with explicit, regular, standardized procedures, and how the plan to encourage transparency in government agencies can be demonstrated. Formalized practices have five states: ''discursive'', ''informal'', ''formal'', ''managed'', and ''corrected''. In Figure 6 we illustrate how these elements indicate the level of each dimension.

A. THE SURVEY
For each dimension we have generated a set of questions, each question is related to a level of maturity, in this way we can classify the answers and determine the level of maturity of each dimension.
The questionnaire consisted of 59 questions, distributed as follows: For the ''Software procurement process'' dimension, for example, a total of seven questions were formulated. Each question is related to the definition of the level (summarized in Table 2). The questions (Q) for this dimension are the following: • Q1: Has any process or suggested steps been followed for the acquisition of software? (Definition of level 1).
• Q2 Is there a defined and differentiated process for the acquisition of software? (Definition of level 2).
• The Q3 questions are related to the definition of internal and external communications, respectively: -Q3a Is there a rule that requires the official to follow a software procurement process?  -Q3b Is the process monitored by any authority?
• Q4 refers to the definition of level 4: -Q4a Is the process controlled by quantifiable indicators, for example, number of applicants, number of companies that ask? -Q4b: Have there been corrections to the process according to the results of the control?
• Q5: Is there a periodic and planned analysis of the procurement process where it is improved based on historical results? (Definition of level 5). The resulting questionnaire was sent to officials at government agencies who were asked to respond to our on-line survey. Initially 18 officials from 16 government agencies responded, all of whom had experience in preparing software procurement tenders. The results generated by 93% of the participants showed that their agencies classified as level 1 in our maturity model. The answers from each respondent were used to review the degree of discrimination for each question. All the questions received both positive and negative responses, confirming that effective discriminatory criteria were used. As proof that such questions allow us to effectively identify different levels, some of those obtained are illustrated in Tables 3 and 4. The difference percentages are calculated on the total answers for each question (yes = 1, no = 0).
Threats to the validity of the tool are varied. For example, we do not yet know whether question interpretation by each respondent is the same; nor is there a simple way of validating the trustworthiness of the tool, since this would require several respondents with the same profile within the same organization. Finally, the most important validation is triangulation, i.e. matching replies from respondents with findings from professional audit teams.

IX. CONCLUSION AND FUTURE WORK
This article describes the complexity or diversity of the variables that influence the transparency of a socio-technical process, such as the procurement of a software product through a development project. To deal with this complexity we propose a maturity model that objectively locates an organization at a given level of transparency for such an exercise. We have used the principles described by [31] as a methodological reference in developing the maturity model, in order to review how maturity models used in e-government, open government, CMMI, the conceptual framework for software transparency, and accountability trends constitute relevant background information for the preparation of our maturity model.
We used these conceptual frameworks to prepare a maturity model describing five levels of transparency for software procurement through development projects: (1) Initial condition, (2) Developing, (3) Coordinating, (4) Managing, (5) Systematic. These enable the maturity levels of agencies to be measured for software tenders in the following dimensions: Institutionalization, Software procurement process, Communication and Accountability. We have proposed these levels and dimensions in order to achieve integration and innovation, based on an exhaustive review of specialized literature on this topic.
In addition, we have prepared the first version of an evaluation tool and we have tested the discrimination criteria with positive results. We can demonstrate that there are no questions that were answered in exactly the same way by all 18 respondents.
However, as the methodological framework adopted indicates, we need to complete various tasks using this first proposal and broaden our experience in order to arrive at an iterative process that will achieve a consolidated position.
JORGE HOCHSTETTER received the Educative Informatics master's degree in information systems engineering from the University of La Frontera, Temuco-Chile, in 2010, and the Ph.D. degree from Alicante University, España, in 2021. He is currently an Academic with the Universidad de La Frontera. His research interests include requirements engineering, software development tenders, and transparency in electronic procedures. She is currently an Associate Professor with the Universidad de Los Andes. Her research interests include big data, data mining, and machine learning. In the area of Analytics, there are several projects that the PI has been working in order to expand the PI's knowledge. In particular, PI wants to highlight her recent admission to the Instituto de Sistemas Complejos de Ingenieria (ISCI), the Centre of Excellence, for a period of two years, subject to evaluation. The center has five lines of research and the PI belongs to the Data Science line.
CARLOS CARES received the degree in civil-informatics engineering from the Universidad de Concepción, Chile, in 1989, the M.Eng. degree from Universidad Federico Santa María, in Chile, in 1996, and the Ph.D. degree from the Technical University of Catalonia, Spain, in 2012. He is currently the Head of the Software Engineering Studies Center, University of La Frontera, Temuco, Chile. His research interests include requirements engineering and software engineering for intelligent systems. His experience is in information security, e-health, and e-government applications.
MAURICIO GARCÍA OJEDA received the master's degree in local and regional human development from the University of The Frontera, Chile, in 2005, and the Ph.D. degree in sociology from the Universitat Autonòma de Barcelona, Spain, in 2015. He is currently an Academic with the Universidad de La Frontera. His research interests include public policies, governance, and territorial development.