RALO: Accessible Learning Objects Assessment Ecosystem Based on Metadata Analysis, Inter-Rater Agreement, and Borda Voting Schemes

The increasing number of people are living with disability in the World and their access to formal education is considered a challenge for the development of the online education and educational resources. This problem is considered one of the 17 sustainable development goals that are focused on inclusive and equitable quality education. Nevertheless, the existing proposals for mainstream accessibility in virtual education are still complex to apply. However, the models, standards, and good practices to contribute to the virtual educational process and the design of learning for all are identified. For these reasons, in this paper, we describe an accessibility evaluation proposal based on 4 interaction domains: user analysis and interaction, intelligent systems, knowledge databases, and evaluation. In the same way, we describe a set of tools that constitute a Repository of Accessible Learning Objects (RALO) from the perspective of accessibility and adaptability metadata. In this line, the knowledge database follows the regulation and educational models focused on the students with disabilities needs and preferences from the conception of universal design. The validation of the proposal is based on the interaction study and analysis of regular and disabled students and teachers who developed the Learning Objects (LO). To determine whether there was consensus among the teacher’s scores, we used Kendall’s Coefficient of Concordance W.


I. INTRODUCTION
The existence of various models, standards and tools used for the application of accessibility in digital educational resources considering metadata, is still considered a ''technical or computer issue'' and is not committed to the diversity in learning of all students and even more in those who have disabilities. The diverse educational experiences of students with The associate editor coordinating the review of this manuscript and approving it for publication was Aasia Khanum . disabilities help to generate resources for a wider universe of needs and requirements. The correct labeling of accessibility metadata in learning objects has a great influence on the effective response of personalized search engines according to interaction requirements that facilitate learning. Although contributions throughout history have generated standards and regulations that have motivated research on the subject, there is a lack of suitable implementation and frequent use for their application, especially in developing countries. The information from quantitative, qualitative or mixed studies VOLUME 11, 2023 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ related to accessible learning objects is insufficient to determine the impact on students with disabilities at a general level, so an evaluation and feedback process is required both for those who generate resources and for those who consume them; this involves and commits all actors within an educational project that supports a virtual environment without barriers. Considering that accessibility and adaptability achieve a synergy in the production and evaluation of a learning object (LO), a referential framework is established to support the proposed analysis from the perspective of the student and his or her adaptability requirements and from the perspective of the teacher or creator and manager of an accessible LO. This document presents the proposal of an ecosystem for the evaluation of learning objects through the development of a feedback repository of tools that automate processes of metadata use and considerations of accessibility experts, is organized as follows: Section II presents the background and the standards and models considered for the proposal. Section III provides details of the proposed architecture for the development of the accessibility and adaptability ecosystem in accessible learning objects. Section IV presents the analysis of results. Section V presents the limitations encountered, and Section VI concludes with the discussion of findings and recommendations.

II. BACKGROUND
The existing relationship between the different digital educational resources that make up a virtual environment and their interaction with the user, demands the establishment of characteristics that allow to analyze the accessibility and adaptability in each of them. Accessibility in a virtual educational resource on the web is based on the existing worldwide standards from the creators of the W3C and its guidelines through the WAI and WCAG, processes that have been endorsed and adopted by several legislations in various countries. Although not all standards are completely correlated with the experience in a virtual educational environment, the fundamental bases of accessibility respond mostly to the application and compliance with guidelines that establish interaction compliance for people with disabilities. Web accessibility is led by the World Wide Web Consortium (W3C) which is the primary source for establishing technical standards to ensure accessibility, [1] including: • WAI ARIA: Defines technologies to make dynamic web applications more accessible.
• WCAG: Establishes guidelines for creating accessible websites.
• ATAG: Establishes guidelines for developing authoring tools with accessibility in mind.
• UAAG: Establishes guidelines for developers of browsers, players, etc. considering accessibility.
In relation to accessibility in virtual learning environments, the guidelines are given by • IMS AfA: Global learning consortium leading standards for access for all [2]. • ISO/IEC 24751 Adaptability and individualized accessibility in e-learning, education and training, based on IMS AfA [3]. When developing accessible digital material it is necessary to consider visual, auditory and textual resources and their different levels of interaction. Reference [4] considers that accessibility is not only framed in technology and its interaction, it also requires feedback from the design of learning experiences for all, considering not only technology and pedagogy, but also ethics. Figure 1 shows a general structure of the components that are necessary in an accessibility and adaptability evaluation considering the guidelines proposed by [2] and [3] A. REGULATIONS RELATED TO ACCESSIBLE EDUCATIONAL RESOURCES ISO 40500: The ISO 40500 Web Content Accessibility Standard was published in 2012, with which several countries were able to adopt it as legislation, since WCAG 2.0 was a proposal from a private organization, which made its implementation difficult. In 2018 WCAG 2.1 was published, which has not yet been approved as an ISO standard, but it is the reference recommendation in legislation at present [1]. Other international standards such as the latest version of the European standard EN 301549 have already adopted WCAG 2.1 as a base [5]. Since 2020 this standard is mandatory for all websites of public bodies in the countries of the European Union, including those of educational institutions and their virtual campuses [6].
Virtual environments usually refer to web content. Therefore, compliance with the WCAG and its guidelines are clearly identified.
Based on the WCAG compliance criteria it is possible to detect levels and requirements for content creators, each of them correlated with the educational resources mostly used in a learning object.
ISO 24751: The ISO 24751 standard, on accessibility and adaptability for e-learning, education and training, provides information on accessibility metadata on both the resource (DRD) and the needs and preferences of users (PNP), through an information model that describes the needs and preferences of learners or users when accessing the digital resource or service; where in addition the conformity criteria depend on the role played by the technology according to the different requirements for educational presentation applications and alternative access systems [7].
ISO 24751 is based on the first versions of the Access For All (AfA) recommendation of the IMS Global Learning Consortium [8], and is made up of 3 parts: • ISO 24751-1 Information technologies. Individualized adaptability and accessibility in e-learning, education and training. Part 1: Framework and reference model [3].
• ISO 24751-2 Information Technologies. Individualized adaptability and accessibility in e-learning, education and training. Part 2: Needs and preferences for the digital provision of ''access for all'' [9] • ISO 24751-3 Information Technologies. Individualized adaptability and accessibility in e-learning, education and training. Part 3: Description of digital resource ''access for all'' [10] B. MODELS RELATED TO ADAPTABLE EDUCATIONAL RESOURCES UDL: Universal Design for Learning, represents the methodological efforts used by the current of universal learning design, understanding the challenge that means the diversity of learning for students and potentiates the construction of flexible materials, techniques and strategies oriented to a greater number of users. It is considered as a teaching model that provides equal learning opportunities for all students. It contains three principles: multiple means of representation, multiple forms of participation strategies and multiple means of expression. These principles of educational technologies allow maximizing the learning of all students using different teaching methods that identify barriers in a timely manner [11]. UDL constitutes a learning reference for students with greater flexibility, technique and strategies, providing multiple means of its 3 principles, which provide feedback on the user's experience. Its principles, or also called networks, are classified into: • Representation: Covers the perception of the information in different formats such as text, also if it includes support elements for users such as audio descriptions, and at the same time if it allows a more developed understanding of the information such as complementary documents or an accessible guide to the information of the resource: --Presents information in different formats (perception) --Uses support elements to decode information --Provides options for understanding • Expression: It considers user interaction and the use of technical aids if necessary, as well as group or individual activities, and finally the ability to provide tools to facilitate understanding, such as tutorials.
--Allows multiple means of interacting with the material --The response pattern in the activities (expression and communication) is varied. --Facilitates the development of executive functions.
• Motivation: Covers the way in which a learning object can capture the interest of a user and the different ways to arouse their curiosity, challenges or challenges; in addition to considering whether it provides elements of reward and incentives that support the effort and persistence of users, based on evaluation instruments such as questionnaires for self-evaluation or co-evaluation.
--Provides options to capture interest --Provides options for sustaining effort and persistence --Provides options for self-regulation LRMI: Learning Resource Metadata Initiative. Its objective is to describe educational resources by adding specific properties (metadata) so that they can be easily located through search engines and services. The specifications are based on the vocabulary offered by Schema.org and other standards.
With support from AEP (Association of Educational Publishers), CC (Creative Commons), Division 501, Bill & Melinda Gates, and the William and Flora Hewlett Foundation, LRMI has developed a metadata framework for tagging learning resources on the web. The LRMI 1.1 schema was adopted by Schema.org in 2013, which makes it viable for resources, through their LRMI metadata, to be recognized by major search engines.
AFA: IMS Access for All (AfA) v3.0 [8] was created with the aim of simplifying the ISO/IEC 24751 standard [9], [10] due to the difficulties encountered at the time of putting it into practice. Both, standard and specification in its version 3.0, cover the whole process from the reading of the user's needs to the search mechanism necessary to find the learning object that satisfies those needs or preferences.
It consists of two data models to describe accessibility [8]: • Personal Needs and Preferences (PNP): Model for describing users' needs and preferences for accessing and interacting with digital resources.
• Digital Resource Description (DRD): Accessibility metadata description model for digital learning resources. Schema.org: It is currently the most widely used vocabulary in the structured data community for Internet search engine optimization. To define accessibility metadata in Schema.org it is necessary to focus on the different types of web content that can be classified by metadata schema [12]. The ''CreativeWork'' category includes books, movies, photographs, videos, etc. The types can in turn have subtypes, for example, ''CreativeWork'' has the type ''MediaObject'' and this in turn ''VideoObject'', among others.  The accessibility metadata defined by Schema.org are based on those specified for the IMS AfA v3.0 DRDs, with a significant subset of these being selected. Each of these metadata may have a possible value that is defined in the specification. In this way it is possible to determine the accessibility characteristics of any digital resource published on the web.

C. ACCESIBILITY METADATA
The metadata of an LO responds to valuable information that efficiently determines a process of search, reuse and feedback on features that support assessment and interaction based on the preferences and needs of the learner and their virtual educational experience.
The AfA DRD specification ( Figure 2) defines the accessibility metadata of a resource that is necessary for the search and use of the learning resource according to each user.
The way of working with accessible learning objects requires the creation of original and adapted learning objects. An original resource corresponds to an initial resource, while an adapted resource presents the same educational information as the initial or original resource, but other characteristics change, such as the sensory form of access to the resource, the language, etc.
The original resources can have any number of adaptations, which can be total or partial, that is, they are either adaptations of the entire educational content or only part of it.
Metadata can support adequate information on original resources, such as: access mode, accessibility features (subtitle, sign language), interaction features (keyboard, mouse, voice) and possible accessibility descriptions of certain programs.
The AfA PNP specification ( Figure 3) is intended to enable the definition of learners' personal preferences and needs (or those due to disabled environments). PNPs are used in combination with the AfA DRD specification to deliver digital resources that meet a user's needs and/or preferences.
The principles for accessibility in e-learning focus on providing customization options based on user preferences, facilitating content equivalents, compatibility with technical aids and full keyboard and mouse access, providing context and orientation information and others associated with following specifications of guidelines and/or standards of governing bodies in the field such as IMS.

III. REGULATIONS AND MODELS
For the proposal of accessibility metadata that contribute to an evaluation, the criteria for content creators were taken as reference [13] and the identification based on the types of digital educational resources analyzed in WCAG with emphasis on the creation, identification and reuse of learning objects. The analysis is correlated with Schema.org metadata for being the most applicable.

A. DRD -ACCESSIBILITY IN LEARNING OBJECTS
With the background and efforts of standards such as [9] and [14], in DRD, as well as standards such as [1] and [15],), in accessibility identifies 4 groups of resources commonly used in the development of learning objects in virtual environments, these are: 1) VISUAL DIGITAL RESOURCES 7 criteria directly related to visual digital resources established in WCAG and their relationship with Schema.org metadata specified in the WCAG are analyzed in Table 1.
2) AUDITORY DIGITAL RESOURCES 11 criteria directly related to digital audio resources established in WCAG are analyzed and their relationship with Schema.org metadata specified in Table 2 3) DIGITAL TEXTUAL RESOURCES 19 criteria directly related to textual digital resources established in WCAG and their relation to Schema.org metadata specified in the WCAG are analyzed in Table 3

4) LEVEL OF INTERACTIVITY
We analyze 6 criteria directly related to the level of interactivity established in WCAG and its relationship with Schema.org metadata specified in Table 4

B. PNP -ADAPTABILITY IN LEARNING OBJECTS
UDL is considered as the supporting model for assessing the adaptability of a learning object based on the learner's needs and preferences. The following tables show the identified correlation of the UDL guidelines and the metadata [8], [16], as well as some proposed metadata.

1) PRINCIPLE OF REPRESENTATION
We analyze 6 criteria directly related to the level of interactivity established in WCAG and its relationship with Schema.org metadata specified in Table 5   TABLE 3. Correlation of accessibility metadata with textual digital resources.

2) PRINCIPLE OF EXPRESSION
3 guidelines and 10 items are analyzed to establish correlation with the Schema metadata explained in Table 6 VOLUME 11, 2023

3) MOTIVATION PRINCIPLE
3 guidelines and 8 items are analyzed to establish correlation with the Schema metadata explained in Table 7 With this background, a model for evaluating the accessibility and adaptability of an LO according to its metadata is proposed.

IV. METHODOLOGY A. ECOSYSTEM ARCHITECTURE
The proposed ecosystem and its architecture (Figure 4), identifies 4 layers of interaction that requires the loading of the learning object with its respective packaging in educational format SCORM, IMS, Common Cartridge, generally the metadata of frequent use responds to LOM and the programs to perform LO generate them by default, however it does not consider the accessibility metadata. That is why the ecosystem considers the creation of tools that facilitate the automatic labeling of accessibility metadata, OER ADAP and LOMPAD WEB Schema. The LO can then be entered into a repository that will facilitate the evaluation. In Figure 4 a diagram of the general architecture is shown.
In relation to the tools generated based on the knowledge base we have:

1) OER ADAPT
This tool aims to support the teacher or educational content creator in the adaptation of a learning object, considering accessibility features ( Figure 5). OerAdap is developed in Django as Backend and Angular as Frontend, it can be accessed through the following url: https://oeradap.edutechproject.org. The tool can be loaded with a learning object in one of the educational formats mentioned above, then the user can choose which multimedia educational resource found in the LO to adapt, whether videos, audios, images, paragraphs or just integrate the accessibility preferences bar. With this parameter, the system sweeps through the different HTML files of the LO in search of the tags that contain these resources and their respective paths. Once the resources have been identified, the user is presented with a pre-visualization of the learning object followed by the adaptations that can be made to the page, which are as follows: • Video adaptation: The system replaces the video player with an accessible one, developed by the ''Floe'' project [17]. This player has an accessible, minimized interface and provides captioning and text synchronized with the audio. The system downloads the video in case it is on another platform, in the case of youtube videos it downloads the video and the subtitle file (.str) in two predefined languages, Spanish and English. In case the video does not have subtitles or is on another platform, the system extracts the audio from the video and analyzes it through Automatic Speech Recognition (ASR) in order to extract the different sentences that make up  the explanation given by a narrator or people involved in the video. This generates the subtitling file to be used by the player. Similarly, the player requires a vtt file that contains video subtitles written with the WebVTT standard to display timed text of the video playback, this file in both cases is generated by the system from the.srt file.
• Adaptation of images: The system shows the user the images found in each of the pages, along with the description found in the case that it has one, otherwise the user can add or modify the description. The system also provides options for the different images that can be found in the LO, being the case of table images, the system allows to manually add one to a table, which will replace the image. In the event of equations, the user can VOLUME 11, 2023 replace the equation image with an equation written in MathML.
• Paragraph adaptation: The system identifies the paragraph tags of the html files and shows the user paragraphs that exceed a certain number of characters, to which it gives the option to add the easy read option or generate an audio of the paragraph. In these cases, icons will be displayed below each paragraph.
• Audio adaptation: The system allows you to identify audios found in learning objects and automatically generate a verbatim transcript or manually transcribe the audio.
• Accessibility Bar: By selecting the accessibility preferences bar option which was taken from the open source Fluid Project [17] Preferences Framework. This bar has a series of parameters that help to make the educational resource accessible, such as: change of font size, screen contrast, etc. The system embeds in the different html files of the preferences framework including CSS, HTML and JavaScript files. When the different adaptations are made, the system embeds in the XML metadata file in LOM format the metadata proposed in the research.

2) LOMPAD WEB
This tool seeks to support in the correct labeling of metadata based on the standard of the LOMPAD tool, and incorporating Schema.org initiatives with LRMI, identifying the accessibility metadata contributed by [18] and [19], and the quality considerations of [20], which aim to add a set of classes and properties to the description of learning resources on a par with other standards with an emphasis on accessibility. For this purpose, a previous analysis of previous proposals and specifications proposed for the LOMPAD tool, which considers IEEE-LOM, was carried out ( Figure 6) IEEE LOM describes a conceptual data schema that defines the metadata structure for learning objects. It establishes a schema divided into 9 categories of metadata elements, each of which includes several metadata elements or subcategories that allow learning objects to be 'tagged'' at a high level of detail, thus we have a: • General: groups general information that describes the learning object as a whole.
• Life cycle: includes characteristics related to the history and current state of the learning object, and everything that has affected it during its evolution.
• Meta-Metadata: allows you to include information about the metadata instance itself.
• Technical requirements: groups information on requirements and technical characteristics of the learning object.
• Pedagogical characteristics: includes information on the pedagogical and educational characteristics of the learning object.
• Rights of use: it groups information on intellectual property and conditions of use for the learning object.
• Relationships: groups characteristics that describe the relationships between this learning object and other related objects.
• Annotations: provides comments on the pedagogical use of the learning object and provides information on when and by whom the comments were created.
• Classification: describes the learning object according to a certain classification system. LOMPAD WEB performs a mapping of the information in the XML metadata file to show it to the user so that he/she can modify it without having to add it again. It also incorporates the Learning Resource Metadata Initiative (LRMI), Schema.org and its accessibility metadata. For this purpose, a new profile known as LRMI is added to the Lompad tool menu, and a 10th category called ''10. Accessibility'' is displayed, which contains the same scheme as other LOMPAD windows considering the fields: Similarly, the addition of the following fields is considered in category 8 ''Annotations'' according to the proposal of. [21]: The tool allows exporting metadata in XML and Json format and is available from the link https://lompads.edutechproject.org.

3) RALO (REPOSITORY OF ACCESSIBLE LEARNING OBJECTS)
Based on the assessment areas defined in section III for both digital resources and for establishing student preferences, a question-fed assessment is provided for the accessibility expert and for the student and their interaction experience.
The questions are based on and distributed within each evaluation area according to the guidelines and metadata analyzed above, these questions have the option to answer Yes, No, Partially and Not Applicable, where the score will be defined according to the positive contribution in accessibility and adaptability of the learning object.
The Learning Objects repository is developed in Django as Backend and Angular as Frontend, and is available through the following link: https://repositorio.edutech-project.org.
The tool has three main usage profiles, these being: • Student Profile: Users with this profile enjoy an LO recommendation system based on their selected accessibility preferences when registering, they can search for LOs by different filters, visualize and interact with the selected LO and perform an evaluation through a form to determine its adaptability by means of the principles.
• Expert Profile: Users with this profile help to maintain a repository of LOs with quality and accessibility. Experts can view the different LOs loaded in the repository, interact with it and perform the evaluation through the form, to determine the accessibility of the same by means of the different concepts: visual, auditory, textual digital resources and level of interactivity.
• Teacher Profile: Users with this profile are mainly in charge of uploading their LOs to the repository. They will be able to see the results of their evaluations through graphs and a score in different areas. At the same time, the teacher will be able to visualize feedback that the system generates automatically based on the answers of the evaluation of the teacher and the student. This will allow to improve or correct which resource can be accessed by all students. In relation to the Metadata Evaluation, this is fed by the contributions of the questions posed to experts and students, as well as the automated metadata of the aforementioned tools, for which their category is considered for both accessibility and adaptability, as well as relevance, generating the following Equation 1 where X 1 , X 2 ,. . . X n represent the score that each metadata obtained in the evaluation. N represents the total amount of accessibility and adaptability metadata identified in the evaluation for each category noted in the proposal.

V. EXPERIMENTATION AND RESULTS
The validation of the model is based on the interaction study and analysis of regular and disabled students (Case 1), as well as teachers who develop LO (Case 2). In relation to the evaluation of accessibility and adaptability experts, a decision support module was developed using Borda voting schemes (Case 3). Likewise, a validation process was carried out using the iter-rater agreement method: Kendall's Coefficient of Concordance W, in order to determine whether there was consensus among the experts' scores. The validation of comparisons in students and teachers was reviewed with the initial evaluations of the LOs without considering accessibility and the subsequent review of the same, considering accessibility and adaptability of LOs to determine differences, assessments and justifications on the methodology or the need for modification in disagreements.
Automatic evaluation effectiveness is intended to assess the tools generated based on their usability and scalability.
Three case studies are established to validate the application of the proposed models, considering an initial baseline of accessibility, adaptability and metadata.
The case studies respond to a procedural analysis for their overall design, data collection, applied analysis, research results, discussion and main conclusions.
The following research questions are addressed: • What are the considerations for generating accessible learning objects?
• What are the characteristics and barriers that do or do not allow a student with a disability to achieve learning with an LO?
• How can existing standards and tools related to accessibility contribute to generate accessible LOs? Data sources for the case studies are used as sources of data: • 10 original LOs and their adapted versions, • 8 students, 3 of them with disabilities, • 5 teachers with experience in the generation of LOs • 5 accessibility experts to validate the model proposal. The mixed method for the research design is determined, considering the primary data from the interviews and secondary data from the evaluation questionnaires.
The evaluation questionnaires respond to a descriptive and comparative approach. The monitoring of an accessible resource considers the recruitment of a control group to compare the experiences of barriers and accessibility features. The research considers demographic questions in its 3 cases.

Case 1: Student Perspective (LO Evaluation in RALO)
We worked with a sample of 8 university students from different careers who were tested in the Gesell Chamber to establish possible emotional parameters. Three students are disabled. The age of the students is in the range of 20 to 26 years (mean 22.375, SD 1.999) and of this group 3 were women and 5 men. Table 8 shows the summary of the interaction process of the students with the LO.
Interaction results According to the challenges posed, questions related to navigability, usability and evaluation are VOLUME 11, 2023   formulated. Each question considers the Likert scale. The results obtained are shown in Table 9: In relation to the analysis of achievements in the challenges posed, according to the proposed protocol, questions were established that invite the interviewee to analyze his or her perception of achievement, establishing as a possibility whether he or she was able to achieve it, if he or she was not able to achieve it or if he or she is not sure if he or she achieved it. The questions related to expectation and usefulness were established as a satisfaction scale from 1 to 10, where 10 is very high and 1 is very low. Table 10 indicates the responses captured: Evaluation Reports With the sample of 8 students, it is established that the time spent interacting with the review of an LO and its evaluation The tool generated to store accessible learning objects and their evaluation, presents an average initial expectation by the students of 8.12 and after the tests the final expectation is 9.37.
In order to determine whether there was a change (error variance) of each student in relation to the original LO and the adapted LO, the paired samples Wilcoxon test was applied (since the data are not normally distributed) [22]. As can be seen in the box-and-whisker plot of Figure 5, on the left we can see the initial expectation of the original LO, while on the right we can see the final expectation in relation to the adapted LO. With this simple graphical inspection it can be seen that there is a substantial improvement in this parameter and an increase in the mean from 8.12 to 9.38.
However, in order to determine whether there was a significant change, the following null hypothesis (and its alternative version) was proposed: • H o : There is no effect or increase in students' expectation in relation to the original LO and the adapted LO.
• H a : The adaptation of the object increased the expectation of the students.
To perform this analysis, we have assumed the following aspects: • The expectation variables (initial and final) are on a scale.
• The differences in the observations of the variables are not normally distributed.
• A baseline and endline measure of student expectation was conducted.
• Data were randomly drawn from the student sample. The Wilcoxon test was calculated using the statistical software R (version 4.2.1) considering that the mean of the students' expectation is lower with the original LO. A p-value equal to 0.04449 was obtained, which is less than the significance level alpha (0.05). We can conclude that the mean expectation of the LO before adaptation is significantly less from mean expectation after the adaptation.
The evaluation of an LO by a student responds to questions focused on the 3 UDL networks, so it is necessary to determine more characteristics than a basic evaluation. This evaluation of LO at a difficult or regular level was particularly detected in older regular students. The feedback delivered by the tool to each student had a particular acceptance to generate a process of constant improvement.

Case 2: Teachers' perspective interaction and evaluation feedback
A sample of 5 university teachers from different careers who were tested in the Gesell Chamber and by videoconference was established. Of this group of participants, the age range is between 37 and 48 years old (mean = 41.6, SD = 4.39), 3 are women and 2 are men, and two teachers are disabled. Below, in Table 10 a summary of the participants' demographics can be seen, as well as the time they required to interact and review with the LOs.
According to the challenges posed, questions related to navigability, usability and evaluation are formulated. Each question considers the Likert scale. In Table 12 the obtained results are presented: In relation to the analysis of achievements in the challenges posed, according to the proposed protocol, questions were established that invite the interviewee to analyze his or her perception of achievement, establishing as a possibility whether he or she was able to achieve it, if he or she was not able to achieve it or if he or she is not sure if he or she achieved it. The questions related to expectation and usefulness were established as a satisfaction scale from 1 to 10, where 10 is very high and 1 is very low. Table 13 indicates the captured responses.
Evaluation Reports With a sample of 5 teachers, it was determined that the time spent interacting with an LO upload and its evaluation feedback ranged from 45 to 63 minutes (mean = 53.8, SD = 6.76).
The tool generated to upload accessible learning objects and their evaluation information on accessibility and adaptability, presents an average initial expectation by teachers of 7.6 and after the tests, the final expectation is 8.2.
In order to determine whether there was a change (error variance) of each teacher in relation to the original LO and the adapted LO, in this case the paired samples Wilcoxon test was also applied (since the data are not normally distributed) [22]. As can be seen in the box-and-whisker plot of Figure 8, on the   left we can see the initial expectation of the original LO, while on the right we can see the final expectation in relation to the adapted LO. With this simple graphical inspection it can be seen that there is no substantial improvement in this parameter and the increase is not very significant (from 7.6 to 8.2). VOLUME 11, 2023 With this, and in the same way as in the case of students, the following null hypothesis (and its alternative version) was proposed: • H o : There is no effect or increase in teachers' expectation in relation to the original LO and the adapted LO.
• H a : The adaptation of the object increased the teachers' expectations. A p-value equal to 0.2914 was obtained, which is greater than the significance level alpha (0.05). We can conclude that the mean expectation of the LO before adaptation is not significantly less from mean expectation after the adaptation.
The accessibility and adaptability feedback of an LO responds to a process that begins with an initial diagnosis to establish basic questions that the teacher must know about his or her learning object. Although most teachers determine the need and relevance of accessibility feedback for their LO, working with resources from different manufacturers and authors does not always make it possible for a teacher with a disability to interact with the resources, even if the page is accessible. The feedback given to each teacher, both by the expert and the student, constitutes a process of constant improvement, where co-design and co-evaluation strengthen the development of accessible LOs with greater reusability and scalability.

Case 3: Decision support system for accessibility and adaptability criteria of LOs: a proposal based on Borda Voting schemes.
The Borda voting scheme is a method that has been successfully used to address problems in various areas such as decision support in the field of psychology (consensus of methods for psychological profiling analysis) [23], elimination of dataset imbalance for financial fraud detection [23], the selection of essential features for the improvement of rice production through the fusion of descriptors obtained from ranking methodologies [24], feature selection based on metaheuristic optimization algorithms (Grey Wolf Algorithm) and Borda voting schemes, [24], among others.
In this line, in the field of analysis of accessible educational resources, the work done by human experts is fundamental and, on the other hand, there is a great variety of criteria regarding the relevance of certain metadata related to the WCAG 2.1 accessibility guidelines. Therefore, a decision support module is proposed to perform this analysis and obtain the consensus of various experts and criteria using Borda voting schemes.
The Borda voting method used in the module supporting the relevance analysis of questions associated with metadata by expert consensus is defined as follows: be M(c) = [m 1 (c), m 2 (), . . . ., m k (c)] finite set of metadata according to criterion (c). The value of k will depend on criterion (c), since for certain criteria there may be up to 10 metadata. It is assumed that E = [e 1 , e 2 , . . . , e m , ] represents the set of expert evaluators. Given these two sets, it is important to take into account that k≥ 3 and m ≥ 3, in order to be able to apply the Borda voting method. Moreover, the following conditions must be satisfied: Given these two sets, it is important to take into account that for this evaluation we had the assessment of priorities of 5 experts in Accessibility and Adaptability who were asked to establish an order of preference. Next, the experts' data is established and the application of a voting scheme with visual digital resources where 4 questions are defined and it is requested to establish the prioritization from 1 to 4, being 1 the highest preference and 4 the lowest preference, explaining that it is not possible to repeat. With the experts' answers, the Table 7 is created.
Given these two sets, it is important to take into account that for this evaluation we had the assessment of priorities of 5 experts in Accessibility and Adaptability who were asked to establish an order of preference. Next, the experts' data is established and the application of a voting scheme with visual digital resources where 4 questions are defined and it is requested to establish the prioritization from 1 to 4, being 1 the highest preference and 4 the lowest preference, explaining that it is not possible to repeat. With the experts' answers, the Table 14 is created. With this, the decision support module automatically generates the preference matrices shown in Figure 9. As can be seen, for the matrix, expert 1 indicated the preference of the first criterion over the others, so the system places 1 in the following row-column pairs: 1 − 2, 1 -4 and 1 -3. Similarly, for expert 1 the second criterion takes precedence over criteria 4 and 3, so the system places 1 in the following row-column pairs: 2 -4 and 2 -3. Finally, for expert 1 the fourth criterion is more important than the third criterion, so 1 is placed in row 4 and column 3.
In the same way the system proceeds to generate the matrices for the other 4 experts. Once all the matrices are available, the system makes a sum by rows, with which the votes for each criterion are established, having in this case 15 votes  for the first criterion, 6 votes for the second, 7 for the third and 2 for the fourth. With these votes an order of preference or relevance of the criteria is established, as can be seen in Table 15: Similar analyses are performed for all categories of DRD and PNP. The experience in the development and use of LOs constitutes an important diagnostic basis to focus the case studies. The interaction with a tool that includes the storage and evaluation of LOs, such as the repository, requires an analysis of the interaction with the tool, which involves several profiles. The teacher or LO generator, the student consumer and evaluator from his learning experience and the accessibility expert, who points out the evaluations made. In the case studies with students and teachers, positive conclusions were reached regarding the ease of use of the tool and visualization of everything it entails, motivating autonomous learning in a more intuitive way, generating a self-assessment of knowledge and skills.
In relation to the time used to interact with the platform and carry out the evaluation, most of them say that it is adequate and can meet the challenges posed. In particular, the use of search filters based on the user's needs and preferences, the simple and intuitive interface when presenting the educational resources, the evaluation feedback and the full screen display of the learning object are particularly pleasing.
In relation to the problems detected, it is established that the evaluation by the student is long to perform and the need for an introduction or help on certain purposes of the tool. The interaction of people with disabilities was optimal, with the exception of total visual impairment, since the different screen readers and browser preferences require a more in-depth review of certain fields.
The contribution of experts in visual impairment both in teaching and students, has established a plan for proofreading and continuous improvement based on co-design and co-evaluation with special emphasis on this disability.
Although Equation 1: considers the evaluation of accessibility and adaptability of an LO through its metadata, both for DRD and PNP, for each accessibility metadata detected, a score of 2 was established as a sign of possessing it, 1 if it partially possesses it and 0 if it does not possess it, with the option of not applicable for such metadata not to be considered in the score. After the review of experts and the establishment of LO comparisons according to the needs and preferences of students with disabilities, it is foreseen that it is necessary to consider the weight, so a greater precision in the scoring is determined Equation 2: where x 1 , x 2 ,. . . x n represent the score that each metadata obtained in the evaluation of Yes / Partially / No / Not applicable; p 1 , p 1 , . . . p 1 represent the weights based on the priorities analyzed with Borda voting and N represents the total amount of accessibility and adaptability metadata identified in the evaluation for each category indicated in the proposed model. As an answer to the research questions formulated, the following is established: • What are the considerations for generating accessible learning objects? According to the interviews to LO developers, it is established that accessibility is not one of the main characteristics to develop digital educational media, since there are many trends to generate them and the proliferation of tools, techniques and strategies is constantly increasing, so most teachers interviewed say that their learning curve is still in the generation of digital resources, and therefore the consideration of accessibility and the proper use of metadata, is an additional work that they prefer to be established automatically, without further emphasis on it.
It is necessary to strengthen the use of support tools that facilitate the correct labeling of a learning object, and whose action does not constitute an additional effort to the developer of a resource, but is pre-established as we move forward with a culture of inclusion based on the diverse learning experiences of the student, analyzed from their variability and detected barriers and not from their disability and limitations. The teaching experience with students with disabilities, raises actions to develop accessible and adaptable educational resources.
• What are the characteristics and barriers that do or do not allow a student with a disability to achieve learning with an LO? The characteristics and interaction barriers detected in an LO that works on the web, are the same as those identified by the WCAG, however, its educational characteristic generates a deepening of the teaching and the need to adapt to VOLUME 11, 2023 the diversity of learning styles. Consequently, the range is widened, since it is not frequent that a person has only one disability, and from this information can consume resources ''labeled'' for that disability. The achievement of learning is given by different evaluations that respond to different times and circumstances. Although a learning object can encourage the search for information, by itself, it does not achieve learning but rather motivation, and digital educational resources facilitate autonomy, reuse and generation of new knowledge.
The analysis of use cases with both regular and disabled students establishes that the duration of interaction of an LO will be correlated with motivation. The more concrete and interactive it is, the longer the student's attention will remain on the resource. The barriers detected, whether in access or comprehension, generate an immediate abandonment of the resource and possibly frustration.
It should be noted that the greatest number of barriers that require constant review based on the tests performed, is detected in the total visual impairment, since multimedia features of a resource overlap and are often not fully covered by a screen reader and its different versions. However, the diversity of visual, auditory, textual and interactive digital resources could become the support required from multiple forms of presentation to advance in the understanding of a major topic.
• How can existing standards and tools related to accessibility contribute to generate accessible LOs? The approach and support of the tools generated for the testing process, respond to the support of applicability in accessible LOs, endorsed in several existing standards for Learning Objects such as LOM and LRMI. For accessibility, ISO 24751-2, AfA, Schema.org, ISO 40500 (WCAG 2.0) and WCAG 2.1 considerations were taken into account. For adaptability, UDL, ISO 24751-3, AfA and Schema were considered.
Learning in a web environment is one of the main ways to access education, so the existing regulations that support accessible interaction are an important basis to advance in the support of online training and all the resources generated in a virtual teaching environment. The different didactic educational materials must be analyzed from the different characteristics of their granularity, where their universal design, flexibility, interoperability and reusability constitute a strong contribution to establish a scalability in the generation of learning.
In order to determine whether there was consensus among the experts, the inter-rater agreement was analyzed. For this purpose, we calculated the Kendall's Coefficient of Concordance [25]. ). The calculation process was carried out using R statistical software (version 4.1.2) and the following hypotheses were made regarding the agreement among the experts: • H o : There is no consensus among experts for the proposed metadata.
• H a : Yes, there is a consensus among experts for the proposed metadata. As can be seen in Table 16, for Kendall's Coefficient of Concordance W we worked with the interpretation proposed by [26]: • 0.00 <= w < 0.20 -Slight agreement • 0.20 <= w < 0.40 -Fair agreement • 0.40 <= w < 0.60 -Moderate agreement • 0.60 <= w < 0.80 -Substantial agreement • w >= 0.80 -Almost perfect agreement According to the Kendall's Coefficient it can be seen that there is a ''Substantial agreement'' for all the metadata, except for the ''Expression network'' and the ''Motivation network'', where we can see that the consensus is ''very weak'' and ''moderate'', respectively. For the metadata where there is ''Substantial agreement'' the null hypothesis is rejected, since the p-value is less than 0.05 (statistically significant).

VI. LIMITATIONS
This research presented limitations during the process and in its search to answer the interaction questions from the experience of students with disabilities. The selection of the sample is limited.
It is established that there is a scarcity of accessibility evaluation in LOs at a general level, and even more so endorsed by people with disabilities. Moreover, the available studies tend to focus more on design recommendations than on evaluating the effectiveness of their implementation and improvement process. The use of accessibility standards is subjective, in several cases it responds to evaluative models that, although they consider accessibility as a metric, it is inconsistent to reach a common implementation process, especially with regulations of different interpretation in the legislations of each country. There is a lack of references that establish an important sample of students with disabilities, their follow-up, monitoring and improvement in learning design and digital competencies, which requires more time to obtain reliable data.

VII. DISCUSSIONS AND RECOMMENDATIONS
The information from quantitative, qualitative or mixed studies focused on the impact on students with disabilities at a general level, requires an audit and continuous improvement process that involves and commits all stakeholders within an educational project that also provides for digital competencies that support the effectiveness of an evaluative model of accessibility and adaptability of an LO. The use of accessibility metadata for models and standards in e-learning environments is supported by scientific research. ISO 24751, AfA, Schema, are analyzed and constitute the basis for the proposal of the evaluation model in DRD (Digital Resource Description). Although the accessibilities on the web led by the standards specified in the WCAG constitute the commanding voice, they do not keep in itself a direct correlation with virtual education, since the complexity of education involves several areas that not only focus on digital resources, but also on the learning characteristics of the learner, as pointed out by research of evaluative models of e-learning. With this, the student's perspective is directly related to their learning, and establishes an analysis from the adaptability of a resource based on personal needs and preferences (PNP).
Although accessibility focuses on generating scenarios that enable autonomy for all people, especially those with disabilities, research often tends to establish guidelines that label certain disabilities, reducing barriers for some and increasing them for others, and thus the digital divide does not diminish, but rather is avoided with specific justifications focused on disability. It is emerging to identify the needs and preferences of the user from their variability in learning and not from the limitation of the disability itself, so the range of options is expanded by the human diversity that is intended to cover from the needs and preferences, which may well be similar to those of another student categorized as ''regular''. In this sense, the best experiences have been given by the SAD model as a response to the diversity and variability in learning rather than labeling a particular disability.
The use of metadata in Learning Objects is not a rewarding practice, it is usually associated with the platform or developer program that may make the effort to apply standard, common or basic metadata by default. If the mandatory metadata of an LO responds to an automatic treatment, the widespread use of accessibility metadata is still not achieved, so it is necessary to generate automatic labeling tools to avoid dependence on additional knowledge to properly label a resource, transferring this effort to the tool and not to the teacher or developer and manager of educational resources.
The analysis of the use of accessibility metadata in LOs is still incipient because there is no formal agreement or consensus regulations for its adoption, especially by governing bodies such as LOM. The research in case studies shows its little use, so it is necessary to use automatic generation tools for a greater experience with the use of current metadata, which allows a better investigation of the requirements of new accessibility metadata and adaptability according to models.
Although the metadata allows the creation of new metadata according to the needs, it is still necessary the massive use of the current accessibility metadata. The proper use of metadata at a general level that responds to a common language is an optimal way to locate, consume and socialize resources. It is necessary the proper use of accessibility metadata respecting the initial standards of the LOs such as LOM and LRMI, strengthening the standardized use focused on the various efforts of ISO 24751, AfA, Schema.org, WCAG and DUA proposed in this model validated with the case studies and the conclusions obtained.

A. RECOMMENDATION AND FUTURE DIRECTIONS
Considering that UNESCO's Sustainable Development Goals (SDGs) 2030 [27] establishes in its SDG 10 the reduction of inequalities, in SDG 4 quality education and in SDG 17 Partnerships for the goals, it is emerging the importance of formalizing networks to enhance the research carried out as solutions at national, regional and Latin American level, with synergies of European experiences such as the ESVIAL project, Edutech of ERASMUS+ and sustainability plans that favor the applicability of present and future research to determine joint actions of socialization and subsequent technology transfer, supported by research networks associated with the subject matter.
It is necessary to elaborate and implement legal regulations especially in developing countries. The importance of demonstrating the benefits of education for all from the implementation of accessible LOs contributes to optimal quality assessment. More research is needed on the needs not only of specific disabilities, but also in the context of the learning experience, platform design, maintenance and inclusion of new features. The constant evaluation and improvement of accessibility in virtual education depends on the identification of needs, so establishing an ecosystem with a framework can provide recommendations and feedback of experiences through machine learning techniques. It is necessary to identify strategies by describing activities and identifying accessibility needs in educational resources. Content analysis and identification of cross-cutting strategies can feed cases that allow a combined flow to achieve change management.
The report on technology and disability [28] points out that the pandemic generated a drastic process in the use of technology in all aspects, making opportunities for labor and educational inclusion viable, however, technological advances must be accompanied by awareness processes for a culture of inclusion that eliminates prejudice, indifference, ignorance and discrimination, strengthening the acquisition of digital skills of people with disabilities, since technological advances and their tools will always be behind people and do not imply a change in social mentality. It is important to consider that the use of technological adaptations in many cases allows the elimination of barriers and the completion of studies under equal conditions. The use of technological resources and applications that facilitate understanding and interaction strengthens the process of co-design and co-evaluation in educational environments. However, it is important to consider that the acquisition of new technology could be costly, so it is important to strengthen research in the development of free hardware and software.
The incorporation of intelligent systems could contribute in the evaluation of accessible resources and in the feedback of profiles and personalization from the user experience [29], [30]. However, the possibilities of learning scenario analysis are diverse especially in the topic of inclusion that in many cases evaluates professional competencies and skills. The implementation of intelligent agents could generate insights that feed back into the system and provide multiple alternatives as didactic strategies in learning for all.

B. CONCLUSION
This research is fed by the various models, standards and tools used for the application of accessible LOs, and proposes an ecosystem that considers the creation of tools that support the accessibility and adaptability evaluation model based on the needs and preferences of the student. It establishes the need for a publication of accessibility information suitable for an effective personalized search response according to the interaction requirements of an educational resource.
The objective of this document is to contribute to establish evaluative metrics of accessible LOs according to their accessibility and adaptability metadata to evaluate a learning object, identifying characteristics of the resource (DRD) in its visual, textual, auditory digital resources and its level of interactivity, as well as its impact on learning according to the needs and preferences (PNP) established in the networks of expression, representation and motivation of DUA. The proposal of the model identifies its potentiality and experiences that it is necessary the frequent use and socialization of the current accessibility metadata in platforms and tools for the creation of LOs, this would achieve the identification or creation of proposals based on specific evaluative models applied in an educational model.
It is established that it is possible to evaluate accessibility and adaptability in e-learning by the information contained in its metadata. However, the curvature of learning guided by awareness for a culture of educational inclusion is still in the process of development, but its growth in this period of pandemic was and continues to be drastic, so it is expected to arrive sooner to measure the impacts of learning for all. In one way or another, Education for All is supported by several issues (accessibility) but learning for all (adaptability) still requires measuring efforts and modeling knowledge. It seeks to positively impact the needs and preferences of all students through the appropriate use of metadata, especially those with disabilities. It is necessary to consider that the measurement of impact is a process of continuous monitoring and follow-up based on the sustainable development of a culture of educational inclusion supported by awareness and knowledge of potentialities and not limitations.
The knowledge base constitutes a timely field of applicability that will respond to the diversity of learning following a phase of implementation of accessibility evaluation tools and culture in LOs.
PAOLA INGAVÉLEZ-GUERRA (Member, IEEE) received the dual master's degree in multimedia design and a specialty in intellectual diagnosis from the Universidad del Azuay, Ecuador, and the Universidad Tecnica Particular de Loja, in 2014 and 2015, respectively. She is currently pursuing the Ph.D. degree in information and communication technologies with the University of Alcalá. For about ten years, she has been involved in ICT and disability research. She received the title of Systems Engineer from the Army Polytechnic School. For 18 years, she has been a Tenured Professor at the Salesian Polytechnic University. She currently works as the Director of the Biomedicine Career. She is the author of book chapters and articles published in various magazines. Her main research interests include ICT and disability, accessibility, universal design of learning, and pedagogy. ANGEL PEREZ-MUÑOZ was born in Quito, Pichincha, Ecuador, in 1995. He received the degree in computer science from the Salesian Polytechnic University, Ecuador, in 2019. Since 2019, he has been a Research Assistant with the Department of Computer Science, Salesian Polytechnic University, where he is currently a member of the UNESCO Chair of Assistive Technologies for Educational Inclusion. His research interests include the application of artificial intelligence techniques to improve the educational inclusion of children, youth, and elderly, as well as serious games and web accessibility.
JOSÉ HILERA-GONZÁLEZ received the B.S. degree in telecommunication engineering from the Polytechnic University of Madrid, Spain, in 1992, and the Ph.D. degree in science (mathematics) from the University of Alcalá, Spain, in 1997. He was a Lecturer at the Polytechnic University of Madrid. He is currently a Full Professor with the Polytechnic School, University of Alcalá. He is also the Director of the master's program in software engineering for the web and the Coordinator of the ESVI-AL cooperation network on accessible virtual education, funded by the European Union's ALFA III Program. He has authored or coauthored over 150 scientific works (books, articles, papers, and research projects), some related to learning technology and accessibility. Since their creation, he has organized several editions of two international conferences on accessibility: the International Conference on Quality and Accessibility in eLearning (CAFVIR) and the International Conference on Application of Information and Communications Technologies to improve Accessibility (ATICAcces).
SALVADOR OTÓN-TORTOSA received the degree in computer sciences engineering from the University of Murcia, in 1996, and the Ph.D. degree from the University of Alcalá, in 2006. He is currently an Associate Professor with the Computer Science Department, University of Alcalá. He coordinates the master's on agile software development for the web. He is the Coordinator of the European Project EduTech about accessible virtual education, granted by the Eras-mus+ Program of the European Union. He is the author or coauthor of more than 80 scientific works (books, articles, papers, and research projects), the majority of them directly related to Learning Technology, Accessibility, and Interoperability. His research interests include learning objects and e-learning standardization and mainly in accessibility, metadata, distributed learning objects repositories, interoperability, and enterprise architectures.
ELENA CAMPO-MONTALVO received the B.S. degree in computer science from the Technical University of Madrid, Spain, and the Ph.D. degree in science (mathematics) from the University of Alcalá (UAH), Spain. She is currently an Associate Professor with the Computer Engineering Department, UAH, and the Director for Cooperation and University Sustainability of the UAH. She has authored or coauthored of chapters, books, articles, and papers related to learning technology, quality in higher education, algebra, CAGD, and symbolic computation. Also, she has directed several projects funded by Spain and the European Union. VOLUME 11, 2023