Maps4Learning : Enacting Geo-Education to Enhance Student Achievement

During the last few years, increasing emphasis has been given to geo-education in middle and high school curricula, transforming the way certain subjects are now taught to provide students with comprehensive knowledge about dynamics and interconnections in the world, which also rely on relevant geographic information. This is the case for history, geography, the earth sciences, and other subjects. However, so far, little has been done to provide adequate learning tools that can support such an important transformation. This paper proposes the integration of advanced geospatial technology in traditional interactive learning tools as a way to describe experiences that help students understand phenomena and improve their competencies. The system is the result of a usability engineering process aimed at providing users with an effective learning experience, iteratively analysing their expectations and needs with respect to geo-referenced content. The resulting learning environment, Maps4Learning , allows users to manage content in terms of learning objects named geoLO+ , which are extended with spatial and temporal components and built according to standardized metadata. The system was tested in the context of a middle school programme. A usability study was carried out to analyse the impact of geoLO+ resources in terms of perceived quality, engagement, and student learning performance, and the outcomes of the resources were compared to the outcomes of traditional teaching methods. The results were encouraging and showed that learning improvement can indeed be achieved using Maps4Learning.


I. INTRODUCTION
Territory represents a unifying element for providing people with powerful heterogeneous information and services during their everyday lives. It is common indeed that data provided by a service also has embedded geospatial components in terms of either input parameters or geographic reference associated with the results. This feature allows users to handle phenomena and local dynamics through unique virtual access to related geographic information in a way that is also useful in other domains, such as cultural heritage, environmental protection and education. In particular, much emphasis has been given to the value of new learning methods, which allow students to acquire comprehensive, spatial-related knowledge in any subject and enhance their skills for their future roles in society. As a consequence, the meaning of the term geoeducation has gradually evolved from 'education in geography' to the wider concept of 'education about our world' [33]. According to the well-known National Geographic community, geo-education is expected to happen across many subjects in traditional curricula to enable young people to analyse problems from different perspectives related to different disciplines. History is one subject that immediately suits this approach. Acquiring knowledge about a historical period becomes easier if it is strongly connected to the underlying territory, which guides data exploration and information visualization tasks. The same applies to environmental science and the earth sciences. Other disciplines instead show their relationship with the territory while exploring a topic of interest. This is the case for the life sciences, for example, which create connections between phenomena under investigation and their possible causes once the territory is taken into account.

THE MOTIVATION
Following the described trend, much research has recently been done in the field of e-learning to pursue the goal of enhanced education in geography and to devise tools that include features related to semantic web and ontologies (see the section on related work). However, no attempts have been made to support a multidisciplinary approach to education through e-learning. Additionally, little attention has been devoted to hiding the increased complexity of the learning components behind usable interfaces, which could help instructors build customized learning paths while improving students' engagement with a given topic.

THE RESEARCH CONTRIBUTION
In this paper, we address the challenges mentioned above and propose the following: • a new model of spatio-temporal learning object • an ontology specially conceived to enrich the knowledge base of such learning objects, and • a usable, innovative geo-education environment that supports both instructors and learners in the adoption of the new model. The geoLO+ model allows users to handle didactic units in terms of metadata used to describe several properties, including spatial and temporal properties. We built an ontology that organizes the concepts of disciplines, topics, and (education) levels in a 3-dimensional space, enhanced by adding both a temporal and a spatial axis, altogether forming a new reference system for resource allocation. Such an ontology was conceived to allow learning content designers (e.g., schoolteachers) to organize resources based on semantic relationships, which may ultimately allow learners to browse interconnections among different didactic topics.
We show that an emphasis on user experience (UX) during the design of a geo-education environment may allow both instructors and learners to better exploit the high potential of geographic information and the semantic web. A usability engineering process was indeed followed to develop the proposed Maps4Learning educational environment, aimed at providing users with an effective learning experience, iteratively analysing their expectations and needs with respect to geo-referenced content. The challenge we faced was integrating advanced geospatial technology inside traditional interactive learning tools.
To experiment with the approach, we developed the new geoLO+ resource in the Moodle learning environment and implemented the Maps4Learning web application, where students can navigate didactic units through their geographic references. The usability engineering process was completed by evaluating the system in the context of a middle school. A usability study was carried out to analyse the impact of geoLO+ resources in terms of perceived quality, engagement, and student learning performance; we also compared the results of the resources to those of traditional teaching methods. The results were encouraging and showed that a learning improvement can indeed be achieved using Maps4Learning.
The paper is organized as follows. Section II recalls some relevant related work. In Section III, the process of usability engineering that was adopted to design the Maps4Learning educational environment is described. In Section IV, we explain the formal process leading to the construction of a geoLO+ knowledge base. In Section V, the Maps4Learning visual environment is described, and the relevant interaction design choices are explained. Section VI describes the experiment conducted in a middle school and analyses the results obtained. Some final remarks are made in Section VII.

II. RELATED WORK
The goal of this section is to provide a brief description of recent work dedicated to e-learning environments conceived to support teaching in geography, which includes features related to the semantic web and ontologies.
The literature about online learning is wide and includes different approaches used in investigations during the last three decades. Starting from well-established learning management systems, possibly enhanced by innovative technologies, such as augmented and virtual reality [27] [16] that is integrated with social media platforms. MOOCs have gained wide popularity and are producing a large amount of data that can be used to monitor both a single user's performance and trends resulting from groups of learners. To this aim, learning analytics has been introduced as an emerging research field to manage information gathered from massive platforms and use it to inform learning design decisions and better satisfy future learners' needs [19,2].
The educational environment of Maps4Learning proposed in this paper represents an innovative online learning tool for geo-education, which supports instructors in building multidisciplinary didactic units, the geoLO+, which are enriched with geographic references. Maps4Learning differs from a MOOC in that it was conceived for a limited number of learners, who may benefit from the multidisciplinary nature of didactic units. Indeed, much work has been done proving that e-learning tools improve students' performances if a multidisciplinary approach is followed when preparing the units, especially in those units dealing with scientific matters and disciplines [3,13].
Moreover, unlike MOOCs, the geo-education tool we present was conceived to be used for closed online courses.
Here, didactic units structured in terms of geoLO+ are distributed by training organizations or schools, are used for groups of regularly enrolled students, and are distributed on the Maps4Learning platform, which can also be part of a traditional training course. These courses are usually led by a teacher or a tutor, who allows communication with the participants and can perform live training in virtual classrooms.
The literature about e-learning includes different topics, such as e-learning systems, e-learning management systems, and mobile learning. In this field, the role of the learning objects is fundamental [10]. A learning object (LO) represents a coherent content unit with a standardized form, which embeds information on how to sequence and organize it into courses. The definition of a LO has been revised and extended to build different versions capable of being better adapted to complex multimedia content. In particular, in [8], LOs were extended to include a geographic component, which could be used, as an example, to reveal to the citizens of a territory the geographic information that is usually hidden in learning content, thus improving that content's searching and traceability.
Geographic information and e-learning also represent the basis for mobile learning (m-learning), a topic related to applications that support learning anywhere and anytime through the usage of mobile devices [29].
The introduction of Web 2.0 basic features has modified the way learning content is created and distributed. It has brought substantial changes to traditional e-learning methods. The introduction, for example, of wikis and blogs, has enhanced students' interaction, thus giving them a more active role in the training process and providing teachers with higher awareness of the results of their work. Moreover, in addition to pre-packaged e-learning content, students may aggregate units by means of embedded applications and may dynamically create customized content. To reach this aim, ontologies have provided geographic e-learning environments with relevant contributions.
In [1], Ali et al. created an adaptive, quiz-based e-learning system by using ontologies that allow users of different levels and capabilities to learn their course content. The learning materials include physics, chemistry, and geography extracted from DBpedia and are presented to the users as a question/answer activity. The adaptive e-learning system is then compared with the baseline system. In [6], Bratsas et al. proposed an online game-based e-learning system derived from the Greek DBpedia that includes topics from geography, history, astronomy, politics, economics, and other general knowledge-related content.
Grivokostopoulou et al. [14] provided an adaptive, webbased e-learning system using ontology and semantic web technologies. The e-learning system allows users to learn different topics of the artificial intelligence system, which is personalized through rules that take into account users' preferences and level of knowledge.
In [12], El Fazazi et al. proposed a personalized e-learning platform by using the semantic web to provide a digital space for universities for academic, higher education and research purposes. The platform allows various students to appear for placement tests in their domain of expertise, which are then evaluated using machine learning models. Based on the classification of the machine learning algorithm, the system suggests specialized training using a meta-heuristic optimization algorithm. Similarly, Rani et al. [23] employed two ontologies to provide users with adaptive learning materials, track and manage their style, and change them accordingly. Specific adaptive agents are employed to monitor and change the user's styles of learning.
Mouromtsev et al. [21] developed an e-learning system using an ontology and the Information Workbench to rectify the list of problems that currently exist in the Russian education system. The authors employed semantic web technologies to cope with problems such as the weak structure of the learning materials and the poor connections between their components.
In [26], Rocha and Faron-Zucker employed semantic web technologies to automatically generate quizzes that contain questions about geography, answers to those questions, and distractors. The approach is implemented and investigated via the use case of the famous French game "Les Incollables".
Yarandi et al. [31] proposed an adaptive ontology-based approach to provide different users with course content according to their knowledge, capabilities, and preferences. The ontology is used to store the learning content as well as to keep track of users' progress. The content is updated for the users based on their learning progress. Similarly, in [30], Tunde et al. designed an ontology-based model for an elearning management system for course outlines, teaching methods, activities, and learning styles. The system also prevents the learners from skipping any courses that are designed as prerequisites for them.
In [4], Borges and Silveira exploit descriptors associated with metadata to improve students' searches in educational semantic portals on the web. In particular, they focus on educational videos and propose adding descriptive information to such resources once they have been properly segmented and associated with semantics for each segmented portion. This approach allows users interested in specific video content to obtain more precise and easier access to the required section.
In [28], Shmelev et al. proposed a technique for sequencing learning objects for different users using a genetic algorithm, which is one of the most widely used meta-heuristic algorithms. The objective function makes use of the ontology and Bloom's taxonomy. The ontology can be used to describe the connection between the learning objects, while Bloom's taxonomy evaluates the quality of that connection.
The geoLO+ learning objects and the interactive learning environment Maps4Learning were designed taking into account the demonstrated benefits coming from the use of ontologies. Each resource described as a geoLO+ can be located in a 3-dimensional space expressed in terms of disciplines, topics, and (education) levels, thus improving the discovery functionality that can be executed by additional metadata. Adding semantic relationships enriches the capability of creating multidisciplinary contents, which can be searched through multiple geographic and temporal references, which are positioned in the 3-dimensional space.

III. THE USABILITY ENGINEERING PROCESS
In the educational domain, the adoption of user-centred approaches to design e-learning systems represents a promising innovative methodology with respect to traditional approaches, which were mainly focused on the technological quality of such systems. HCI theories and methodologies have been proven to support the design of appropriate e-learning settings responding to the complex and rapidly changing requirements of both the academic and business contexts of our society [9]. In the present section, we summarize the usability engineering process we followed for the Maps4Learning educational environment, aim at providing users with an effective learning experience, and iteratively analyse their expectations and needs with respect to georeferenced content.
An initial contextual inquiry was carried out among a group of 10 teachers from primary and middle schools, with a twofold goal. First, we wished to understand the extent to which geographical references are used while teaching, to identify the subjects where they are most commonly used and to gain insight into the methods and tools that are currently adopted to enrich a given subject with geographic references. The second part of the inquiry was then devoted to determining the extent to which digital teaching support is pursued at school and/or for homework. The interviewees reported several examples of subjects taught with the support of geographic references, from history, biology, and physics to the Earth sciences, Italian literature, and more.
In many cases, the temporal dimension of the geographic information was also relevant (e.g., for biology, the bubonic plague in the territory of Milan, Italy, in 1630-1631, or, for history, the growth of the Roman Empire from 264 B.C. to 44 B.C.). For the instructional tools adopted in a classroom, the interviewees usually rely on the use of Internet-connected multimedia interactive whiteboards, and many of the interviewees adopt cloud platforms to share teaching materials with students and manage homework. The teachers consider such tools pedagogically valid, yet they admit that higher effectiveness could be achieved through the ability to connect materials to their geographical references in terms of historical map visualization.
As a result of the contextual inquiry, we started a scenariobased design process meant to build a geo-education component, which could enhance learners' experience with didactic content inside a traditional e-learning system. The challenge was, on the one hand, to provide instructors with some usable interfaces able to support geo-educational activities and, on the other hand, to improve students' engagement through innovative and customized learning paths.
The major requirements were as follows: management of geospatial didactic resources (instructor), design of a geo-educational topic (instructor), and interactive visualization of any didactic geoeducational topic in a given discipline for a given education level (student). Satisfying the first requirement was a prerequisite for the instructors. An appropriate knowledge base was designed that consisted of didactic resources, which properly encompass any (possibly indirect) historical or spatial reference to other objects/events of interest inside a given topic and can be associated with multiple disciplines. A Web of Data-oriented approach was followed to realize such a knowledge base to connect ontologies and complex data on a world scale, as explained in detail in Section V.
A user-centred interaction design process was carried out to provide Maps4Learning with usable interfaces. As illustrated in the use-case diagram of Fig. 1, the instructor may interact with the system to add/modify/delete didactic resources managing the underlying geoLO+ knowledge base, or create a geoLO+ starting from existing resources, or give a lesson organizing the available geoLO+ resources. The student, who represents the second type of primary user for the Maps4Learning platform, may browse a geoLO+ offered as part of a lesson and may download didactic materials that also include thematic maps associated with certain topics.
As anticipated, a participatory design approach was adopted. Among the 10 teachers who took part in the initial contextual inquiry, the 4 who appeared most motivated and keen to contribute were asked to participate and actively give their feedback in crucial design decisions. They were also involved in formative evaluation activities performed to test the interface prototypes throughout the design process. In the present section, we describe the creation process of a geoLO+ knowledge base. Before that, some preliminaries are given on the set of metadata applied to model a document as a spatio-temporal resource.

A. PRELIMINARIES
Each resource (i.e., text, photo, video, and didactic material) available on the web can be structured according to the Dublin Core (DC) Metadata Element Set [11]. This set represents a vocabulary of fifteen broad and generic properties to be used to facilitate information discovery on the rapidly growing web. Table I lists the Dublin Core metadata elements. The fifteen Italic terms represent the basic elements to which subelements and qualifiers have been added.
Each term is, in turn, specified through a set of 5 attributes, namely, Name, Label, URI, Definition, and Type of Term and, when applicable, also by additional attributes that provide further information, namely, Comment, See, References, Refines, Broader Than, Narrower Than, Has Domain, Has Range, Member Of, Instance Of, Version, Equivalent Property. As an example, the DC metadata element Coverage is defined as follows: Term Name: Coverage URI: http://purl.org/dc/terms/coverage LABEL: Coverage DEFINITION: The spatial or temporal topic of the resource, the spatial applicability of the resource, or the jurisdiction under which the resource is relevant. COMMENT: Spatial topic and spatial applicability may be a named place or a location specified by its geographic coordinates. The temporal topic may be a named period, date, or date range. A jurisdiction may be a named administrative entity or a geographic place to which the resource applies. The recommended best practice is to use a controlled vocabulary such as the Thesaurus of Geographic Names (TGN). Where appropriate, named places or time periods can be used in preference to numeric identifiers such as sets of coordinates or date ranges.

B. THE geoLO+ KNOWLEDGE BASE CONSTRUCTION
To organize didactic units in terms of knowledge base elements, each resource can be associated with a subset of the above DC metadata elements. In particular, the term Coverage is used to express the spatial and temporal properties enriched, when necessary, by details in terms of geometry and format, which allow locating it in a 3-dimensional space according to the concepts of disciplines, topics and levels. The purpose is to build a knowledge base where the role that a document (resource) plays can be fully exploited, that is, it is automatically selected anytime it contains a direct or indirect and historical or spatial reference about any object/event of interest for a topic in a given discipline. This goal is achieved by adopting a Web of Data-oriented approach, where resources are also linked through semantic relationships between (parts of) the resources, thus enriching the knowledge base and projecting it on a world scale, always keeping in mind the goals of availability and accessibility by both humans and software applications [15]. Fig. 2 shows an ancient document about the history of Naples, which can be used to illustrate the proposed approach. It describes an event involving a famous point of interest (POI), "Il Castel Nuovo e il Molo Grande", an Angevin fortress with its pier, also known as "Il Maschio Angioino", built by Carlo d'Angiò starting in 1279.
In this exceptional document, Castel Nuovo and Molo Grande are represented with the arrival of boats. Information describing the event is shown in Table II.  The resource has three different (implicit and explicit) historical references. The first two explicit links refer to the date of the described event, namely, 1477, and to the document date (i.e., a range 1498 -1503), respectively. The third one, an implicit reference that can be added to the metadata of the document, corresponds to the year of monument construction, namely, 1279. To properly handle such a multivalued feature, the Coverage element has to be extended so that each historical reference is captured by a suitable format and is associated with a corresponding geometric primitive (e.g., polygon or point).
Semantic relationships are necessary for building the underlying knowledge base and establishing links among documents referring in some way to the same resources. The model adopted for semantic relationships is RDF [25]. Each statement is enunciated as a triple, subject -predicate -object, where subject and predicate are expressed as resources (i.e., they are specified as URIs/IRIs) while an object may be a resource or a literal object.
In the previous example, many semantic relationships can be initially stated as RDF triples suitable to express relationships among individuals or statements with the rdf:type (a, for short) predicate. As an example, to state that the resource Castel-nuovo-nel-1477 is a manuscript authored by Melchiorre Ferraiolo and dated between 1498 and 1503, the following triples (subject -predicate -object) can be specified: where dcterms and gc correspond to the Dublin Core and geoLO+ prefixes, respectively.
Such an initial set of metadata and semantic/temporal/spatial relationships is enough to help users capture every basic feature associated with the item of interest. As an example, it is possible to select documents about POIs referring to the Aragonese period and obtain information about their location, authorship, state of conservation, etc.
This initial level of data structuring underlies the exploration of the whole knowledge base in terms of selections and conditions applied to specific known parameters. To enhance the capability of modelling concepts and relationships in the domain of interest, more expressive constructs and operators were adopted, namely, some properties from RDF Schema and OWL [32]. Then, an ontology was built that provides designers with a data model and vocabularies for describing the domain content. It is based on a model for the basic disciplines and organizes geoLO+ to let users (students) discover and explore content. In particular, the ontology organizes the concepts of disciplines, topics and levels in a 3dimensional space that is enhanced by adding both a temporal and a spatial axis, thus allowing document allocation with reference to a discipline, a topic relevant to it, an education level, one or more temporal tags and one or more spatial positions.
Once the triples have been specified, they can be serialized according to the RDF/XML syntax [24]. The ontology was built in Protegé [22]. Fig. 3 displays part of the taxonomy and some classes of interest.
Far from being complete, it contains basic topics concerning the geo-history ('geostoria') discipline, and the class hierarchy is organized in terms of lesson (lezione), place (luogo), period (period) and subject (argomento). In particular, place and period are structured according to some well-established online ontologies, such as GeoNames and Time, subject is organized in agreement with the curriculum and syllabus of the school's didactic offerings, and lesson is organized as a set of geo-history (geostoria) classes. This last component is then divided into subclasses of different points of interest (punto di interesse, POI), such as environmental POI (for example, natural bay and forest), cultural POI (for example, museum and archaeological site) and historical POI (for example, church and monument). These georeferenced POIs are acquired by posing SPARQL queries to DBpedia. Additional queries can be posed to populate the underlying knowledge base through the Virtuoso SPARQL Query Editor to the

endpoint for DBpedia. A procedure in Python was built that derives and translates results into csv files for open dataoriented management.
In the following section, some of the more expressive statements are specified; these were validated by the reasoner used to build the ontology. The indentation emphasizes the triple structure, thus avoiding redundancy in their components where dbpedia corresponds to the DBpedia prefix.
Such an improvement in terms of relationships enhances users' exploration capability. As an example, it is possible to capture the locations of POIs whose historical descriptions cite the presence of Queen Giovanna of Aragon, thus visually creating a geographic link among places where she was in a period chosen by the user.

III. Maps4Learning -AN ENVIRONMENT FOR GEO-EDUCATION EMPOWERMENT
The goal of this section is twofold. It describes how an expert in a discipline (e.g., a teacher) can create a geoLO+ structure according to the model described in Section III. Then, this section shows how such resources are displayed in the visual environment for knowledge exploration by different typologies of users.

A. THE INTERACTION DESIGN FOR geoLO+ CREATION
The following figures show the activities performed for resource creation and modification. Such a capability is offered as an additional functionality in Moodle, the popular modular open source platform used to support e-learning processes [20]. In Moodle, users can add content and generalpurpose stuff and develop, plan and handle different kinds of learning evaluation activities, such as assignments and questionnaires. The open source nature of Moodle also allows for the creation of customized functionality. Fig. 4 shows a geoLO+ resource that was developed and added to the set of default activities and resources.
A geoLO+ is conceived to model multidisciplinary content according to the set of mandatory metadata described in Section III. Once the geoLO+ option is selected, the insertion of a new document, handled as a new resource, may start (Fig.  5). The Coverage element can be managed either as a set of timestamps and a couple coordinates or in a more complex way, such as a hierarchy of temporal attributes and polygonal areas (set of vertices). Fig. 6 shows the geo-referencing task and the corresponding parameters for the new resource. Coordinates are automatically acquired by positioning the geoLO+ on the map. To establish semantic relationships, it is possible to connect a geoLO+ and every resource semantically connected to it, thus allowing semantic queries in terms of the graph matching of triples (Fig. 7).   Fig. 8 shows the Maps4Learning visual environment used to query and explore the knowledge base that has been populated to date. In addition to using a legend that lists the POI categories stored in the knowledge base, it is possible to choose a temporal range and a geographic area through which POIs can be filtered. Additionally, criteria involving specific parameters related to didactic aspects, such as teaching level and disciplines, can be applied. Once a query is executed, a historical map may be overlapped that contains POIs referring to the period selected through the temporal slider. On the bottom, a brief description of the period is recalled. By clicking on a POI, a link to DBpedia is shown with a preliminary description embedded, as shown in Fig. 9.
Finally, starting from a specific POI, semantically related topics can be accessed by simply invoking them from the preliminary description of the POI itself. Fig. 10 shows POIs related to the Veiled Christ (Il Cristo Velato) sculpted by Giuseppe Sanmartino. Four POIs are displayed, each of them sharing a property with the statue, thus creating a  where a terracotta scale model of the Christ by Corradini, the sculptor initially commissioned to complete the work, is preserved, and finally, -Palazzo di Sangro, which belonged to the Sansevero princes. When one of these POIs is selected, historically, geographically and semantically related POIs can be immediately displayed to emphasize their relationships and allow users to capture both different aspects of the same topic and the presence of a feature in different topics/disciplines. Author Name: Preparation of Papers for IEEE Access (February 2017) VOLUME XX, 2017

C. Maps4Learning ARCHITECTURE
As for the underlying architecture, Maps4Learning is a client server platform, as shown in Fig. 11. Clients can be different thanks to the usage of software for building responsive applications. In particular, the following software components have been used: • NodeJS for running JavaScript applications outside the browser, • Angular JS and IONIC to create graphic interfaces for mobile hybrid interfaces, • Apache Cordova for building native applications for mobile multi platforms embedding web applications, • Jena, a Java framework, for building semantic web and linked data applications. HTTP requests are posed to and resolved by Jena through NodeJS. JSON is also in charge of exchanging data between clients and servers.
As for the server, both Altervista and the ones underlying the UNISA e-learning platform have been used. The former is used by students developing modules for topics. The latter is used to embed and provide users with the validated versions of documents.

VI. USABILITY EVALUATION OF MAPS4LEARNING
Usability testing is recognized as a crucial activity to improve the quality and effectiveness of e-learning systems [7]. Usability concerns for such systems include not only traditional properties related to the ease of use of the hosting environment but also to the efficacy of the environment as a learning tool and to students' engagement with the provided content. The final goal of our usability study was therefore to analyse the impact of geoLO+ resources in middle schools in terms of perceived quality, engagement, and student learning performance. The study was carried out in two different phases. In the first phase, we performed an experiment meant to compare the adoption of geoLO+ resources with a traditional teaching method. In the second phase, we assessed the instructors' and students' perspectives, asking questions of both groups of stakeholders.

INSTRUCTORS
The teachers who took part in the study came from three middle schools. The sample consisted of 21 teachers, 16 women and 5 men. All of them had at least 8 years of teaching experience. The subject areas taught by the teachers were history and geography (n= 7), science (n=6), foreign language (n=5), and technology (n=3). Before forming the sample, we ensured that all of the selected participants were well acquainted with computer technology and had a positive attitude towards digital learning tools, having used multimedia interactive whiteboards and/or cloud platforms to share teaching materials.

STUDENTS
The students involved in the study were distributed over 7 middle school classes, chosen on the basis of the selected teachers. Thus, the sample of students consisted of 4 classes from year 2 and 3 classes from year 3, with an overall population of 179 students (92 girls and 87 boys), aged from 11 to 13. As digital natives, all of them were regular web users and had experienced the use of interactive whiteboards and cloud-based platforms for downloading teachers' didactic materials at least once.

B. PROCEDURE
During the first phase of the experiment, teachers were distributed among 7 teams, each with 3 people teaching different subjects but sharing the same class of students. Each team was asked to create a geoLO+ multidisciplinary resource and to choose a leader, who was expected to give an interactive lesson starting from his/her subject (the first subject in Table  III). Most of the participants had never used Moodle, so they were preliminarily trained on the use of the Moodle-based platform Maps4Learning. Specifically, the team of experimenters spent one day training participants on how to create geoLO+ resources through the specification of appropriate metadata attributes and how to assess students' learning progress on related topics through evaluation tests. Then, the teachers' teams were instructed to start by choosing a basic resource (e.g., a document, a video, an image) and specifying its metadata properties while also performing the georeferencing task. Each team member was also asked to add some evaluation tests that could allow student learning assessment on the specific subject each teacher taught. After the creation session, each participant was asked to respond to a survey on the use of the platform to create geoLO+ resources and on the motivation for the adoption of the created geoLO+ inside a classroom lesson, which follows the consolidated approaches available in the literature [17].
To involve students in the experiment, we prepared a written formal consent form, which parents were asked to sign. Then, each class was split into two groups of students of approximately equal size, forming the control group (CG) and the experimental group (EG). The former group received traditional lessons on the chosen topic; the latter took part in an interactive Maps4Learning lesson. In regard to the validity of the experiment, in order to avoid biases coming from the quality of students in each group, students were distributed by their teachers so that the average grades were comparable between the two groups. Considering the chosen topic, the first group of each class received three separate traditional lessons by the teachers of the associated team, and the second group received a comprehensive lesson by the lead teacher using the created geoLO+ resource. All of the students were asked to undertake a class test prepared by the team of teachers. Grades below 6/10 were considered failures.
After evaluating the post-lesson test taken by students, teachers were asked to respond to a questionnaire meant to determine how much they perceived students had learned and the level of the students' engagement with the geoLO+ used while also considering the hosting Maps4Learning visual environment.
Students who were in the second group of each class (forming the experimental group), in turn, had to complete an anonymous online survey about their use of the geoLO+ resource embedded in the given visual environment, responding to questions about the degree of personal engagement and also with reference to the multidisciplinary nature of the explored resource and the degree of difficulty encountered when performing the tests [5], [18].
Overall, it took approximately three months to complete the experiment: from March to May 2019.

C. RESULTS
The experimental results show that a learning improvement can be achieved using Maps4Learning.

THE CONSTRUCTION OF geoLO+ RESOURCES
The first part of the study was conducted in the HCI-UsE usability engineering lab of the Department of Computer Science, University of Salerno. Teams were given two hours for brainstorming to decide the topics they would embedin the geoLO+ resource, they were expected to build and to prepare Author Name: Preparation of Papers for IEEE Access (February 2017) VOLUME XX, 2017 a list of questions that could be used for the learning assessment tests. When the construction session started, an observer sat behind each team, annotating the errors, the time to complete the tasks, and any comments from the participants, who were encouraged to think aloud while collaborating on the tasks. The creation task was divided into two subtasks, the geoLO+ construction and the learning assessment test definition. For the former task, each teacher in the group was asked to interact with the system in turn to specify metadata properties relevant to his/her own subject. The performance of each teacher in the creation of the 7 geoLO+ resources was recorded in terms of task completion time (see Table IV). When asked about it later, all of the teachers judged that the time required to construct the geoLO+ was reasonable, especially as they were all experiencing the system as novice users. In addition, they were aware that they would be able to repeatedly use the resulting learning object again in the future and hence to capitalize on their effort. For the second subtask, to ensure uniformity, we asked each team to build an assessment module made of 9 T/F questions, 3 per subject, chosen from the list they had prepared during the brainstorming session. Table IV summarizes the results of the teacher group performances on the creation of the 7 geoLO+ resources. In particular, the overall time to complete the task was recorded as the sum of the single actions taken.

STUDENTS' LEARNING ASSESSMENT
The second part of the experiment, performed during regular school activities, was focused on students' learning assessment and on their reaction to the use of geoLO+ inside the Maps4Learning platform. Students in each class took the test consisting of 9 T/F questions as devised by their teachers in the first part of the experiment. The test consisted of 3 questions for each of the 3 subjects chosen for the class (see Table IV) 1 . For each class, Table III reports the average scores 1 An example of class questionnaire can be found at https://forms.gle/5cEh6gikUjcCqVXE7 achieved by each group of students on the three subjects and the corresponding success rate. In all cases, a better learning effect was recorded for the group of students who used the geoLO+ resources (EG in table III). Fig. 12 indicates that given a fair distribution of students between the EG and CG groups, a higher success rate was achieved in each class by EG students.
To prove that a statistically significant improvement is achieved in terms of learning effect by using Maps4Learning instead of the traditional method, we conducted an independent sample t-test (see Table V).
Before applying the t-test, we verified that the right conditions existed, namely, we tested for the following: • a normal distribution of data for each CG and EG group using the Kolmogorov-Smirnov test and • a homogeneity of variances tested using Levene's test. As reported in Table V, the Levene test returned an f-ratio value of 0.459, and the p-value was 0.499. That means that the result is not significant at p < 0.05 and that the hypothesis that variances are not equal is rejected.
The hypotheses for the t-test were expressed as: H0: µEG -µCG = 0 ("the difference of the means between the two groups is equal to zero") H1: µEG -µCG ≠ 0 ("the difference of the means between the two groups is not equal to zero").
In the table, t=-4.029 is the computed test statistic, df is the degrees of freedom, and Sig (2-tailed) is the p-value corresponding to the given test statistic and degrees of freedom. Since p < 0.001 is less than the chosen significance level α = 0.05, we can reject the null hypothesis H0 in favour   of the alternative hypothesis H1 and conclude that the mean scores for CG students and EG students are significantly different. This proves that the learning improvement of EG students over that of CG students is not due to the specific case of the sample.

THE SURVEY RESULTS
The teachers' feedback on the perceived teaching efficacy of the proposed tool was considered paramount for deriving a qualitative measure for the tool's validity. Therefore, teachers were asked to respond to a brief questionnaire meant to evaluate students' ability to work in the hosting environment, the perceived learning progress and their engagement with the geoLO+ they used during the experiment. Table VI reports the average responses gained from the interviews carried out with the 21 teachers to understand how they perceived the effect of the proposed geoLO+ on students in terms of system usability, learning progress and engagement. Responses were based on a 5-point Likert scale to derive a qualitative measure of the three considered aspects. To gain insights into how the students perceived the use of geoLO+ within the given environment, we asked the 90 students in the experiment group to respond to a questionnaire, again covering the aspects of usability, learning achievement, and engagement. Table VII reports the average responses received from students. As in the first questionnaire, responses were based on a 5-point Likert scale to derive a qualitative measure.

(0.71)
The bar chart in Fig. 13 summarizes the survey results and compares the means of perceived usability, learning progress, and engagement between the group of teachers and the group of students.

Teachers Students
In all cases, the mean threshold value of 3 (the middle point in the Likert scale used) was exceeded, indicating that the platform was indeed recognized as a valuable tool by both groups of stakeholders.
In summary, the described experimental study allowed us to infer the following: 1. a statistically significant improvement was achieved, in terms of learning effect, using Maps4Learning with respect to traditional teaching methods 2. the learning progress was positively perceived by both groups of stakeholders, i.e., teachers and students, and 3. the proposed platform gained a satisfactory level of user acceptance.

VII. CONCLUSION
The term geo-education is related to education about the world. Its goal is to let people build knowledge about human and natural worlds and interactions between them. To reach that goal, geo-educators stimulate students to engage in geographical thinking, which is based on the examination of the Earth's past and the assessment of the present, thus acquiring information from different points of view useful in studying possible future scenarios. Fostering training on geographical thinking produces an equally important priority effect, namely, that geo-education represents a new approach to facing the "geographically illiterate" state of many people, a critical issue that is due to the shift of priority from geography to the English language and STEM subjects (science, technology, engineering, and math).
Maps4Learning was also developed to pursue this goal. Using Maps4Learning, students can both follow didactic paths by searching among geoLO+ and discover further relationships among them by invoking appropriate semantic web functionality.
The usability testing described in Section VI has also been useful in this aim. It showed that analysing territorial elements, searching information, and studying past events through geography can help students reach a deeper comprehension of world phenomena. Similarly, students who experienced the use of Maps4Learning have had the territory. This can help them memorize connections among topics more easily, thus improving their geographical literacy and preparing them for a geographical thinking model.
The developed system based on geo-information is quite universal and can be meaningfully adapted to any subject. The technology is implemented on specific examples but can be used to train students of various fields and different levels, thereby ensuring the continuity of education.
We also plan to bring this new learning system to mobile devices. Some say that mobile learning is dead because, so far, it has not brought the changes to our formal educational systems that it is capable of bringing. We believe that naysayers arguing as much take a much too narrow perspective. We suggest that a mobile version of the proposed system can allow us to look at learning in ways that supplement and possibly oppose formal education, i.e., learning that can happen anywhere and anytime, that is personalized, situated, and authentic, and that is connected to the everyday use of mobile devices. This is the focus of our current research within the Map4Learning project.