Usability and Accessibility Assessment of Saudi Arabia Mobile E-Government Websites

The rapid spread of smart mobile technology is transforming the way how governments provide information and services to their citizens. We all are rely more on our devices, namely smartphones, laptops, or desktop to get information and a wide range of services from government websites. Such heavy usage of government websites results in an increased need for efficient and effective delivery of government services. Therefore, mobile government websites’ usability and accessibility are essential dimensions that determine the quality and accessibility of mobile e-government. The main objective of this research is to analyze the accessibility and usability aspects of selected public sector websites in Saudi Arabia. This study investigates how well the Saudi mobile e-government websites comply with usability standards and accessibility guidelines recommended in the Web Content Accessibility Guidelines (WCAG). Websites assessments were conducted using manual evaluation and complemented by different automated analysis tools. This study applies a number of evaluation techniques to assess Saudi government websites accessed from desktop or mobile devices, such as an automated website testing technique, mobile-friendliness testing, and content observation technique. Various tools have been used for site evaluation, such as GTmetrix (PageSpeed Score, YSlow Score), WAVE, Google mobile-friendly test, and Dareboost for mobile websites. The study uncovers shortcomings regarding non-compliance to international web standards recommendations. The findings revealed usability and accessibility problems that affect the performance of government websites. In order to improve these websites within these aspects, several recommendations were suggested for improving the usability and accessibility of websites in Saudi Arabia that will make sure different groups of citizens are satisfied with the website features and services provided by them.


I. INTRODUCTION
Advancements in wireless technologies play a major role in the growing number of mobile internet subscribers worldwide. The development of smartphones and associated tools created the possibility of wireless access to countless online recourses through mobile devices. This is the reason behind the increased importance of moving from e-government to mobile government; it is the driving force toward the adoption of m-government [1]. As such, e-government websites should expand their services and design mobile websites to meet the needs of citizens.
Mobile government has the potential to liberate users from the location-and time-related constraints of traditional The associate editor coordinating the review of this manuscript and approving it for publication was Gianmaria Silvello . e-government services. By using wireless and mobile devices, governments can provide citizens with efficient public services and share information within and between agencies. As smartphones are becoming more important in carrying out a variety of functions in comparison to regular phones, citizens are increasingly reliant on their mobile devices to explore and obtain information and services from government websites. Furthermore, as smartphones are increasingly replacing desktop computers to become the primary internet access devices for m-government, it is important to make sure that all government mobile websites are mobile-friendly and provide easy access to citizens to find needed information and services.
Smartphones provide better accuracy and personalization in targeting citizens and delivering content because mobile devices are designed to be used by a single user. The mobile government can improve the quality of life for many previously digitally excluded individuals. It could make the delivery of government services better and reduce their costs; mobile services will reach more citizens and do so faster.
Every citizen has the right to access government websites, including people with disabilities and seniors; therefore, these websites and their content should be accessible, usable, and mobile-device ready for all categories of users. It is essential to make mobile government services more attractive to everyone. The performance of a mobile website as a service platform depends on accessibility and usability. These issues are central in providing reliable and efficient information and services. To stay relevant, governments should take actions and use any opportunity to enhance the usability and accessibility of their websites, which are accessed by citizens via different devices working on different platforms.
When creating websites, the fact that accessibility and usability are significantly overlapping with each other has to be taken into consideration. There is a strong positive relationship between these criteria [2]. Over the years, it has been proved that their quality is crucial to the success of any organization's website. To help with the process of building usable and accessible websites, guidelines for improving these particular factors have been set up.
Usability is recognized as one of the most important quality dimensions that define the success of mobile websites [3]. Mobile e-government services have the potential to improve government performance, quality of services and increase user satisfaction [4]. Therefore, it is in every government's interest to make their public services more efficient and more available [5]. This makes mobile government services a matter of everyone's attention. As m-government services improve in quality and meet the expectations of citizens, the number of users of the government services increases [6]. By making public services more attractive for citizens, the perception of service will shift from being government-centered to citizen-centered [7].
There are several problems associated with delivering mobile e-government services. The most important one is how to make mobile services more convenient to citizens [8]. Governments understand that the use of mobile devices and tools is an immediate challenge [9]. Some governments implemented full or partial m-government services, even though mobile e-government, in general, is still in an early stage in many countries.
Due to a substantial increase in importance, m-government has become a promising field of scientific research. This study aims to provide up-to-date information on the Saudi mobile e-government websites to find out whether they follow the best international standards.
The following two questions arise: 1) Do mobile e-government websites in Saudi Arabia satisfy the WCAG standards? 2) Do they satisfy global usability standards? To answer these questions, we conducted an assessment study to check the state of the Saudi mobile e-government websites over desktop and mobile platforms and assisted in developing recommendations for problems that may arise.
In this work, we investigate and evaluate manually the current accessibility and usability provided by the Saudi government websites, followed by complementing automated evaluation tools of the websites over desktop and mobile devices.
The rest of the paper is organized as follows. In Section II, there is an overview of mobile e-government, its advantages, and the Saudi mobile e-government case study. A literature review of usability and accessibility issues and a summary of the current state of research on evaluation of usability and accessibility of mobile e-government websites are presented in Section III. Section IV describes the methodology used in this research. Discussion on manual and automated evaluation for both desktop and mobile sites is given in Section V. Section VI evaluates the usability and accessibility of mobile websites. Section VII provides a discussion of the results and recommendations. Section VIII concludes the paper, covers limitations, and suggests directions for future work.

II. MOBILE E-GOVERNMENT
Recently, we have observed the rapid evolution of wireless technologies and Internet-enabled mobile devices. There are remarkable advances in the development of mobile applications and services that use these mobile technologies. Moreover, mobile phones are a relatively low-cost technology, so practically everyone can afford it, and their usage is relatively simple compared to Internet technology. Mobile phones have become the most popular and widespread personal user device because of this progress in wireless mobile communication infrastructure, mobile phone technologies, and web technologies. Citizens, businesses, and public agencies are using wireless technologies to do most of their daily transactions. The number of users is increasing dramatically as well. Between 2016 and 2019, the number of global mobile users increased by more than 134%, from 3.79 billion to 5.11 billion, according to DataReportal [10]. The latest data from GSMA Intelligence show that there are 5.16 billion unique mobile phone users in the world today [11]. The number of mobile users in the world population is increasing rapidly; in 2023, according to an international statistic, there will be 7.33 billion mobile subscriptions.
In terms of usage, mobile devices seem to have taken over desktop computers. A survey by WebAIM [12] states that more than 85% of screen reader users are using mobile devices.
Currently, many different mobile devices are available to view a website. They come with different screen sizes and resolutions, have different hardware capabilities and different operating systems, and operate on different network speeds. Only 72% of responsive websites deliver the same resources regardless of the screen size [13]. Therefore, a website should be created so that a user can access it using any device. Regarding website quality, user expectations are starting from performance and ending up with the content. Users are unlikely to revisit a website if they encounter low usability and an unfriendly interface.
Mobile e-government means that the government delivers information and services to its citizens and firms not only by means of desktop/laptop but also by means of mobile devices such as tablets and smartphones [14]. Many look at m-government as another method to deliver governmental services from distributed heterogeneous data sources by means of different transmission channels and technologies. It is regarded as being a flexible provider of public services and applications via mobile and wireless technologies to support citizens, businesses, and all governmental agencies independently of their location.
Mobile government practices are classified as informational, transactional, and operational. Informational functions are one-way transmissions of information from government to citizen. These functions provide governmental information to citizens via publishing and broadcasting. It can also send alerts and notifications to citizens by SMS or e-mails. Transactional functions are two-way transmissions of information from government to citizen and vice versa. These practices allow a user to interact with the m-government system, such as online procurement and payments. Some of the important transaction categories are governmental transactions, such as submitting and following up applications and querying records. Sources of these data are distributed over many systems at ministries that may use different software applications and apply different restrictions on accessing data. In other words, the data sources are heterogeneous in the structure and naming conventions. Mobile technologies, or what is called mobility nowadays, offer a lot of services and applications that enhance our lifestyle. Operational functions aim to handle the internal governmental operations that facilitate government employees' access to important information on remote locations using their mobile devices. The use of e-government tools and techniques delivers its services to businesses, its employees, and citizens through different types of interaction.
One way to improve the user experience is to allow them access to government websites in the ways they prefer, whether through a desktop, a smartphone, or a tablet. To make it happen, the website design should be more accessible, usable, and portable. A design of a website page should be adopted based on how a user is accessing the page.
Some organizations maintain two versions of their websites, the desktop/laptop, and the mobile version. Having independent dedicated mobile websites can provide fast page loading and optimal browsing experiences to mobile end users [15]. At the same time, mobile versions of government websites are easy to access, so everyone can benefit from using them, and a much wider audience can be reached. However, this attractive solution generates overhead costs for organizations because of extra costs and effort for developing and maintaining multiple sites. Traditionally, desktop websites are designed to display content to fit defined screen sizes and specific resolutions. A new web architecture called responsive web design (RWD) is known as a one-site-fits-all approach regardless of screen size and resolution [16]. The RWD uses viewport [17], which decides on a visible area of a web page based on a screen size of a device used to access the page. Such technology allows users to change an appearance of a website dynamically, depending on the user's screen size and orientation of the device being used, to view specific content. This new technology avoids specific content from being displayed when a viewport gets small.

A. ADVANTAGES OF M-GOVERNMENT
Mobile versions of e-government websites have distinctive differences when compared with desktop versions. There are relative advantages of m-government over e-government, which is a multi-dimensional variable with two main key characteristics: mobility and customization [18]. Relative advantages were expanded into four main key characteristics: mobility, localizability, personalization, and security [19]. It seems that a mobile version of the e-government website is attracting more citizens than the desktop. Mobile government has several advantages in terms of providing affordable services to users [20]. From citizens' point of view, the main advantage of m-government services is mobility. It provides citizens with access to a network independent of time and location, which will contribute to saving effort and time. Mobile government can offer personalized services to satisfy personal requirements, preferences, and interests [21]. Furthermore, operating a mobile phone is easier and does not require special training, as minimal knowledge is required to operate m-government services. The presence of mobile websites increases the desire of citizens to engage with e-government because it is easier for them to gather needed information [22]. Citizens can instantly receive messages from government service providers. Being able to access government services via mobile devices might be the best way for them [23], and it means they neither have to visit a service provider in person nor go home to use their computer when they want to access government services and information [24]. The mobile government can provide more dynamic personalized information and services based on user requirements and location (mobile phone technology provides location-based services), which creates better conditions for the delivery of personalized services. It also makes citizens aware of the information presented and appropriate actions to be taken; for example, it can send citizens early warning information when necessary [25].
In spite of all benefits and advantages discussed so far, implementing m-government is complex. Interoperability of various platforms, accessibility, and usability are some of the challenges that the implementers of m-government are facing. It takes a lot of resources and effort to establish an excellent mobile e-government website that will be accepted by citizens for information, services, and transactions purposes. The success of mobile government will depend mainly on how easy and convenient it is to access government websites to get information and services.

B. MOBILEE-GOVERNMENT IN SAUDI ARABIA
More and more governments have developed and continue to develop mobile websites and consider them a new alternative means of providing public information and services [26]. Some countries have not yet launched their mgovernment, and its potential is largely unexplored [27] due to infrastructure problems or regulatory standards. Meanwhile, others have been actively working on its implementation. Studies showed a positive attitude toward the adoption of m-government services in the future and investigated the drivers and barriers of adoption of e-government services in India [28], [29]. The United States has been implementing a number of m-government platforms in different sectors [30]. China has been providing a number of its e-government services via mobile technologies. One report investigated the extent to which the future use of mobile apps is possible among Chinese users; for example, m-tax is a new service frontier in China [31]. Taiwan has adopted m-government as early as 2004 [32]. Greece supports m-government through a number of initiatives like Pythagoras [33]. Mobile government services, such as submitting maintenance complaints via text message, are available to citizens in the United Kingdom [34]. Dubai has launched an initiative known as m-Dubai where information and services are provided to citizens via mobile phones. However, m-government development has remained uneven across the globe due to differences in the infrastructure of mobile technologies and the magnitude of e-government implementation in a country.
Saudi Arabia started the implementation of m-government in 2013 [35]. In Saudi Arabia, m-government is considered a supplementary approach for the delivery of governmental services from distributed heterogeneous data sources using different transmission channels and mobile technologies, services, applications, and devices. We witnessed significant growth in the number of mobile phone subscribers over the years. The number of smartphone users in Saudi Arabia is expected to rise from 22.34 million in 2017 to 36.17 million in 2025 [36]. According to the Ministry of Communication and Information Technology [37], the Kingdom has the highest number of mobile phone users per capita in the world. In 2020, Saudi Arabia moved up in the E-Government Development Index to the 27 th spot globally [38], compared to the 44 th spot in 2016 [39]. With an increasing number of users accessing the web via smartphone devices in Saudi Arabia, it is important that the government ensure that users all over the country can reach e-government information and services on these devices. The Saudi government provides a multitude of e-services that are delivered through m-government websites to distribute information and services. Saudi Arabia is a fairly big country, so by using mobile government services, citizens will not have to travel to the main cities if they need to contact ministries. Networks and related infrastructure reach everywhere in Saudi Arabia, even the small villages. For accessing m-government services, there is a mobile online portal (https://www.my.gov.sa/), which displays services available for citizens. Mobile e-government has a number of benefits that are relevant to Saudi Arabia. The public sector especially will appreciate low delivery costs of government information and services, reduced administrative costs, a larger number of citizens being able to reach governmental services, quick access to integrated data, increased productivity, and effectiveness of public services. Benefits also will include personalized and active household services, such as location-and device-based storage.
A useful website should provide interactive forms, online applications, and interaction functions; in addition, a website should be able to convince users that it is a trustworthy source of information. Users are more motivated to adopt mgovernment services if there is evidence of the credibility of such services.
Mobile e-government websites in Saudi Arabia are becoming more important as a source of information for the majority of citizens. Hence, there is a need for these websites to be usable and accessible to people regardless of their age, abilities, and technical experience. This study has taken an in-depth usability and accessibility evaluation of the mobile e-government websites in Saudi Arabia.

III. LITERATURE REVIEW
There are a number of factors that affect the ability of citizens to use websites to their advantage. Accessibility and usability are just two of them.

A. WEBSITE USABILITY
Usability is the most important factor in convincing users of an e-government website credibility [40]. The quality of a government website is critical for most citizens and companies seeking specific information or services. Usability refers to how easy it is for anybody to use a website and access the requested content effectively and efficiently. It is also about the simplicity of navigation between pages, allowing users to search for specific pages or sections without any difficulties. If the website usability is poor, citizens may not be able to find the information they require [41]. On the other hand, high usability has a positive impact on user satisfaction, trust, and loyalty toward the website.
Usability focuses on these main parameters: 1) Efficiency: The speed of completing a task that affects how fast and easy navigation is on a website 2) Effectiveness: Accuracy and completeness to achieve indicated goals by user 3) Satisfaction: The level of user enjoyment while using the website 4) Understandability: The level of understanding and the possibility of learning faster, and ease of remembering, as well as tolerance for errors There is a strong positive relationship between perceived usability and credibility of government websites [42]. The usability of e-government websites is vital for transactional user experiences, especially in two-way communication, requesting information, and providing feedback, thus VOLUME 9, 2021 allowing users to navigate these websites effectively. The usability measurement is a series of steps to evaluate efficiency, effectiveness, and user satisfaction. Websites were investigated for quality factors, such as simplicity, effectiveness, privacy, responsiveness, learnability, governance, simultaneousness, organization, memorability, and announcement issues [43]. The study stated that the usability factors could be very helpful when improving the design of mobile e-government services and assist in the creation of easy and intuition-driven applications that can be used by citizens regardless of their digital skills without any instructions or help. When the layout and components of a website are coherent, users can often navigate the site straightforwardly. From the government's point of view, good usability aids in reducing development and maintenance costs and maintaining higher customer retention [44]. For identifying potential problems, usability must be tested, which is difficult and usually done manually, to learn how easy it is for any category of users to find the necessary information or desired services available on the website; or it can be done using analytics tools. Based on the opinion of software developers, usability evaluation can be performed with the help of usability metrics. Other options are also available for carrying out evaluations in e-government. It could include the use of expert evaluators, heuristic-based studies, or focusing on user expectations. The heuristic-based studies based on guidelines generated from research-based best practices [45].

B. WEBSITE ACCESSIBILITY
Accessibility allows users to perceive, understand, and interact with websites and their services regardless of physical limitations [46]. It implies that such categories of people can easily observe, comprehend, navigate, and interact with websites [47]. Web Content Accessibility Guidelines 2.1 (WCAG 2.1) includes all criteria for WCAG 2.0 with 17 additional criteria on mobile accessibility, for people who are blind or visually impaired, and for people with learning issues and cognitive limitations. The WCAG 2.1 is a recommended new version of web accessibility, which is an internationally accepted standard. WCAG 2.1 [48] founded on four principles of accessibility. The main four accessibility principles recommendations [49] are: 1) Perceivable: Information and user interface must be presentable to users in ways they can perceive. 2) Operable: Users are able to interact and navigate with the possibility to browse content comfortably. 3) Understandable: Users must be able to understand the information and the operation of the interface. 4) Robust: A wide range of users can interpret content reliably.
Enhanced accessibility of websites allows elderly and people with disabilities independent, full or limited, effective access and browsing through content and functionalities comfortably [50]. The Web Accessibility Initiative (WAI) considers mobile accessibility as making websites more accessible to people with disabilities when they are using a broad range of devices, such as smartphones or tablets, to interact with the web [51]. It is not only limited to people with disabilities. Improving website accessibility could also generate positive reactions from other mobile e-government users, who constitute the majority of Internet users. Better usability of a website means automatically improved accessibility for all categories of users [52]. This fact will motivate governments to invest more in accessible website design. Other users may need an enhanced accessibility approach because of the following: using small screens, partially sighted users, users suffering from color blindness, and movability issues related to fingers. These issues can affect site readability to a greater extent.
There are quite a few accessibility issues [53]: 1) Missing alt text on images: It is a word or phrase that tells users the nature or contents of an image. 2) Low contrast on the text: It can be hard to read text that has a low contrast ratio or text with brightness too close to the background brightness.

3) Missing link text. 4) Ambiguous link text: Link does not provide informa-
tion to the user about what it is for or where it leads. 5) Too many navigation links. 6) Empty form labels: The link contains no text. This can cause misunderstanding for screen reader users. 7) Unclear form controls. 8) Uncontrollable time-outs: Users will not be able to know when the form is about to expire.
The most common among them are low contrast on text, missing alt text on images, missing link text, and ambiguous link text.
Accessibility investigation focuses on testing if a website is accessible to people with disabilities and can be used on different devices.
The assessment of website perception done according to the three priority levels of conformance with the WCAG 2.1. Basic (A): a developer must satisfy priority 1 guidelines to make a site accessible; extended (AA): a developer should satisfy priority 2 to make a site accessible; and full (AAA): a developer may address priority 3 to make a site accessible [48].
Recently, the Accessibility Guidelines Working Group (AGWG) has been working on the development of WCAG 2.2. The official release of WCAG 2.2 standards was initially expected in November 2020, but it still remains in draft status. Now the release is expected in summer 2021 [54]. The WCAG 2.2 extends WCAG 2.1 by adding new success criteria and guidelines that address the needs of people with cognitive disabilities, users of eBooks, and users of mobile devices. Draft criteria address the following: 1) Accessible Authentication: Login and access to content should be secure, easy to use, and accessible. For people with specific cognitive disabilities, remembering a password is a problematic issue. Therefore, other authentication methods must be available that do not require the user to remember, manipulate, or transcribe information. 2) Dragging: To ensure functionality that uses a dragging movement, there is another single pointer mode of operation without the need to drag elements. It applies to web content that interprets pointer actions. Users with mobility impairments can use a pointer to benefit from this functionality. 3) Findable Help: A mechanism provides direct help for completing tasks on the page or transfers a user to a different page containing the requested information via a direct link. 4) Fixed Reference Points: A mechanism for a web page or a set of web pages with page break locators navigates to each locator and maintains its place in the flow of content. 5) Focus Appearance (Minimum): To ensure a keyboard focus indicator is clearly visible, a keyboard focus indicator may have parts that do not meet the 3:1 contrast ratio for the change of contrast, as long as an area equal to the minimum does meet the contrast ratio. 6) Focus Appearance (Enhanced): A keyboard focus indicator must be highly visible. It is closely related to Focus Appearance (Minimum) and defines a higher level of visibility for the focus indicator. 7) Hidden Controls: People with cognitive disabilities, like impaired memory, may not be able to find controls needed to progress if they are hidden until focus is placed. This success criterion ensures that controls needed to progress or complete a process can be easily found when they are needed. 8) Pointer Target Spacing: To ensure targets can be activated easily without accidentally activating another target close to it, bigger spacing between targets should be provided. 9) Redundant Entry: This technique ensures users can successfully navigate multi-step processes. Assessing whether a website has good enough accessibility is a challenging task. Usually, accessibility evaluation is conducted manually by human judgment (specialists or users) to check its compliance with accessibility standards, especially for crucial cases. A manual approach requires a considerable effort in terms of time and budget, thus often resulting in insufficient evaluation [55]. One of the best solutions to this problem is to automate the evaluation process, which is faster and achieves better reproducibility and accuracy [56]. There are several tools to check the accessibility of mobile websites, including Screen Readers, Color Ratio Analyzers, HTML Validator, Dareboost, and WCAG 2.0 Checkpoints [57]. This research focuses on the assessment of website perception according to the WCAG, and compliance of Saudi government websites accessibility with WCAG for desktop and mobile devices.

C. RELATED WORK
Much effort has been spent on the usability and accessibility evaluation of e-government websites in different countries. Researchers used different methods and approaches to carry out website assessment. Some conducted both accessibility and usability studies, while others focused only on one of them. Evaluation of the websites' content, accessibility, usability, and mobile readiness, which used a combination of automated evaluation tools and manual inspection was performed in Alabama. Results indicated that websites are not in full agreement with accessibility, usability, and mobile readiness standards [41], [42]. A study conducted in Poland [58] investigated accessibility of public sector websites using automated and manual testing tools. That study indicated that there is an urgent need for improvement in accessibility and that most of accessibility problems are related to content instead of technicalities. Governments of the United Kingdom, Germany, and Denmark have established e-government usability standards [59]. The automatic accessibility tool was used in accessibility investigation of the Slovenian public government websites that lasted two consecutive years [60]. The main objective is to check whether the websites complied with the European Union Standard EN 301 549. The results of the analysis showed accessibility improvement for that period. A framework for performance evaluation of Chinese government websites from both the usability and accessibility aspects was proposed in [61]. The developed automatic assessment system is capable of extracting data from a government website and producing the evaluation report. The accessibility of Korean government and public agency healthcare websites were tested by disabled and non-disabled users. The study revealed mobile accessibility issues and problems for people with visual impairment [62]. An investigation of selected Turkish e-government websites has been conducted to assess whether they are following standards of accessibility, usability, performance, and mobile readiness. The findings from this study indicated that tested websites perform poorly with regard to the above criteria [63]. An examination of website usability and accessibility was conducted using automatic evaluation tools bearing in mind people with disabilities [64]. The findings showed that the users of desktop devices have a better experience than mobile device users and indicate that the accessibility is strongly dependent on page load time and page size. In Bangladesh, the usability of e-government websites was evaluated using heuristics methods [65]. Results indicated that tested websites had severe usability problems. Investigation of accessibility and usability of 30 Pakistani government websites for compliance with WCAG 2.0 demonstrated that these websites needed improvement in accordance with accessibility and usability standards [66]. A study of usability and accessibility of e-government websites in Libya that implement heuristics methods indicated that tested websites had significant usability and accessibility problems [67]. In Nigeria, an investigation was conducted to check the level of accessibility of e-government websites with the help of automated VOLUME 9, 2021 tools [68]; results reported that tested websites were not in conformity with WCAG 2.0 standards. A number of studies evaluated selected government websites by applying different automated accessibility and usability testing tools (WAVE, Google's mobile-friendly, GTmetrix) to analyze the usability, accessibility, security, and mobile-friendliness of public sector websites, to determine if websites meet WCAG and usability standards [69]- [73].
The accessibility of government websites in Jordan was examined manually and using an automated checking tool [74]. Results indicated that most websites did not meet the WCAG 2.0 accessibility requirements, and all websites had serious accessibility problems. Accessibility of government websites in 10 Arabic countries was examined to assess the Arabic and English versions using automated tools [75]. According to their results, some websites have good accessibility in English versions while others in Arabic versions.
Mobile services have their own unique characteristics such as mobility, portability, location, and personalization [14]. Reference [6] provides a framework that can be used for the assessment of m-government service quality. A comparison study of mobile services of the Ministries of Interior conducted in the United Arab Emirates and Saudi Arabia reported that mobile services provided by both ministries are technically oriented while tangible services are the last priority. Evaluation of m-government services quality was conducted based on four parameters: connectivity, interactivity, understandability, and authenticity [76], [26]. Reference [77] examines key determinants of mobile government attractiveness and citizen's perspective of mobile government usage in public administration. A study [32] investigated factors (perceived mobility, interactivity, psychomotor, and cognitive aspects) that encourage a person to use mobile e-government websites in Indonesia. An empirical study to evaluate the usability of mobile e-government was presented in [78]. There are previous studies of using, adoption, citizen engagement, and issue of mobile e-government acceptance in Saudi Arabia [79]- [83], while very few of usability and accessibility of government websites. Al-Khalifa [84] examined 36 Saudi websites of public sector bodies based on accessibility requirements. As a result of this research, no websites were found able to complete the WCAG 2.0 conformance test in 2010. Al-Khalifa et al. [85] in 2017 reexamined the Saudi websites of public sector bodies. Analysis has shown that over the past five years, the number of violations of WCAG 2.0 rules has been drastically reduced, and more websites have supported mobile appearance. However, there is still room for improvement.

IV. RESEARCH METHODOLOGY
There are different approaches to the evaluation of website usability and accessibility: heuristic evaluation, automated testing, user testing, expert evaluation, surveys, and policy analysis [86].
This study implements expert inspection and conformance testing strategies by utilizing a mix of manual inspection and automated evaluation methods in an effort to triangulate findings, provide up-to-date information on the Saudi government websites and develop suggestions for problems that may be found. To conduct the assessment, we followed the steps as shown in Fig. 1.
The Saudi governmental websites evaluation was performed in two stages. The first stage involved two in-depth manual empirical assessments of each investigated ministry website conducted at two different time points (i.e., March 2015 and January 2020). The second stage was completed in June 2020 using automated tools for testing website usability and accessibility based on WCAG recommendations.
The sample selection process was based on the government website profile, functionalities, and type of services provided to citizens and businesses. We gathered information about these websites from the Saudi Unified National Platform [87]. These websites have diverse purposes and designs and are very popular with the majority of users. This distinctive selection may help ensure that the majority of accessibility and usability guidelines are covered and assist in identifying as many violations as possible. However, due to security restrictions, some of the Saudi e-government websites cannot be accessed by automated tools. Therefore, these sites were excluded from our sample. The sample consists of 22 government websites that represent a wide range of sectors including various ministries such as Education, Health, Human Resources and Social Development, Environment, Water and Agriculture, Justice, Foreign Affairs, Interior, Finance, Commerce, Economy and Planning, Communications and Information Technology, and Housing.
We conducted a manual assessment of the usability and accessibility of each website, followed by an automated evaluation. Automated tools are translating user requirements or needs into technical requirements, with specific implementation. They have several advantages compared to manual methods, such as suitability for large-scale evaluation and less effort in terms of time [88]. Using automated tools provide unique help in testing a large number of websites, efficiency in finding types of existing problems, and occurrence of each type of problem [89]. However, it is essential to note that automated tools cannot completely replace manual testing.
They can find a problem but cannot show how severe the issue is. Usability and accessibility testing should not focus only on the home page. Usually, problems that appear on the home page also show up on other pages of the same website [90].

V. ASSESSMENT OF USABILITY AND ACCESSIBILITY OF SAUDI ARABIA MOBILE E-GOVERNMENT WEBSITES
In this section, we investigate the status of usability and accessibility of websites at ministries and governmental agencies in Saudi Arabia. The criteria for evaluating these websites are expressed as an in-depth questionnaire designed to find out the websites' current state of compliance with international standards.
Our questionnaire was designed to evaluate the accessibility and usability of the e-government websites of Saudi Arabia manually. The first manual assessment was completed in March 2015; the second manual assessment took place in January 2020 using the same questionnaire. In order to assess the overall usability and to find out the usability problems of e-government websites, we used a heuristic evaluation method (expert-based evaluation method) to discover the inadequacies and problems in a user interface design.
A variety of questionnaires have been proposed and used in the literature for evaluating the usability, including Questionnaire for User Interface Satisfaction (QUIS), System Usability Scale (SUS), Computer System Usability Questionnaire (CSUQ), Website Evaluation Questionnaire (WEQ), Microsoft's Product Reaction Cards, Website Analysis and Measurement Inventory (AMMI), among others. An extensive review of these questionnaires was presented in [91]. The usability part of the questionnaire is based mainly on the Website Evaluation Questionnaire (WEQ), which was specifically designed for the usability evaluation of governmental websites [92]. These are the main aspects of the questionnaire: information should be easy to find, the content should be easy to understand, and the layout should be clear and should support the users' adequate task performance. To measure the quality of each aspect, the correct relevant questions should be asked. The usability part of the questionnaire consists of the following categories: general, design, content, navigation, and performance. Therefore, the components of the questionnaire consist of four parts: usability, accessibility, mobile readiness, and messaging.
The study intends to assess the following significant issues: 1) How well do Saudi government websites meet usability and accessibility guidelines? 2) How mobile-ready are existing websites?
In addition, we will evaluate the quality of the website, including performance, link quality, and website accessibility for users with disabilities.
As we have mentioned previously, e-government informational functions can provide end-users with published online information and send alerts and notifications to users, while operational functions enable government employees to access any needed information from remote locations.
Website assessment attributes were categorized as follows: 1) Usability: It measures how well the website provides information in an effective and efficient way. It relates to issues such as general information (easy navigation between pages), screen design, content, performance, and navigation. Navigation is a critical factor in the quality and, consequently, the success of a website. This category should consider different classes of content such as text content, image content, audio content, video content, advertisements, etc. 2) Accessibility: It checks for compliance with WCAG, choosing of color and contrast, support for people with special needs, zooming, etc. 3) Mobile readiness: An independent website would be available for desktop and mobile users, mobile services, and additional services such as downloadable maps, m-books, m-brochures, location-based services, news, etc. 4) SMS: Message notifications include sending messages and receiving notifications from ministries, for example, disseminating emergency information.

A. MANUAL INSPECTION
The first manual assessment in March 2015 was completed by our own experienced professional users. Its results are outdated, and some did not specifically focus on websites of public sector bodies. The second manual assessment in January 2020 used the same questionnaire, and it was completed with the help of Ph.D. students taking full-time courses in management information systems. The participants were asked to interact by the guidelines stated in the questionnaire that was the target of the research. After the task is completed, participants are asked to fill out an evaluation questionnaire. The results of both evaluations are shown in Table 1.
All items were measured using a five-point Likert scale [93], with ''strongly disagree'' (1), ''disagree'' (2), ''do not know'' (3), ''agree'' (4) and five points for the statement ''strongly agree''. Then each score was calculated as the average of all heuristic elements for each evaluation feature of each ministry. Table 1 displays descriptive statistics of usability, accessibility, and mobile readiness of analyzed Saudi government websites for two stages.
In the first assessment, performed in 2015, participants perceived information provided by websites in an understandable, clear, and easy to remember manner, with an acceptable loading time for each page. More than half of the websites had accounts on social networks. On the other hand, this study identified substantial problems in both usability and accessibility. The most violated usability problems of evaluated government websites in Saudi Arabia were the quality of interactive communications and data entry forms. User experience was such that websites provided links to informa- tion, where other links were available to the same information provided by the e-government websites. Results showed that most of the websites of examined Saudi public sector bodies did not have complete accessibility compliance. Most of the websites have at least some violations of WCAG (mean = 2.7 and SD = 1.81). During an accessibility investigation, we found that almost all websites contained empty links, used text equivalent for non-text items, images that did not have alternative text or buttons without descriptions. Most of the websites contained links referred to by an adjacent element.
Participants of the first evaluation study were not completely satisfied with the integration of Saudi e-government websites (mean = 2.2 and SD = 1.88). Our study also found that fewer than half of the websites appeared to offer website mobile views. Regarding mobility, results showed that 9 (40%) of the 22 tested websites used responsive services for mobility. A few contained independent design for tablet or smartphones (mean = 1.5 and SD = 1.05). No website at that time provided any extra services such as downloadable maps, mbooks, or m-brochures; they also did not offer an interactive capability through SMS. Clearly, websites were not as mobile-ready as one would hope.
Most websites failed to meet accessibility standards regarding users with disabilities. Results showed that support for people with special needs is below satisfactory for most e-government websites. As exhibited in Table 1, in the first evaluation, most of the 22 websites did not have their user interfaces optimized for people with disabilities. Therefore, they did not meet accessibility standards to facilitate access to government information and services that people with disabilities required (mean = 1 and SD = 0).
In the second evaluation in June 2020, the results showed that the usability and accessibility of ministry websites significantly improved. Results showed quick access to information on the websites (i.e., significantly improving efficiency). Websites provide very good tools (mean = 4.1) to assist users to move across pages and reach content quickly and without trouble. All had a primary navigation bar or tools at the top of a page. In terms of mobile readiness, all websites were augmented with mobile versions, and most of them were mobile-friendly and performed reasonably well. Most evaluated websites (84%) used responsive services for mobility, but only a small number of them provided extra mobile services, including downloadable maps, m-books, and m-brochures. Results of the second evaluation showed that the percentage of websites following the WCAG increased from 54% to 71%.
The second evaluation showed that most websites did not optimize their user interfaces for people with disabilities, which is similar to the results of the first evaluation. So again, websites did not meet accessibility standards to facilitate access to government information and services that people with disabilities required. Furthermore, none of the websites offered an interactive capability through text messages (SMS).
Providing user help caused improvement of websites usability and accessibility as the results of the evaluation in June 2020 showed that 73% of the websites now offer user help compared to 38% in the first evaluation. In general, a comparison of both government website evaluations showed significant advances in website quality.
We made several suggestions to address the major detected problems, to help website developers and administrators improve and enhance the accessibility and usability aspects of these e-government websites.

B. AUTOMATED EVALUATION
Automated online evaluation tools were implemented to test the robustness of the usability and accessibility of websites. A large variety of such tools is available on the Internet for evaluation of both desktop/laptop and mobile website usability and accessibility. In our case, the selection of the tools was performed based on test specifics and evaluation types. Evaluators choose tools according to their ability to assess and identify potential usability and accessibility issues as well as measuring the level of adherence to the desired web standards. We completed the evaluation of 22 websites of ministries and public sector bodies of Saudi Arabia. The measurements were carried out in different test settings for desktop and mobile platforms.

1) DESKTOP USABILITY TESTING
There are a number of tools for checking the usability of a website [94]. The following criteria were used for selecting usability tools: 1) The tool reports quantitative web performance metrics (load time, number of requests, and page size).
2) The tool can evaluate desktop and mobile devices.
3) The tool has different test server locations. 4) The tool gives recommendations on how to improve the performance of the evaluated webpage. Usability tools that match our selection criteria are GTmetrix, Lighthouse, PageSpeed Insights, Dareboost, and Test My Site. In our case, we found that GTmetrix is superior for desktop, while Dareboost performed better for the mobile website.
The online usability testing tool GTmetrix [95] is concerned with page speed analytics. It is based on Google's PageSpeed [96], Yahoo's YSlow [97], and specific performance indices (fully load time, total page size, and the number of HTTP requests). Google's PageSpeed and Yahoo's YSlow are tools that suggest best practices for optimizing web performance. The tool provides page performance analysis reports, page loading speed, suggestions for improvement, and overall performance scores. The PageSpeed score is the percentage of PageSpeed recommendations that a website can fulfill [98]. In addition, the tool provides a PDF version of the report. The practices can be applied well when developing websites for desktops. When targeting mobile devices, the performance was noticeably different for some of the practices. With YSlow, the tool works by crawling a website and comparing it against a list of 23 rules based on Yahoo's rules for high-performance websites. Then YSlow scores the website against these 23 rules and gives users an overall score based on the average. The grading scale is 0-100, where a score between 90-100 is an A, 80-90 is a B, and so on. With GTmetrix, the tool displays a score of the tested website showing the compliance percentage of each rule and compares them with the average values.
Fully load time option calculates the total time (in seconds) needed to load the complete content of a page. The total page size and number of HTTP requests are directly proportional to the page load time. Large page size or large number of HTTP requests will increase the page load time.
All 22 websites of ministries and public sector bodies were diagnosed, one by one, using GTmetrix. The Chrome browser (desktop/laptop) was selected in the GTmetrix options. Using GTmetrix is very simple. To begin the investigation, the name of the website must be entered on the website of GTmetrix. Results were manually copied into a spreadsheet and include recital scores of Google PageSpeed and Yahoo YSlow. Other   factors included were the page details, which are fully loaded time, total page size, and number of requests.
The results were analyzed. Table 2 (given in the Appendix) shows results for each factor rated excellent, good, average, below average, and bad.

a: PageSpeed ANALYSIS
The investigation revealed that none of the examined websites achieved excellent nor good. Nine of the websites were able to achieve average and below-average ratings, while more than half of websites reached the worst level of compliance. The most common PageSpeed problems are listed in Table 3. Table 2 shows how many websites passed the rules of YSlow recommendations. One website was able to achieve a good rating. Eight sites scored average, and 12 sites scored below average results. A rating of ''good'' means that the website had fulfilled most of the YSlow recommendations. None of the websites received bad ratings. The most common warnings are listed in Table 4.

c: FULLY LOADED TIME ANALYSIS
Website success is strongly associated with website download delay. Fully loaded time specifies the total number of seconds that are needed for the whole page content to load. Examination of websites' total load time is shown in Table 2. The study discovered that 12 (54%) websites achieved an excellent score with a load time up to 5.5 seconds, followed by 1 achieving a good ranking, and 3 websites achieving below average results.
The last 6 websites achieved bad results, where the loading time was more than 9 seconds.

d: PAGE SIZE ANALYSIS
The total page size of tested websites was examined, and the results are displayed in Table 2. The results showed that 2 websites (9%) achieved excellent results with a page size less than 2MB, followed by five websites with a good ranking. Seven websites received average and below average. Eight websites were graded as bad, where the increase of the page loading time may have been caused by a page size that exceeded 6MB.

e: HTTP REQUEST ANALYSIS
The numbers of HTTP requests for all websites are listed in Table 2. Our results show that six websites got excellent results with fewer than 69 HTTP requests, followed by one website that received good results, and three websites achieved average and below average results. Half of the tested websites received a bad score where the number of requests exceeded 120. A slow website negatively affects the user experience.

2) DESKTOP ACCESSIBILITY TESTING
There are several methods to check the accessibility of a desktop/laptop website. Web accessibility evaluation tools help to determine if websites follow accessibility guidelines. To study accessibility, the websites were checked for compliance with WCAG.
Accessibility testing focuses on checking if a website is accessible to people with disabilities and if it can be used by them. At present, there are different evaluation tools available for checking a website's accessibility, such as TAW, WAVE, Web Accessibility Checker, AChecker, Cynthia Says, Accessibility Valet, FAE, MAGENTA, and OCAWA. Some of them are listed on the W3C's recommendation Web Accessibility Evaluation Tools List [99]- [101]. All of the tools have numerous diverse functions for evaluation needs. They follow the guidelines of WCAG and other standards. These accessibility evaluation tools highlight the design, content, and other accessibility issues. To verify the accessibility standards, we selected WAVE to check whether evaluated sites use an integrated or an isolated accessibility solution. One of the most commonly used free online tools, WAVE [102], allows investigators to evaluate the accessibility of websites. It was developed by WebAIM [103], a non-profit organization. As a result of the continuous updating of the software, a large number of WCAG 2.1 [104] can be checked with WAVE's help. It is unique because it gives a straightforward report of accessibility violations and indicates the instances of each accessibility problem. The tool produces very detailed results in terms of detection of the number of errors, contrast errors, alerts, features, and structural elements. The documentation section provides enough information on each error and warning; the information also includes which WCAG covers it, why it matters, and a recommendation on how to fix the error.
The WAVE online checker has several limitations, such as lacking the ability to test the entire website, working only with one page at a time, and not being able to export results. We carried out complete website examinations of the 22 ministries and public sector of the Saudi government. In order to begin the inspection, the name of the website must be entered into the WAVE website. The results were entered manually into a spreadsheet consisting of the following criteria: contrast, errors, and warnings. Results for each factor were analyzed, rated (excellent, good, average, below average, or bad), and summarized in Table 5.

a: ANALYSIS OF CONTRAST
The low contrast of website page text is the most common accessibility issue. Text that has a low contrast ratio or text whose brightness is too close to the background brightness can make it hard to read. While this issue is particularly challenging for people with low vision, low-contrast text can  negatively affect the reading experience for all users. Table 4 shows that six websites (27%) tested excellent with up to 6 (27%) low contrast occurrences on the webpage, 6 had good grades, while 7 (31%) rated as average and below average. One site had real problems with a very high number of low contrast occurrences.

b: ANALYSIS OF ERRORS
As shown in Table 5, one site with the number of errors in the range (0 -4) rated as excellent. Five sites had a good rating with 5 to 11 errors. Four websites contained up to 20 errors. More than half of the sites (13 sites) achieved average and below average rates, and three sites had more than 30 errors. The most common errors are listed in Table 6. Table 5 shows the WAVE verification result of the warnings. The results point out that three sites achieved excellent ratings and did not have any warnings, while all other websites were issued a warning(s).

c: ANALYSIS OF WARNINGS
Eight (36%) websites had relatively few warnings. Seven websites (31%) contained up to 45 issued warnings, which could lead to inadequate users' perception of the site's quality. Three sites had more than 46 warnings. The most common warnings received during the investigation are shown in Table 7.

3) MOBILE FRIENDLYNESS
A mobile-friendly site means easy to view, readable, immediately usable, and provides an encouraging experience for anyone who tries to visit the mobile website on a mobile device. VOLUME 9, 2021  If not mobile-friendly, a site can be difficult to view and use on a mobile device. A non-mobile-friendly site requires users to pan and zoom in order to read the content. Users may find this to be a frustrating experience and are likely to abandon the site. Fully mobile-friendly websites can adjust their layout to the viewing device so that users on different devices can view the same website with an improved experience.
To ensure a web page is designed to fit a mobile device screen, Google offers a tool to test a mobile website to find if it is mobile-friendly [105]. If it is not, Google will provide reasons as to why not. Google has criteria for mobilefriendliness: (1) content is sized to the phone screen to minimize scrolling horizontally or zooming; (2) web page text is understandable without zooming; (3) software, uncommon on mobile devices, like Flash, cannot be used; and (4) links are adequately separated from each other so that the correct one can be easily selected. A mobile-friendly testing tool is not concerned with usability regardless of the device on which it is being viewed.
Most of the Saudi mobile government websites are fully mobile-friendly (86%) and passed the Google mobile-friendly test for the viewports of smartphones and tablets. Among these passed websites, only 4 achieved average and below average results. The remaining websites (14%) did not pass because of the following issues: 1) Touch elements are too close together. Buttons and links should be far enough apart to be easily usable and without the risk of accidentally touching the wrong button or link. 2) Text too small to read. 3) Content wider than screen. Some content container on the page was forced to a width larger than the device. 4) The viewport was not set. For a mobile-friendly site, the viewport should be set to adjustable width to be presented by the device.

VI. USABILITY AND ACCESSIBILITY OF MOBILE WEBSITES
Mobile website testing is still in its initial stage, but with the support of accurate mobile testing tools, website testing will become easier and more correct. In this section, we are going to discuss how the mobile websites in Saudi Arabia were tested. Adequate usability and accessibility are the most important requirements of mobile websites. The main concern of usability is how users can efficiently navigate the mobile website and access required content and web services. Accessibility allows users to perceive, understand, and interact with websites. Mobile users expect short load times, sites that look good, a user-friendly interface, and a site optimized for any device.
The usability and accessibility of mobile websites of Saudi ministries and public sector bodies were examined using Dareboost [106], which is a mobile website testing tool for usability and accessibility.
The tool measures performance metrics of weaknesses, warnings, successes, load time, total page size, and the number of HTTP requests and presents results for each tested factor. Dareboost provides test report results with an overall page score, number of issues, improvements, and success for both usability and accessibility of a website. The report also displays the number of issues to include the time to first byte, server response speed, how long the page takes to start rendering, and how long the page is completely loaded. Dareboost offers a separate mobile website speed test option and provides choices of global geolocation that affect the testing results and type of browser (Android or IOS). The tool displays scores (excellent, good, average, below average, or bad) of the websites based on their metrics, and provides suggestions on how the accessibility and usability of a website can be improved.
During the test, the environment was set based on different parameters: test location and browser. The location of the test computer is Washington D.C.; and browsers for Android (Galaxy) or IOS (iPhone) platforms have been selected separately for each mobile website. These devices are different brands and have different operating systems and screen sizes. We selected Android and IOS because they represent the two most popular platforms in the targeted audience in Saudi Arabia. The URL of each website needs to be first loaded into the Dareboost. After the selected website is displayed on the viewport, a list of default mobile devices would be shown to users for selection. Tables 8 and 9 (given in the Appendix) show results for each usability performance factor: number of websites with identified weaknesses, warnings, success practices, website load time, total page size, and the number of HTTP requests for Android (Galaxy) and (iPhone), respectively. Performance factors, obtained from Dareboost evaluation, for both Android (Galaxy) and IOS (iPhone) devices are displayed in Fig. 2 and 3, respectively.

a: WEAKNESSES (PROBLEMS) ANALYSIS
Weaknesses of mobile websites for Android devices were examined and the results are shown in Table 8. None of    the websites achieved an excellent score (less than 10 weaknesses). Only 2 websites were rated as good with number of weaknesses less than 13. The investigation revealed that 19 (86%) of websites were able to achieve average and below average rating with up to 21 weaknesses, while only 1 reached the worst level of compliance.
Mobile websites usability weaknesses for IOS devices were studied and the results are displayed in Table 9. Results show that none of evaluated websites achieved an excellent result; only 2 out of 22 websites rated as good with number of weaknesses less than 13. The investigation revealed that 20 (90%) websites were able to achieve average and below average rating with up to 21 weaknesses.
The most common weaknesses for both Android and IOS devices are listed in Table 10 (given in the Appendix). The top 5 weaknesses and their frequencies are listed in Table 11.

b: WARNINGS ANALYSIS
Test results for warnings issued for mobile websites of Android devices are presented in Table 8. None of the websites achieved an excellent result with fewer than ten warnings. Three websites achieved good results with fewer than 13 warnings; 19 (86%) websites were rated average and below average with up to 21 warnings, and just one website got the worst level of compliance with more than 22 warnings. Usability warnings issued for the websites intended for IOS devices were investigated, and the results are shown in Table 9. Two websites achieved good ratings with a number of warnings fewer than 13. The investigation revealed that 19 (86%) websites were able to achieve average and below average ratings with up to 21 warnings, and just 1 website was the worst level of compliance with more than 22 warnings. The most common warnings for both Android and IOS devices are listed in Table 12 (given in the Appendix). The top 5 warnings and their frequencies are listed in Table 13.

c: SUCCESSFUL PRACTICE ANALYSIS
Outcomes of successful practice evaluation for mobile websites accessed by Android and IOS devices are displayed in Tables VIII and IX, respectively. Results for both types of devices are almost the same and show that most of the websites had a very good number of successful practices.

d: LOAD TIME ANALYSIS
Total load time values for mobile websites accessed by Android devices are displayed in Table 8. Six (27%) mobile websites achieved excellent results with a load time of fewer than 4 seconds, followed by one (4%) website that achieved a good ranking. Four websites achieved average and below average results. Table 9 displays Dareboost results for mobile websites accessed by IOS devices: 4 (18%) achieved excellent results with a load time less than 4 seconds, 3 (13%) websites achieved good ranking, and three websites with average and below average results. More than half of the websites showed bad results, where the loading time was more than 11 seconds to load the website.

e: TOTAL PAGE SIZE ANALYSIS
The total page size of the mobile websites used by Android is presented in Table 8. Results show that 1 (4%) mobile website achieved excellent results with a page size less than 1MB, and ten mobile websites (44%) were graded as average and below average. Half of the websites got bad scores, where the page size was more than 4.5 MB, which may increase the loading time of the page.
Examination results of mobile websites total page size for IOS devices show that 1 (4%) mobile website displayed excellent results with a page size of less than 1MB, and 11 (49%) achieved average and below average results, and the rest rated as bad.

f: HTTP REQUEST ANALYSIS
The number of HTTP requests on websites for Android devices are listed in Table 8.
Test results show that none of the websites achieved excellent or good results with fewer than 60 HTTP requests. Six (27%) websites achieved average and below average results with more than 60 HTTP requests. More than half of the websites got the worst rating where the number of requests was more than 80. A large number of requests negatively affect the loading time of the page, and a slow website affects user experience.
The number of HTTP requests on websites for IOS devices is listed in Table 8. Results show that one website achieved good results with fewer than 59 HTTP requests, while the other websites had similar performance problems as the Android device. Table 14 displays the listing of the most common accessibility issues, according to WCAG 2.0 for both the Android and IOS platforms, based on the outcomes of Dareboost evaluation tool. The top 5 Accessibility issues and their frequencies are listed in Table 15. Table 16, none of the websites achieved excellent scores for the Android or IOS, and just one website achieved a good score for both devices. The majority of websites showed average and below average results for both devices.

VII. DISCUSSION
This section discusses the results obtained by examining the usability and accessibility of the Saudi Arabia public sector websites using different methods. The objective of this discussion is to establish prospects for improving mobile e-government websites.
We analyzed how all the Saudi ministries' desktop and mobile websites fulfill the requirements, and we investigated   how well they comply with accessibility and usability guidelines.

A. USABILITY
The GTmetrix, a desktop evaluation tool, described usability problems with Saudi e-government websites, as shown in Tables 3 and 4. These problems are related to performance indices: fully load time, total page size, and the number of HTTP requests. Performance improvements can be implemented for each problem. These are the recommendations, based on the GTmetrix, listed in Table 3 and described in the following: 1) Optimize images: Image size can significantly affect the webpage size. There are tools that can automatically crop some nonessential data without affecting webpage quality and thus reduce the user's image size, which ultimately reduces the page load times by loading appropriately sized images. 2) Serve scaled image: The browser may download unnecessary data and do unsupervised resizing when images are too large for their display area. VOLUME 9, 2021 3) Leverage browser caching: When website visitors are caching locally and reusing the content files included in the website (HTML, CSS, JavaScript, or images), page load times can be significantly reduced. This will lessen the webserver load, and in turn optimizing the load time for users. 4) Enable compression: It takes time for a website to transfer the page files and style sheets. This transfer time and the load time of a website can be decreased by using compression. 5) Serve resources from a consistent URL: Identical resources must be served from the same URLs to avoid problems with duplicate downloads and additional requests. 6) Defer parsing of JavaScript: If an external script requires downloading, JavaScript can significantly delay a page display. Avoid overuse of JavaScript to have a quicker start for the page display. 7) Avoid landing page redirects: Any redirections increase delays to the page load. Necessary redirects should be performed on the server-side instead of the client-side to lessen the client-side round trip. 8) Combine images using CSS sprites: When a browser loads a webpage, this browser makes an HTTP request for each image placed in a separate file. This procedure slows down the overall time of the page load. Using CSS sprites will combine several images into a single image called a sprite sheet. 9) Minify CSS: Minification is the process of eliminating unnecessary or redundant data without affecting how the resource is processed by the browser. It can save many bytes of data and speed up download and parse times.
List of the usability warning as stated in Table 4: 1) Use a Content Delivery Network (CDN). Web server response time may be improved by using CDN, which is defined as a collection of web servers scattered across multiple locations. Its job is to deliver efficient content to users. If a web server is located close to the user, the response times will be shorter. Content is deployed across multiple, geographically dispersed servers to make pages load faster from the user perspective. Usually, the closest server with the least number of hops or the one with the shortest response time is selected to deliver content to a specific user. 2) Add Expires headers. If a content-rich page (stylesheets, scripts, video, images) is visited for the first time, it is more likely that a number of HTTP requests will be made. By using the Expires headers, this problem may be resolved. This header will make those components cacheable, minimizing redundant HTTP requests. 3) Lessen the number of HTTP requests. Roundtrip time and bandwidth usage depend on the number of HTTP requests. Curtailing the number of components will reduce the number of HTTP requests required to deliver a page and make the whole process faster. 4) Use cookie-free domains. Cookies may increase loading time. Keeping their number and size down will affect loading time positively. 5) Compress components. It reduces the size of requested resources and, accordingly, their download time. User data usage will be reduced, and initial website pages will be displayed faster. 6) Compact JavaScript and CSS. Compacting JavaScript and CSS codes can save many bytes of data and speed up downloading, parsing, and execution time. 7) Reduce DNS lookups. The DNS lookups can be cached for better performance.
Not using a Content Delivery Network, loading time, and serving unoptimized images and large amounts of JavaScript are the most common problems (seen in Table 4). Most of these problems, such as download time and overload of images, are reported in previous studies [107].
Decreasing webpage size can be achieved by removing unnecessary images, with subsequent optimization of what is left by doing the following: compressing images, so they take less time to download; using the right file types; resizing large images to fit their viewport; splitting huge JavaScript files, and utilizing caching. Combining several JavaScript scripts into a single script and multiple CSS styles to a single stylesheet file will decrease the number of HTTP requests. This way, the browser will deal with a smaller number of HTTP requests, and better performance will be achieved. In terms of usability, the result suggests that the web pages performed reasonably well.

B. ACCESSIBILITY
The WAVE, a desktop evaluation tool, reported accessibility problems with Saudi e-government websites, as shown in Table 5. Based on the result, the average number of low contrast occurrences on webpages is 33, the average number of errors is 16, and the average number of alerts is 22. The examination shows that the accessibility standard was partially adopted in most of the tested Saudi e-government websites. Regardless, the fact that there are accessibility improvements compared to the study done by Al-Khalifa et al. [32], who evaluated Saudi e-government websites in 2017, many accessibility issues still need further improvements.
Evaluators detected very low contrast between foreground and background colors on some webpages. This issue is particularly challenging for people with impaired vision. Adequate contrast is necessary for all users, especially users with vision impairment.
Based on findings given by WAVE, the most commonly listed problems (Table 6 ) and methods to improve them are described as follows: 1) Empty link detected: A link contains no text and does not lead anywhere. Therefore, the function or purpose of the link will not be presented to the user. This can introduce confusion for keyboard and screen reader users. The empty link should be removed or provide text within the link that describes the functionality of that link. 2) Missing alternative text: Provide descriptive text.
3) Empty heading detected: The purpose of the heading should be described by providing descriptive text. 4) Linked image missing alternative text: A word or phrase should be inserted to tell website viewers the nature or contents of an image. 5) Empty button detected: The function or purpose of the button should be presented to the user by placing text content within the button.
List of the accessibility warnings detected by WAVE and recommendations for improving the situation suggested for the most warnings stated in

C. MOBILE USABILITY AND ACCESSIBILITY
Adequate accessibility and usability have a significant effect on the delivery of mobile e-government services to citizens effectively. Therefore, mobile e-government website designers and developers need to pay more attention to these issues to make government services accessible and usable to citizens.
Most of the Saudi mobile government websites are fully mobile-friendly (86%) and passed the Google mobile-friendliness test for the viewports of smartphones and tablets. These results show that approximately 75% of studied e-government websites can be adapted for mobile devices.
Results indicated little difference in the usability of the mobile website whether Android or IOS mobile phones have been used. The most common usability problems for both Android and IOS devices reported by the tool are large page size, some loaded images have not been displayed, identical resources are served from different URLs, and the number of the Document Object Model (DOM) elements is high.
The tools' reported warnings are CSS selectors are duplicated and too complex; compression is needed for some images to reduce the size; CSS properties are duplicated; some resources do not define their content type. There are some problems with the loading time. The total load time of the mobile websites for the Android device is slightly better than for the IOS device. The loading speed for most of the websites is not encouraging. Therefore, there is a need to minimize the volume of information placed on each website. For most evaluated websites, the total loading time is below average for both devices. It is because most of the websites showed bad results in terms of large page size, which is likely to increase the loading time of the page. Also, most websites have reached the worst level of the request number, which affects the loading time of the page. Therefore, to improve the usability of these websites, there is a need to reduce the total load time by reducing the page size, avoiding requests to unreachable resources, compressing images, scripts, and CSS files, and grouping duplicated CSS selectors to reduce the file size and optimize the rendering times.
Based on the evaluation tool and according to WCAG, these are the most common accessibility issues for both Android and IOS platforms an inadequate explanation of the purpose of each form field, empty elements that may disturb screen readers, some forms do not include a submit button, contrast errors, and some labels do not refer to an element. Other common accessibility issues: missing alt text on images, missing link text, and ambiguous link text that does not tell the user what the link is for or where it goes. Link and navigation texts must uniquely describe the target link. Alternatives must briefly describe the purpose of the element. Clarifying the purpose of each field will simplify and thus enhance the user experience on the website. Removing empty elements so that the screen readers will not have difficulties interpreting their presence is also important. A link attractiveness increases if the text describes what it is about. All forms must include a submit button.
Overall, the results of this study indicate: 1) The manual evaluations have shown that over the past six years, the number of violations of usability and accessibility has been significantly reduced, and most of the websites have supported mobile appearance. 2) Based on the manual evaluation of websites done by experts, the overall findings of the study reveal that evaluators have a positive opinion about Saudi e-government website. This is despite some usability problems and severe accessibility issues, which could negatively affect citizens' experience, still existing. 3) Most websites for both desktop and mobile devices do not offer an adequate accessibility for the user with disabilities. For instance, a few websites provide interactive capability or interfaces for people with disabilities or low technical skills. In order to solve this problem, the websites should follow WCAG 2.1. 4) An efficient technique to improve the performance of a website is by reducing HTTP requests [108]. This suggestion can be adopted to websites for desktops and to mobile devices as well. This is particularly important for first-time visitors who have to download all resources instead of loading them from their browser's cache. Every element on the web page, like a script, stylesheet, image, or embedded video, will add an additional HTTP request for each element from the user's browser to the server. If the browser has to make more requests, the time to fully load the page will take longer. That is particularly important if a website is popular and the server has to respond to multiple simultaneous requests at the same time. This will slow website responses thus affecting the user experience, which in turn will result in fewer users accessing governmental websites. 5) The analysis showed that the performance on the desktop was in the high-end range, while on mobile, the performance was not as good due to the lower processing power compared to desktop computers. 6) The overwhelming majority of websites followed most of the usability design standards. Some of the problems can be solved simply by improving loading speed. 7) As we notice from the analysis, some of the usability and accessibility issues noticed by human evaluators have not been detected by automated tools and vice versa. 8) The criteria used for evaluating the accessibility and usability of desktop websites cannot be generalized to mobile websites.

D. RECOMMENDATIONS
Based on the results, we have developed several recommendations to address the major problems for improvement. Website administrators and developers of the public sector bodies can improve the accessibility aspects by complying with WCAG 2.1. In addition, redesigning the mobile/desktop websites will make them more suitable for a broader category of users, including people with disabilities. According to a report issued in 2017 by the Saudi General Authority for Statistics (GaStat) [109], the most common disability among the disabled Saudi population is a visual impairment (46.02%), followed by physical difficulties (29.13%). GaStat stated that the number of elderly people in the future is likely to continue to increase and, correspondently, the population of elderly with disabilities also will increase. A majority of these people with disabilities and the elderly would benefit from easy access to government websites. Therefore, the website design should consider the special requirements of these citizens with disabilities and elderly people. Otherwise, it will be very difficult for them to access government websites, especially from their mobile devices.
Based on the results presented, a suggestion for practitioners is to improve the usability of mobile/desktop websites using a CDN, optimizing images, reducing the loading time of web page, concatenating CSS and JS files, preloading key requests, and reducing HTTP requests. In addition, it is recommended to explore why these problems occur by talking with those responsible for mobile e-government websites. A presence of problems indicates there is a great opportunity for improvement, and people who work in this area will be able to pinpoint them, thus making sure that nothing is missed.
It is recommended that issues of accessibility and usability regulations will be included in the curricula of university courses related to website design. That will make students aware of the needs of people.
It is also advisable to repeat the evaluation tests at least every two years. Validation can be done by automated testing tools, but it should be supported with manual validation. Therefore, the analysis results are expected to better reflect the readiness of mobile e-government website services to the citizens of Saudi Arabia.

VIII. CONCLUSION AND LIMITATIONS
The usability and accessibility of government websites represent the key features of the services provided to citizens. In this study, a methodology has been established to evaluate the usability and accessibility of selected Saudi government websites of ministries and public sectors. The study checked the quality of these websites and, in particular, the compliance with the standard guidelines. It also established how performance could be improved.
Manual testing is always required because testing tools cannot find all accessibility and usability problems. The investigation of desktop and mobile websites of the Saudi ministries and public sectors indicated that some of them have a modern appearance, are responsive, and follow accessibility recommendations. On the other hand, based on WCAG 2.0 and WCAG 2.1, the results show accessibility errors do exist. There are usability issues related to downloading time and broken links. Some of them remain partially accessible and require better usability. The results indicate that there is plenty of room for improvement.
A set of recommendations that can be used to improve further the experience and satisfaction of users of the websites have been compiled. The improvement of websites will be beneficial for both people with and without disabilities and could improve the online experience for all of them. Therefore, investing in the improvement of government websites is likely to have a higher return than expected. We suggest website designers and developers improve websites by doing the following: 1) Use automated accessibility tools, such as WAVE and GTmetrix, to detect accessibility and usability problems, respectively. 2) Apply best practice design standards to existing websites. 3) Develop a usability and accessibility evaluation plan to be performed by actual users to test the government websites using different devices. 4) Consider annual revising of usability and accessibility of all government websites.
This study could provide guidance to web administrators and designers in the development of mobile e-government websites to satisfy user needs and identify the ways to maintain and update these websites regularly.
It takes a lot of resources and effort to establish a good mobile e-government website that will be accepted by citizens for information, services, and transactions purposes. Therefore, this study emphasizes the need for policymakers at the Saudi mobile government websites to encourage regular desktop and mobile website evaluations to be up-to-date with the fast-changing web technologies and to ensure that the respective website provides clear, precise, and reliable user-oriented services. This research will be a good addition to the literature in the field of assessment of usability and accessibility of mobile e-government.
This study has some limitations, which constrain the generalizability of its research findings. Firstly, given that only 22 Saudi government websites were evaluated, the results presented in this study may not be generalized to all other websites managed by different public agencies. An evaluation with a larger number of websites would definitely make the analysis more accurate. However, we tried to select representative e-government websites, and thus, our results are representative and valuable. Moreover, analytic evaluation tools could not access some of the websites due to different technical and non-technical issues.
Secondly, some tools do not give performance improvement suggestions; there are tools that can give suggestions but not evaluate performance. Such tools were not part of this study. The experiments revealed that the value of an attribute might be different depending on the test settings. The quality tests are affected by external circumstances such as server location, time of measurement, Internet connection, etc. Moreover, this study cannot claim to be widely generalizable due to the wide range of smartphones existing in the market. To overcome these limitations, future researchers should include more public sector websites for analysis, and a wider range of devices should be considered for assessment. Another future research area is to explore further why government agencies fail to implement the higher usability and accessibility standards.
Since the mobile government is still an active field of research, further study is necessary to improve the usability and accessibility of its services. Therefore, this report can be a valuable starting point to investigate the influence of perceived design considering the guidelines of WCAG 2.2 and local Saudi culture on the usability and accessibility of mobile government services.
Future research should explore why mobile e-government websites continue to have usability and accessibility problems, either through employing an automated in-depth analysis tool or working directly with designers using interviews, surveys, and walkthrough technique.