Evaluating Public Service Delivery Smartness and Impact on Citizens’ Well-being

Public sectors are increasingly adopting emerging technologies to innovate and deliver smart services to enhance citizens’ overall well-being. Although the idea of smartness in public sector service innovations has been explored from the perspective of service providers, limited study has been done from the perspective of end users. Furthermore, the impact of smart service delivery on citizens’ quality of life has been widely studied quantitatively, but qualitative evidence has been sparse. This paper aims to address those research gaps using a mobile-based innovation in the motor vehicle annual registration services (SAMBARA) recently introduced by the West Java Province in Indonesia as a case study. We use a qualitative smartness measurement framework based on efficiency, effectiveness, transparency, and collaboration metrics to assess the smartness of the service. We also evaluate the service’s influence on enhancing citizens’ overall quality of life using the well-being framework, which is based on aspects of usefulness, safety, and convenience experiences. We verified the significance of our findings across various participants’ background using statistical analysis ANOVA. The outcome of the study shows that mobile-based innovation services not only create smartness in public service delivery but also improve citizens’ well-being regardless of their various backgrounds. This research contributes to the public service innovation knowledge base and offers a baseline study for researchers and practitioners to carry out similar study in other emergent nations like Indonesia.


I. INTRODUCTION
The rapid expansion of digitalized innovation and smart devices are providing a new set of tools to the government of today to deliver services and interact with their community in new ways that positively impact the lives of its citizens. Citizen-centered smart government services, accessible with any device anytime from anywhere [1], enable the governments to offer services to the public in the most costeffective and efficient manner [2]. As the result, there have been significant efforts by public sectors worldwide in adopting advanced and smart technologies to innovate public services [3], [4] and promote the well-being of citizens [5], [6], [7]. For example, smart technologies have been leveraged to address vital issues such as managing natural disasters such as floods, tsunamis, earthquakes, forest fires, and pandemics such as COVID-19 [8], [9], [10], [11]. These emerging technologies have also been integrated into a range of services that include transport, public health, energy consumption, public safety, education, and waste management [4], [12], [13]. Citizen-centered smart government services have enormous potential in innovating public services and delivery, especially in low-income economies such as Indonesia where more than half of the citizens have had access to the internet and smartphone since 2017 [14].
Recently, research in evaluating smart government initiatives such as assessing smartness in service delivery and the improvement in citizens' well-being have received significant attention [1], [15]. Smartness in service delivery can be seen as the collaboration between technology and innovation in delivering citizen-centered services [16], [17]. It may also be considered optimum cooperation with different organizations, communities, and institutions to deliver the service to users with openness of government [18], [19], [20]. Here openness in government refers to sharing information equally and transparently among the citizen so that they may aware how governments perform, make decisions, and find solutions [18], [21], [22]. From a technological standpoint, smartness in government services is related to the adoption of Internet of Things (IoTs), big data [23] analytics, and Artificial Intelligence (AI) in public service delivery [1], [3], [24]. For example, the development of AI that commonly depend on data retrieve from IoT [25] have been deployed and utilized in several sectors such as public transport, electricity, water and gas consumption, well-being facility, learning sector, and community safety [26], [27], [28]. With respect to citizens' well-being, the quantitative studies [5], [29] involving citizens in smart cities found that as more often citizens use smart services then their quality of life will improve significantly.
Although the innovations in public service delivery have been approached through citizen engagement and policies codevelopment [30], [31], [32], limited significant study has been done so far in understanding smartness of government service delivery from end users' point of view. Recent qualitative studies have demonstrated the technology utilization for creating smartness of government service initiatives from the perspective of service providers [17], [33], [34] but ignored to demonstrate how smart the service delivery is from the perspective of users. In addition, smart devices can improve the public service delivery as highlighted in studies [35], [36]. But no further investigation has been carried out to examine their claim. Therefore, there is a need for more study that will explore the role of technology especially smart devices in improving public service delivery from citizens' view to add an extensive concept of smartness in public services [17]. In addition, the impact of smart service delivery on citizens' well-being is still inadequately investigated [6], [37]. There are several quantitative studies that highlight the impact of smart government services utilization on citizens' quality of life within the smart city platform [5], [29], [38]. However, there is limited evidence of the investigations on the impact of government service delivery accessible through smart devices on citizens' well-being [37], [39]. Moreover, it has been recommended for a qualitative study to demonstrate and comprehend the problem more deeply and comprehensively to analyze and explain some quantitative data [17], [40]. Therefore, we developed a qualitative study to address the following question: How does the use of smart devices to access public services contribute to the smartness of public service delivery and the well-being of citizens?
We address the above research question in this paper. As a case study, we used a new vehicle administration service provisioning named SAMBARA [41], [42], which is developed by West Java Province government. According to Hakim et al. [42], there were 17,172,607 motorcycles in West Java Province in 2019. However, six million of them or 35% did not complete their annual registration for various reasons [66]. Although the West Java Province has established public service offices throughout the province to address the problem [67], these offices do not cover the entire West Java region. With advances in technology and wide use of mobile phones in Indonesia, the authorities introduced a new mobile-based motor vehicle annual registration service provisioning called SAMBARA [42] to provide better vehicle annual registration services and subsequently improve the vehicle annual registration. SAMBARA allows citizens to receive services through mobile phone from three different organizations namely the Regional Revenue Agency, the Police Department, and the Traffic Accident Insurance Company all in one place. The app allows citizens to pay their annual registration at anytime from anywhere, get access to all information related to their vehicle registration using mobile phones. Citizens can also check FAQs (Frequent Asked Questions) to search information related to vehicle registration process and other vehicle services. In addition, the app allows citizens to lodge any complain and get feedback from the service providers. We conducted online interviews with people who had used SAMBARA in the previous year. In addition, we examined the significance of our findings across various participants' background using statistical analysis ANOVA. This study adds to the body of knowledge on public service innovation and provides a foundation for researchers and practitioners to conduct similar studies in other emerging countries such as Indonesia. Specifically, this paper makes the following contributions: • We assess the smartness of public service delivery using a qualitative smartness measurement framework to investigate SAMBARA's smartness from the standpoint of citizens. • We assess the impact of SAMBARA on enhancing the citizens' overall quality of life using citizens' well-being framework, which is based on usefulness, safety, and convenience parameters. • We present the outcome of the study that shows that mobile-based innovation services not only create smartness in public service delivery but also improve citizens' well-being. • We examined the significance of our findings across various participants' background using statistical analysis ANOVA.
The rest of this paper is structured as follows. Section II is the literature review on public services accessible through smart devices, smartness framework in public services delivery also the correlation between the service delivery and citizens' wellbeing. Section III describes the methodology and followed by the findings discussed in Section IV. Discussion of the research findings is described on Section V. Conclusions and implications of this study is presented in Section VI. Meanwhile on the last section, Section VII, we describe the limitations and future studies.

II. LITERATURE REVIEW
There is a significant improvement in smartphone utilization worldwide [43]. Smart devices have widely been used as a new way for delivering public services [8], [44]. This allows cost-effective and on-time information and service delivery as compared to the traditional face-to-face services [45]. Various government services to date have been accessible through smart devices, such as smartphones, I-pads, or tablets [44], from administration services to healthcare, tourism, and educations [35], [46]. In Oman for example, some common government services that use mobile applications are transactional and operational services in education sector, goods consume and supply sector, security sector, and employment sector [44]. In China, there were more than 3,000 government mobile applications in 2019, ranging from public transport services to administrative services [47].
Mobile application-based government services are extensively used to provide emergency services [48]. However, government services accessible through smart devices still face many problems due to low adoption rate by citizens [44], [45], [48]. The probable reasons of insignificant number of adoption is that citizens are concerned about security and privacy issues [27], [48], information accuracy and system reliability issues [27], also lack of mobile-service quality measurement framework [46]. Furthermore, mobile application-based government services can improve citizens' quality of life [39]. For example, online public services in healthcare sector that support chronic diseases patient caregivers can enhance it users' quality of life [37]. Mobile based services are considered more convenient and accessible without time and location restrictions. Therefore, as noted in [39], [45], citizens prefer to access public services through smart devices than face-to-face services. However, there is a limited study conducted and discussed the effectiveness of this mobile health services for caregivers' well-being [37].
Researchers have recently discussed how to create effective mobile government services in the context of public service adoption [35], [48], [49] and smart city services [50]. In [35], [48], [49] authors highlighted theoretical frameworks to improve effectiveness of smart government service adoption among citizens. In [48], researchers noted how to improve the effectiveness of mobile government service adoption in security response system in China. Jaradat et al. [49] proposed a smart government service adoption framework with case study in Jordan. However, no empirical evidence was reported in study [35] to support their theory. Meanwhile Hartmann et al. [50] reported that mobile-based innovation service can enhance the effectiveness of non-emergency services since it can widen the alternative service access and provides citizens to report urban problems by attaching documents such as pictures. However, according to [16], [34], [51], effectiveness is only one of smart service delivery dimensions. Table 1 shown the parameters of public service delivery smartness and citizens' well-being that we found in previous studies.
This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and content may change prior to final publication. A theoretical framework of smartness in government that combines fourteen dimensions (e.g., integration, citizencentricity, efficiency, effectiveness, citizen engagement, and technology savviness) is proposed by Gill-Garcia et al. [16]. A different smart government framework that consists of three focus areas, which are the smart government concept, the subset of each concept and the implementation is suggested in [51]. The smart government concept can be achieved by improving electronic government initiatives, implementing open government programs, or smart city project realization with data, technology, and innovation as the key enablers [51]. The main goal of the online communication by the government is to enhance the public service outcome with using less effort, time, and resources [51]. This is mostly derived from digital based government services such as the implementation of electronic identification, procurement system, and interoperability. The focus of the open government strategy is on the government openness regarding their actions in delivering the services and how they involve their citizens in the process of policy development [20], [22], [51]. In smart city approaches the focus is to provide smart solutions to address smart cities problems that are not only enhance the local economic growth but also improve decision making process [51].
Velsberg et al. [17] recommend that smart government could be achieved by implementing IoT in public sectors to improve both service delivery and management process. To measure smartness in public service delivery, the framework uses four different measurement dimensions (i.e., efficiency, effectiveness, transparency, and collaboration). This measurement framework was suggested by Nam and Pardo [34] in 2014. As described in Table 2, we need to maximize the output while at the same time minimizing the input such as expenses, time, and effort in order to improve the efficiency [17], [55]. For example as described in Saputra et al. [56] the emergence of an online single application service system in Magelang Regency in Indonesia has increased the efficiency of government service delivery for processing trade business permits from 5 days to an hour. On the other hand, effectiveness is defined as the contribution of the smart service delivery to the stakeholders such as citizens and private sectors such as by improving communications and interactions with them [17]. Hartmann et al. [50] reported that mobile application is more effective compare to the other 311 non-emergency channels to report street conditions for example since citizens can send pictures to document an issue. Meanwhile, in transparency, government needs to involve the community to understand events that occur in the government by providing relevant information in due time [17], [52], [57]. Government social media utilization for example to produce information and communication between government and citizens can improve citizens' perception on government transparency [47], [58].
Conclusively, smartness in service delivery is characterized by collaboration between government [17], [19], [33] and non-government organizations including citizens [31], [32]. Further, study [33] suggested that collaboration among stakeholders in smart city can improve not only the smart city services but also smart city governance. Collaboration among government agencies and within citizens in several municipal operation centres in Brazil for example have improved government ability in handling problems in smart city [33]. Furthermore, Sokhn et al. [53] suggest that collaboration with citizens through digital civic engagement can play a vital role to enhance public service transformation. However, there is no validation for theoretical framework of smart government proposed by studies [16], [51]. Even though studies [17] and [33] have validated their theory with empirical data, by conducting their case studies from a government perception.
Many studies have discussed the impact of smart technologies on the citizens' quality of life in the context of smart cities projects [5], [6], [54]. Since the smart city is part of the broader smart government transformation [51], we use the literature on citizens' quality of life in smart cities to understanding the impact of smart government service delivery in citizens' quality of life. The World Health Organization (WHO) defines quality of life as "an individual's perception of their position in life in the context of the culture and the environment in which they live. It also depends on their goals, expectations, standards, and concerns" [59]. However, some smart cities studies using the phrase subjective well-being rather than quality of life to demonstrate the impact of the services on citizens' happiness and living environment [38], [60], [61]. Subjective wellbeing is a holistic evaluation of a personal life condition and their quality of life during a given period of time, which include their life satisfaction as well as, positive and negative emotions [38]. Lin et al. [38] described subjective well-being of citizens in smart city can be measured by their quality of life. They also defined subjective well-being is the value of the smart city in the lives of its residents. However, the wellbeing term is related more with the individual level of analysis [6]. On the other hand, the quality of life is closely related with the people or community level of analysis [6]. Since both phrases have the same context meaning, which is representing experience, appraisals, and emotional attitude to certain situation [62], in this study we use citizens' wellbeing and citizens' quality of life interchangeably.
Previous studies noted that government services through smart city programs have positive impact on citizens' wellbeing because it promotes the information, education, and citizens participation, and generating better services to the citizens [5], [6], [29]. Yeh [5] collected quantitative data from Taiwan' smart cities. Meanwhile, Chatterjee et al. [29] conducted an empirical study on smart cities in India. Both studies [5], [29] concluded that material, financial and health welfare, personal safety, independence, development and achievement of success, relations with others as well as social and recreational activities are indicators for measuring citizens' quality of life in smart cities. Both quantitative studies [5], [29] emphasized the important of user experiences when using smart city services as the factor that directly gives influence on the citizens' well-being regardless of their demographic background. Macke et al. [6] conducted a similar study on smart cities in Brazil. They concluded that citizens' well-being could be achieved by promoting four elements such as socio-structural relations, environmental welfare, material security and community assimilation. In contrast, De Guimaraes et al. [54] highlighted that citizens' quality of life in smart city has positive correlation with the factors of smart city governance, for example government transparency in promoting accountability, decision-making processes, and their collaboration with the citizens to overcome the problems of smart cities.
According to [38] subjective well-being could be influenced by the residents' experiences when using smart city services that impact by several aspects. They are safety, usefulness, and convenience experiences. As demonstrated in [38], safety experiences refer to how residents feel about their privacy and security when they are using smart services, especially the idea that personal information will not be disclosed and or utilized by unauthorized people [38]. However, study [38], considers the security of the service process including credibility of the information services and the quality of the service too. Meanwhile, usefulness experiences refer to the user's perception of how a given service adds to the improvement of the system performance, benefit, and value [4], [38]. Usefulness is also described as city dwellers' perception regarding how beneficial the information and services received from a smart government in improving their quality of life and employment opportunities. [2], [38]. On the other hand, convenience experiences refer to the situation faced by urban residents for smart services, particularly regarding the time and the effort, when accessing and completing the services [38], [63]. This is including how easy it is to understand the information or learn the services, how straightforward the process of acquiring information or services is, how clear and simple the directions for obtaining information or services are, and how soon this information or service may be accessible [38]. Table 2. Smartness characteristics in service delivery and influencing factors on citizens' well-being.

Category
Parameter References Smartness characteristics in service delivery 1. Efficiency: How the government services accessible through smart devices could enhance the creation and delivery of government services with affordable expenses, time, and effort but give higher results.
[2], [4], [38] 3. Convenience experience: the degree of ease in the use of services that accessible through smart devices.
[38], [65] In conclusion, smartness service delivery is mostly discussed from service provider's point of view and is limited in providing evidence from users' opinion. Moreover, a significant number of researchers proposed mostly quantitative studies to evaluate the impact of smart government services on citizens' well-being but inadequate in using qualitative studies. Therefore, to provide broader and complete understanding of the effectiveness of mobile based innovation government service and its impact on citizens' well-being, we investigated them through a qualitative study approach. In this study, we propose a framework for evaluating public service delivery smartness and impacts on citizens' well-being by combining parameters suggested by Velsberg et al. [17] and Lin et al. [38] as summarized in Table 1. The definition of each parameter regarding the smartness characteristics in service delivery and influencing factors on citizens' well-being used in this paper are given in Table 2. In Table 2 we describe the definition of each parameter that we use in our framework.
We also developed a hypothesis, described in Figure 1, to study the influence of participants' background on their opinion regarding the role of smart devices to improve public service delivery smartness and citizens' well-being as follows: H0. There is no difference in the views of all participants who have various backgrounds about the role of smart devices in creating public service delivery smartness and improving the well-being of citizens.

III. METHODOLOGY
We conducted a qualitative study from October 2020 to December 2020 in the West Java Province Indonesia. We interviewed residents who have used this smart motor vehicle registration payment service through handheld smart devices (smartphones) for checking and paying vehicle annual registration fees. In this section, we will highlight the methodology we used.

A. DATA COLLECTION
In this study we developed a semi-structured interview process as described in the Appendix by focusing on citizens' experiences using SAMBARA smart app services to understand the service usefulness and effectiveness. The problems that users faced when using the services, the quality of services and how convenience this online service is compared to the conventional one. Furthermore, in our interview questions we asked for their experiences and seek suggestions on how service can be improved. We developed interview questions in both English and Indonesian since many of the citizens were not fluent in English. Furthermore, before we conducted the research, we have done ethics clarence both from Deakin University and the West Java Province government. Due to the COVID-19 pandemic, the data collection strategies were conducted online. We asked the participants to voluntary join our phone or online interviews through Google Meet. However, in this paper we only discuss the interviews since the results of our quantitative study has been previously discussed in [41]. We complement the participants with a mobile phone voucher or an electronic wallet of 100,000 IDR (USD 7). We invited potential participants to join the interview through email and WhatsApp. But only 176 participants initially responded and finally 69 participants attended the interview. In the invitation we informed the participants a flexible interview time. Each interview continued between 25 minutes to 1.5 hours.

B. SERVICE USER EXPERIENCES
In addition, we also collected service user experiences as shown in Table 3. The participants were already familiar with SAMBARA services since most of them have used it for more than a year (58%) and a small number of participants have used for more than 2 years (13%). In contrast, only 29% have used SAMBARA for less than a year.  This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2022.3186325 account for 21.7 percent of the total, while those earning between 2,000,0001 and 5,000,000 IDR (USD 139-349) account for 24.6 percent. Finally, 21.7% of the participants whose monthly income was below or equal to IDR 2,000,000 (≤ USD 139) and those who have no monthly income was account for 10.1 percent.

D. DATA ANALYSIS
In this study we analyze the data we collected using qualitative research process. We transcribed the interview data that conduct in Indonesian into English. In this step we also asked Indonesian and English teacher to review the transcription both in English and Indonesia. We employed a systematic iterative process to code and analyze the transcripts. For this study, coding is comprised of processes that enabled data to be collected, assembled, categorized, and thematically sorted, providing an organized platform for the construction of meaning [40].
We developed codes for each category based on parameters that we found in our literature review, as shown in Table 5.
There are four codes for smartness service delivery category and three codes to identified well-being support factors category. We also collected response from participants about SAMBARA deficiencies and we summarize them into two categories, which are service quality and citizens engagement as described on Table 6. There are five codes for service quality and two codes for citizens engagement. We also analyzed responders' suggestion for service improvement and summarize them into three categories, which are service quality, citizens engagement and collaboration as listed on Table 7. We conducted statistical analysis using ANOVA to examine the significance of citizens' background (age, gender, income, and education level) on the service delivery smartness and citizens well-being as shown in Table 8-11.

IV. FINDINGS
In this section, we highlight the findings of the study. We present the smartness factor in the government service delivery followed by the citizens' well-being support factors and statistical analysis using ANOVA.

A. GOVERNMENT SERVICE DELIVERY SMARTNESS
In this sub section we discuss smartness of public service provisions in delivering service to citizens. Table 5 summarizes the participants' responses. As shown in Figure 2, we compared participant comments on each of the variables of smartness, efficiency, effectiveness, transparency, and collaboration. Figure 2 shows that the 'effective' metric gets the highest score (85.5 percent) when compared to the other smartness parameters. Most of the participants feel services through the new service provisioning method are more effective than faceto-face services. Citizens can check annual registration fees and process annual registration payment through this new service provisioning method. In addition, this new service provisioning method is effective because it provides information and program related to other SAMSAT services and has supporting features such as FAQ, feedback, and complaint in one application. Therefore, citizens and government can communicate easily through this new service provisioning. In conventional services, to request information, citizens need to travel and ask staff at government office. This finding is also highlighted by earlier study which reported that in conventional services, to ask about information and services, citizens need to visit different counter in government offices and ask different government staff [69].
In terms of efficiency, 71 percent of participants noted that the service is more efficient. This is because citizens are not required to travel and spend their money for transportation to go to the government's public service offices to check and pay their annual registration fees. Moreover, when citizens come to the government office, they will spend significant time waiting to receive the service. The smart app relieves them from these because it allows them to check and pay their registration fees online anytime from anywhere without worrying about losing their working hours. Previous studies have also noted that in conventional services to complete annual registration, citizens need to travel and queue long for hours at government offices [69], [70], [71].
Regarding transparency, 37.7% of the participants mentioned that the new service provisioning method provides transparent process in delivering registration payment services. The participants noted that they prefer to use the new service as compared to the conventional services provisioning method. This is because the new service provisioning is more transparent and provides detailed information as well as transparent payment process. In conventional services, there is no information about the total amount that need to pay prior to 98.6 • Make citizens easy to pay their annual registration.
• Citizens can obtain annual registration fees before they pay.
• Using SAMBARA can protect citizens from the transmission of COVID-19. Convenience experience 62 89.9 • Simple and easy to use.
• Easy to understand.
• User friendly and easy to operate.
• Prevent from brokers.
• Prevent from non-procedural fees.
visit government office. Previous studies reported that in conventional services citizens face complicated payment procedures, unclear information about the requirements and total payment, therefore make citizens reluctant to pay their vehicle registration through conventional services [70], [68], [69].
In terms of collaborative services, 24.6% of the participants commented that the new service provisioning method improves government collaboration with both government institution and private sectors, especially in providing various payment channels. This new service provisioning makes citizens to be able to pay their annual registration through Alfamart and Indomaret stores and their branches around the West Java region. Citizens can also pay through e-commerce businesses such as Bukalapak and Tokopedia. Citizens can pay through various ATMs, Banks, and M-banking. In addition, citizens also feel that this new service is more collaborative since their local government disseminate this new service provisioning even though it was developed by provincial government. Previous study has demonstrated that this new service provisioning method have collaborated with various banks and enable citizens to pay their registration easily, compared to conventional services that only facilitate payment through government offices that limited by working hours [68]. • Some citizens still have limited technology awareness.
• Some citizens prefer to use offline services to pay their annual registration.

B. CITIZENS' WELL-BEING SUPPORT FACTORS
In this sub section we discuss how the new service provisioning method can impact citizens' well-being through understanding factors that can influencing citizens' wellbeing. Figure 3 depicts the total number and percentage of comments from each variable of citizens' well-being support. From Figure 3, we can observe that 79.7% of participants feel that the new service provisioning method provides improved quality of service and thus has impacted their safety experience in terms of privacy and security. They feel that it protects user personal data because it does not show owner information, machine number, and frame number. The participants also feel that the new service provisioning method provides valid and accurate information. When citizens want to pay their annual registration through this new service, they need to validate their vehicle data prior to pay. In conventional service, the payment and information sometimes does not accurate since citizens face a different policy and payment when they visit government office [70]. In terms of convenience experiences factor, 89.9% of the responders are satisfied with the new service provisioning method and noted that this new service provisioning method is simple, easy to use and easy to understand. The features on this new service provisioning method are also user friendly and easy to operate. • Activate service delivery feature.
• Improve system security and privacy by asking users to login before paying for registration. • Add information and statements about terms of use SAMBARA including how SAMBARA protects user data. • Improve the annual registration services to become fully online so that citizens do not need to travel to government office to obtain payment obligation letter and certificate validation. • Improve service quality and system reliability.
• More responsive in responding citizens' feedback or complaints sent through SAMBARA or government social media. • Add more services in SAMBARA such as certificate renewal and vehicle transfers. • Add personal account feature in SAMBARA so the system can record user's vehicle and transaction data, send notification for every transaction, and notify users through email or text message for registration payment due date. • Upload announcement on government social medias and website prior to system maintenance. • Provide SAMBARA for Apple's iOS.
• Improve and add more information in SAMBARA's manual and FAQs including all abbreviations related to vehicle registration as well as information on traffic accident insurance. Citizens' engagement 55 79.7 • Improve the dissemination of SAMBARA services to all community levels through various engagement channels (both online and offline) continuously. • Educate and train more citizens especially those living in regional areas about SAMBARA services. • Provide and disseminate various engagement channels for feedback and complaints. Collaboration 22 31.9 • Collaborate with more banks, e-commerce businesses, also adding digital wallet options such as OVO, GoPay, or LinkAja for payment channel.  This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2022.3186325 In conventional services, citizens need a lot of time to process their annual registration. Some citizens prefer to use a broker to complete their vehicle registration as reported by Nayaka and Darma [70]. However, some citizens feel that the presence of brokers in the government offices makes the situation becomes uncomfortable [68]. This new service provisioning can prevent them from brokers and therefore makes them more comfortable to pay their vehicle annual registration. In addition, this new service also protects citizens from nonprocedural fees. Previous study noted that some citizens have an experience to pay additional fees that is not stated on their receipt to complete their services at government office [70].
Most of participants feel that they have usefulness experience when using this new service provisioning. They found easiness when they want to pay annual registration using this service compared to when using conventional services. This new service provisioning can also support their productivity in their everyday life as citizens can complete their vehicle annual registration without interfering with their work or business. When using conventional services, citizens need to spend several hours or even a day to complete their annual registration, therefore lowering their productivity [69]. Citizens can also obtain annual registration fees before they pay. In conventional services, citizens do not know the exact amount of registration payment because there is no information available for them [70]. In addition, using this new service provisioning can protect citizens from the transmission of COVID-19 since citizens do not need to travel to government offices to queue and meet many people.
In summary, among the various well-being parameters, usefulness experience has the highest proportion (98.6 percent), whereas the convenience experience parameter and the safety experience parameter each has 89.8% and 79.7% respectively.

C. ANALYSIS WITH ANOVA
The one-way analysis of variance was conducted to evaluate the null hypothesis for each parameter based on participants' age, gender, income per month, and education level.      F(2,66) = 0.578, p = 0.564. Since the result of p for all parameters with independent variable education were found more than 0.05, these mean that the results are not significant.
To sum up, since the result of the one-way analysis of variance for each parameter based on participants age, gender, income per month and education was not significant, therefore, the ANOVA test failed to reject the null hypothesis. Table 6 shows a summary of participants' responses regarding the shortcomings of the smart government service providing strategy. There are two categories of deficiencies that were highlighted by participants, which are the service quality and the citizens engagement. In service quality category, we found five types of deficiencies based on citizen comments. These include incomplete services, lack of system privacy and security, complicated features, and lack of responsiveness on suggestions or complaints.

1) DEFICIENCIES IN SERVICE DELIVERY
As described in Figure 4, the incomplete service criteria have the highest number or percentage compared to other deficiency criteria in the service quality category. We found that 59.4% of participants from total responders suggest that the new service provisioning method provides incomplete services because citizens still need to travel to the government office after paying annual registration to change electronic payment obligation letter with the physical one and to validate their vehicle certificate. There is no further explanation which government office that they need to visit because according to responders, not all government offices can print electronic payment obligation letter and validate vehicle certificate.
Another deficiency is that the online service does not provide feature to renew vehicle certificate. Moreover, there is no call center number available on this smart app. Furthermore, this new service provisioning does not provide clear explanation on which channel can be used by citizens for sending feedback and complaint. This is currently conducted through email or feedback and complaint feature. There is no further information as well regarding how long it will take to process feedback and complaint and then respond to the citizens.
24.6% participants feel that this online service still has lack of system privacy and security. Participants commented that the procedure to obtain vehicle data is too easy since citizens do not need to login to obtain annual registration information. Furthermore, there is no user account available in this online service. Moreover, there is no explanation how this application protects data security and privacy.
17.4% of the participants have also identified the system to lack reliability. It sometimes takes too long to load vehicle data query. Also, sometimes the service is unavailable and there is no prior announcement of the service interruptions. Furthermore, there is a feature that is not working properly.
There is no further information on the system why this feature is unavailable.
8.7% of participants stated that the service has complicated features because it has too many colors and icons. All features in this new service provisioning method are displayed on the first page and there is no feature categorization. Therefore, users might be confused to use this application. Some features are also difficult to understand, and it is not user friendly. This application uses many abbreviations and there is no further explanation of these abbreviations that use in this application.
7.2% responders commented that the service was less responsiveness on suggestions or complaints. This is because when they submitted suggestions or complaints through this application, they did not receive any response. Responders also noted that when they sent feedback or complaint through government social media, they did not receive any reply. This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2022.3186325 In citizens engagement category, we highlighted two deficiencies associated with the service. These are lack of service utilization and lack of dissemination and education as described in Table 6. Among these criteria, lack of service utilization has the highest number and percentage as shown in Figure 4. 63.8% responders commented that the service is still lacking in service utilization. This lack of utilization is because not many citizens know about the new services. Moreover, some citizens are also having limited technology awareness that prevents them from using the service well. Furthermore, some citizens prefer to use offline services to pay their annual registration because in offline services, even though it takes more time, citizens can receive all documents in a time. If citizens use online services, they need to visit the government office in the other day to exchange their electronic obligation payment letter and validate their vehicle certificate. In addition, not all regions have internet connection. Therefore, some citizens in certain regions cannot access this new service provisioning method and still use offline services.
There is also lack of dissemination and education as stated by 49.3% responders. These responders reported that the responsible authorities provide less education on how to use the service and less promotion about the benefit of the services. The social media of the responsible office is also still lacking in promoting the service and its features. Moreover, some participants hardly find any promotions about service on the government onsite services.

2) RECOMMENDATION FOR SERVICE IMPROVEMENT
The recommendations given by interviewees, as highlighted in Table 7, can be divided into three areas, which are service quality, citizens' engagement, and collaboration. Among those three areas, improving service quality is the most recommended for improvement as illustrated in Figure 5.
88.4% of participants reported that the quality of this new service needs to be improved. These responders suggested to make its features simpler, easy to use, and easy to understand. They also suggest providing a call center number and live chat feature on the application for feedback and complaints channels. Furthermore, responders also suggested adding information not only the requirements but also cost estimation for other services. Responders also requested to activate service delivery feature. To improve system security and privacy, this smart app needs to ask users to login before they pay for registration. Moreover, interview participants also asked to add information and statements about the terms of use of this application including how it protects users' data.
Participants suggest government to improve the annual registration services through this new service provisioning to become fully online. So, that citizens do not need to travel to government offices to obtain payment obligation letter and validate their vehicle certificate. Participants also suggest improving the service quality and system reliability. The government authorities also need to be more responsive in responding citizens' feedback or complaints sent through the smart app or social media.
Another suggestion is to add more services in the app such as renew vehicle certificate and transfer vehicles. Participants also requested to add personal account feature in the app so the system can record user's vehicle and transaction data. This personal account also allows the system to send notifications for each transaction and notify users via email or text message for the registration payment due date. If there will be system maintenance, participants want the government to announce it first on their social media and website. Participants also suggested that this smart app service has a version for Apple's iOS. Furthermore, participants commented to improve and add more information in application manual and FAQs including all terms related with vehicle services, as well as information about traffic accident insurance.
Among all participants, 79.7% of them suggested that the responsible authorities to improve their engagement with citizens. Participants wanted the authorities to increase the dissemination of this new service to all community levels through various engagement channels both online and offline continuously. Furthermore, participants also proposed that the authorities produce some materials to educate and train more citizens, especially those who live in regional areas about this application service. In addition, participants also wanted the authorities to provide and disseminate various engagement channels for feedback and complaints to their service users.
Respondents also desired improved government engagement, with 31.9 percent suggesting that the authorities partner with more banks and e-commerce enterprises. These participants also suggested adding digital wallet option for payment channel on the app. Responders also suggested the authorities to increase cooperation with local governments to educate and train citizens about this application and how to use it. Furthermore, responders also suggested the authorities collaborate with local communities to inform and educate citizens, and other information related with government services.

V. DISCUSSION
As discussed in the literature review, smart devices have become one of the most important tools for government in the worldwide to make their services smarter. Moreover, prior studies have noted the importance of efficiency, effectiveness, transparency, and collaboration to measure how smart the government service delivery is. In reviewing the literature, we found few studies that discuss the role of smart devices in creating smartness in public service delivery from users' perspective and how this smart service delivery impacts on citizens' well-being.
The results of this study indicate that the new service provisioning that accessible through smart devices in the West Java Province are more efficient than face-to-face services, as illustrated in Table 5. This finding is consistent with study [16], [17], [34] that concludes using ICT in the public sector can increase the efficiency by reducing cost, time, and effort to receive the service. According to the research findings, the new service provision method is more effective than conventional services. This result is consistent with previous studies which have confirmed about the effectiveness of using technology to increase service delivery and government and citizens communication [17], [51].
This research also suggests that this government application service is more transparent than face-to-face services. This conclusion is supported by earlier studies that described the significance of IT-enable government services to improve transparency [8], [17], [72]. Transparency is one of the main government objectives to advance their accountability, citizens' trust, transparency in decision making and information for their stakeholders [1], [8], [72]. In this current study we also highlight that this new online service is more collaborative than face-to-face services. The authorities have collaborated with different stakeholders such as banks, e-commerce businesses, also minimarkets to provide easy access to pay annual registration. This result supports the idea of smartness in service delivery measured by how collaborative the government is in delivering services to users [17], [19], [33].
Another important finding was that the services of this smart app are also support citizens' well-being because the services are improved citizens' usefulness experience. This result reflect those of in study [38], which also found that citizens' usefulness experiences positively increase their well-being.
We also conclude based on our results that this smart app can also support citizens' well-being as it supports the user's safety experience. This result is aligned with previous study which confirmed that safety experience such as user experience in security and privacy aspect when using smart government services are positively influence citizens' well-being [38]. In addition, we also found that the services of this smart app can support citizens' well-being because it supports user's convenience experience. This finding broadly supports the work of other study in this area linking convenience experience with citizens' well-being [38].
This research also demonstrated that there is no significance difference in the views of all participants who have various backgrounds (age, gender, education, and income level) about the role of smart devices in creating public service delivery smartness and improving the well-being of citizens. This argument is supported by earlier study, which concluded that demographics profile do not have an impact on citizens' attitudes and behaviors toward smart government services [5]. An explanation of the lack of significance of the demographics in this study is that participants in this study were mostly age between 18-40 years old. This age of range according to Vaportzis et al. [73] use smartphone to go online higher that citizens above 65 years old.
This study also revealed the deficiencies of this new service provisioning. Its service quality needs to be improved because it still has incomplete services, lack of system privacy and security and lack of system reliability. Furthermore, for some users, it has complicated features and less responsiveness to suggestions or complaints. In addition, it also still has shortcomings in citizens engagement due to lack of service utilization and lack of dissemination and education.
We also discovered important findings from this study on how to improve this new service to comply with citizens' need. The services should improve reliability, security, privacy, and provide clear information through their system about other services. It also needs to add notifications about system updates, registration payment processes, and complaints handling processes.
Another important finding from this research is that the authorities should improve their service quality such as implementing full online services for annual registration payments and starting to develop online services for other public services, such as vehicle certificate renewal. The government authorities also need to carry out system maintenance outside of working hours and provide clear information about complaint procedures.
This study supports the evidence from previous observations that conclude the importance of system performance, service information and service quality for increasing service adoption [5], [44]. Furthermore, we also found from our study there are suggestions for improving the collaboration of government with their stakeholders such as citizens, other government institutions and non-government institutions.
The government needs to disseminate the benefits of this new service, as well as continuously provide instructions on how to use it to their users. This dissemination strategy can also be addressed by increasing cooperation with local governments and local communities in the West Java Province to spread the information to their regions. The authorities need to collaborate with non-government organizations as well to improve their service delivery.

VI. CONCLUSIONS AND IMPLICATIONS
This study set out with the aim to understand: How does the use of smart devices to access public services contribute to the smartness of public service delivery and the well-being of citizens? Earlier studies have only offered smartness exploration of government initiatives from service providers' point of view [17], [33]. Moreover, previous research highlights the effect of the services delivered on citizens' quality life in a quantitative approach [5], [6], [29].
We applied the existing smartness framework that already used in study [17] to investigate the role of smart devices in creating smartness in the authorities services. We used a qualitative study through the utilization of SAMBARA case study for vehicle annual registration services in West Java Province. We also explored the impact of this service on citizens' well-being used current citizens' well-being framework proposed by study [38]. Furthermore, we also investigated the influence of various participants' background on their opinion regarding the role of smart devices to improve public service delivery smartness and citizens' well-being. The framework was validated through empirical qualitative study involving 69 citizens in West Java Province who has used this new service provisioning.
The empirical result of this study reported that the use of smart devices to access public services could create smartness in public service delivery because citizens access services more efficiently, effectively, transparently, and collaboratively compared to conventional services. This research finding also noted that accessing public services using smart devices could improve citizens' well-being as these services enhance citizens' safety experience, usefulness experience, and convenience experience.
Taken together, these findings suggest that the role of smart devices have, not only promoted smartness in public service delivery, but also citizens' quality of life. This research contributes to the public service innovation knowledge base and offers a baseline study for researchers and practitioners to carry out similar study in other emergent nations like Indonesia. This present study adds to the growing body of research that indicates how the use of smart devices to access public services, provides benefits and affects components of the smartness of the service delivery and citizens' well-being.
These findings have important implications for developing government services since our study covered users' perspective on how smart devices can create smart public services and how this service can improve their quality of life. We also examined how the utilization of the new service provisioning for vehicle annual registration in West Java Province not only transformed the old services, but also led to the transformation of paradigms and procedures on delivering public services.
The stated outcomes illustrate that the existing smartness service delivery framework is valuable for assessing the role of smart devices in creating smart government programs and significant for identifying the potential outcomes of emerging technologies also to understand how far the services have fulfilled the citizens' need. The other important results of this study also describe that the existing well-being measurement framework is valuable for evaluating the impact of government services programs on users' well-being. For public sector experts, this research suggests that the use of smart devices to access public services might promote digitalization in the public services when combined with service quality improvement and citizens engagement improvement. We also conclude that smartness in service delivery allows a new era for collaboration with external organization, including with the community.

VII. LIMITATIONS AND DIRECTIONS FOR FUTURE STUDIES
The smartness framework that we utilized, proposes smartness in government initiative measured from internal and external perspective. However, in this study we only measured the smartness from external perspective, the citizens as the service users. Therefore, this study is limited by the lack of information on how the internal authorities perceive the smartness of this new service. Further studies need to be carried out in order to validate how internal management perceive the way government function to promote efficiency, effectiveness, transparency, and collaboration after this new service existed. In addition, how the other stakeholders such as the private sectors that cooperate with the West Java Province government assess the smartness of this service, which is also important to improve the understanding of smartness of the service from the private sectors' point of view. Different scope might influence the result. Therefore, it is necessary in future studies to develop or test existing framework to measure the impact of smart government services on citizens' well-being with case studies from different regions or countries.