A Comprehensive and Reproducible Comparison of Cryptographic Primitives Execution on Android Devices

With technology evolving rapidly and proliferating, it is imperative to pay attention to mobile devices’ security being currently responsible for various sensitive data processing. This phase is essential as an intermediate before the cloud or distributed ledger storage delivery and should be considered additional care due to its inevitability. This paper analyzes the security mechanisms applied for internal use in the Android OS and the communication between the Android OS and the remote server. Presented work aims to examine these mechanisms and evaluate which cryptographic methods and procedures are most advantageous in terms of energy efficiency derived from execution time. Nonetheless, the dataset with the measurements collected from 17 mobile devices and the code for reproducibility is also provided. After analyzing the collected data, specific cryptographic algorithms are recommended to implement an application that utilizes native cryptographic operations on modern Android devices. In particular, selected algorithms for symmetric encryption are AES256 / GCM / No Padding; for digital signature – SHA512 with RSA2048 / PSS, and for asymmetric encryption – RSA3072 / OAEP with SHA512 and MGF1 Padding.


I. INTRODUCTION
The evolution of the modern Information and Communication Technology (ICT) ecosystem has paved the way for one of the smallest form factor devices, smartphones, and wearables, in various areas of our life [1]. These are computing devices that can function independently and are small enough to be held in the hand [2]. Mobile devices typically connect to the Internet or communicate with other devices in close proximity, forming personal clouds within the new Internet of Wearable Things (IoWT) paradigm [3].
Indeed, the connectivity and flexible interaction are essential since most of those devices process personal data and manage access to it if stored remotely, centrally, or distributed [4], [5]. To comply with current security standards, mobile devices may include biometric sensors and The associate editor coordinating the review of this manuscript and approving it for publication was Aniello Castiglione . specialized cryptographic primitives [6] and provide a number of primitives for the data processing.
Smartphones and smartwatches are some of the fastestgrowing and most widely available personal devices [7]. With the rapid growth in the number of users and hardware/software features, security issues become more pressing [8]. Today, people are more likely to use smartphones for everyday tasks such as browsing, emails, internet banking, and mobile payments but still want to have fast response times as well as the assurance of the security guarantees. Said devices also collect and process their unique vital signs for a variety of purposes. All of these tasks require sensitive user data stored directly on the smartphone (for example, for preprocessing) or uploaded to remote/distributed storage, and the system security levels are shown in Fig. 1.
With a device that comes where the user goes and contains sensitive data, security must go beyond its current level. To ensure a smooth experience for all users (even those who VOLUME 9, 2021 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ are not security savvy or do not want to interact directly with it), mobile device manufacturers and app developers must implement security measures that protect user data, even if the smartphone is stolen.
The future wearable-oriented security ecosystem.
As a representative example, this paper focuses on the Android ecosystem as it is easier to develop and evaluate this market segment than other vendors. 1 The Android ecosystem must keep user data safe, and various techniques are integrated it. In particular, the authentication mechanisms are used to provide this kind of guarantee and deny access to unauthenticated users. Android uses the concept of user authentication cryptographic keys, which require a cryptographic key store, a cryptographic key service provider, and a user authenticator [9]. Although the ongoing research is not explicitly targeted at distributed ledger technology applications, the results obtained can also be used to develop appropriate [10] applications. Understanding latency for mobile devices serving as part of a distributed infrastructure with different consensus mechanisms is an integral part of application design, considering the trade-offs between different system characteristics.
This paper is the result of a research project that resulted in the successful defense of the thesis [11]. The presented text is intended to explain the security model on sixteen smartphones and one smartwatch using the Android OS and compare existing cryptographic protocols and the resulting run time as a direct translation to battery life. Unfortunately, providing the actual power consumption of the cryptographic primitives' execution becomes close to impossible due to the present devices' manufacturing concept of irremovable battery, but we also supplement the data with the battery properties that, along with the execution time, allows for better qualitative analysis of the results, similarly to [12], [10].
The main goals of this work are 2 : • To provide the list of information security primitives available on modern Android devices; • To give an example (in Kotlin language) of the primitives execution for the ease of other developers attempting to repeat this trial in the future; • To visualize the comparison of the execution time of the same primitive on different mobile devices; • To allow other researchers to reuse the collected measurement data by providing a complete dataset (available in IEEE DataPort [13]); • To facilitate the assumptions on the primitives' executability compared to real-life measurements. Broadly, we attempt to provide a toolbox and a dataset to be used by the researchers/integrators while selecting an appropriate configuration of one selected system, e.g., Rivest-Shamir-Adleman (RSA), depending on the execution time vs. key size trade-off rather than compare different types of the primitives between each other, which is a well-studied topic with expected outcomes.
The rest of the paper is organized as follows. The next section details what parts the Android security model comprises, their purpose, and how those work with other system components to provide sophisticated security for user data. The third section provides the description of the dataset as well as visualizes and summarizes the results obtained from the created benchmark application. Finally, the last section concludes this work.

II. BACKGROUND INFORMATION
This section outlines the main cryptographic primitives available on modern Android devices and provides sample implementation listings.

A. SUPPORTED CRYPTOGRAPHIC PRIMITIVES
Cryptographic primitives are well-established low-level cryptographic constructs [14]. The Android OS supports various categories of cryptographic primitives recommended by the National Institute of Standards and Technology (NIST) through its Keystore, 3 including: • Hashing functions convert the data of different sizes to data of a fixed size that makes it irreversible, for example, the family of secure hashing algorithms (SHA) (including HMAC-SHA) [15].
• Symmetric key cryptography is an encryption scheme that allows the use of the same key to encrypt and decrypt messages (for example, Advanced Encryption Standard (AES)) [16].
• Asymmetric key cryptography (or sometimes Public Key Cryptography (PKI) is a scheme in which a public key for encryption and a private key for decryption is used for each node, i.e., user A can encrypt a message addressed to user B with PK B , and only user B can decrypt this message with his own SK B . Therefore, it becomes possible not only to establish an asymmetric communication environment for securely exchanging symmetric keys in the future by securely exchanging public keys between two users. This group includes, for example, RSA [17], Digital Signature Algorithm (DSA) [18] and various approaches to Elliptic Curve cryptography (EC) [19]. Cryptographic primitives are often used to build cryptographic protocols. On Android, the Keystore system uses cryptographic primitives to provide multifunctional cryptographic operations, which include but are not limited to: • Key generation; • Import and export of asymmetric keys; • Import of raw symmetric keys; • Asymmetric encryption and decryption with appropriate padding modes; • Digital signature, and verification; • Symmetric encryption and decryption in appropriate modes, including an Authenticated encryption (AEAD) mode; • Generation and verification of symmetric message authentication codes; • Random number generation.
The key target, padding, access control restrictions, or any other protocol element is defined when generating or importing a key and is permanently bound to the key. The protocol elements associated with the key ensure that the key cannot be used in any other way. Random number generation is not exposed to the public Application Programming Interface (API). It is used internally to generate keys, initialization vectors, random padding, and other security protocol elements that require randomness. KeyStore can be used as a provider and with supported algorithms that operate in their respective classes: • Cipher allows for encryption and decryption; • KeyGenerator allows for the generation of secret keys for symmetric algorithms; • KeyPairGenerator allows for the generation of key-pairs for asymmetric algorithms; • Signature provides support for the cryptographic digital signature algorithms; • KeyFactory allows for interoperability of cryptographic keys and providing the key specifications; • SecretKeyFactory has a similar functionality as the one above but operating solely with symmetric keys. Indeed, the Android security system provides a wide range of next-generation information security systems' support. The following implementation options are presented detailing the supported OS versions for the above primitives.

B. PRIMITIVES IMPLEMENTATION EXAMPLES
The following framework will include all available implementation algorithms in Android KeyStore, including snapshots of source code to facilitate testing by other developers.

1) KEY GENERATION
To generate a key, one can use the KeyGenerator or KeyPair-Generator class. KeyGenerator provides the functionality of a symmetric key generator. KeyPairGenerator provides the functionality of an asymmetric key generator.

a: KeyGenerator
There are two approaches to generate a key using KeyGenerator: an algorithm-independent and an algorithm-dependent way. The difference between them is generator initialization. Listing 1 lists all of the KeyGenerator initialization methods. The Init methods that do not use the AlgorithmParameterSpec are independent of the algorithm. The AlgorithmParameter-Spec Init method is used in situations where a set of parameters for a particular algorithm already exists. In case the user does not utilize any of the available Init methods, the provider specified during creation must provide a default initialization. LISTING 1. KeyGenerator init methods supported algorithms of KeyGenerator are listed in Table 1.

b: EXAMPLE OF THE KEY GENERATION WITH KeyGenerator
Listing 2 shows the AES symmetric key generation procedure in Galois / Counter Mode (GCM) [20], which aims to encrypt and decrypt without padding encryption. AndroidKeyStore is defined as the KeyGenerator provider, so the key is generated in the Keystore hardware key store. Other restrictions may apply to the KeyGenParameterSpec builder. SetUser-AuthenticationRequired and SetUserAuthenticationValidity-DurationSeconds can be applied to the builder to condition the receipt of the key to a time window starting from the last unlocking of the phone, or the user can be prompted to log in directly into the application via Lockscreen.
The key can be obtained from AndroidKeyStore, as shown in Listing 3. AndroidKeyStore is run through a static function in the Keystore object. In contrast, AndroidKeyStore is passed VOLUME 9, 2021 LISTING 2. AES Key generation. as a type after the method load() is called for the KeyStore object being initialized. The load() method loads the Keystore using the given LoadStoreParameter, which can be null. After KeyStore is initialized, the key can be obtained by calling the getKey() method. The key alias passed to getKey() must match the alias during key creation. LISTING 3. Retrieve AES key from keystore.

c: KeyPairGenerator
As with KeyGenerator, there are two ways to create a key pair: algorithm-independent or algorithm-dependent methods. The difference between the two is explained in subsection II-B1 and the supported algorithms are provided in Table 2.

d: EXAMPLE OF KEY PAIR GENERATION WITH KeyPairGenerator
Listing 4 depicts the procedure of an EC key pair generation for encryption and decryption. An authenticated user can only use the key within 5 minutes from the last successful authentication. AndroidKeyStore is defined as the KeyPairGenerator provider, so the key pair is generated in the hardware key storage Keystore.

2) IMPORT AND EXPORT OF ASYMMETRIC KEYS
Keystore supports importing PKCS8 standard Distinguished Encoding Rules (DER) [22] key pairs without password-based encryption. According to the X.509 [23], export is only supported for public keys. Two different origin tags are used to distinguish imported keys from reliably generated keys. Here, imported keys use the imported tag, and secure keys use the generated tag.

a: EXAMPLE OF RSA PRIVATE KEY IMPORT
In order to import the private key into KeyStore, the Pri-vateKey instance and X.509 certificate for the public key corresponding to the private key represented as the X509Certificate are needed. It is mainly because the KeyStore abstraction does not support storing private keys without a certificate. Listing 5 shows how to generate a DER-formatted RSA private key and an X.509 certificate or public key. For demonstration purposes, the key and certificate files directly import raw application resources. Listing 6 shows the way how to convert a DER-encoded private key with an X.509 public-key certificate in the PrivateKey and Certificate instances that are used to import the private key into the Key-Store. Although it is possible to import externally generated keys into Keystore, it is not recommended. The private key accesses the main memory and can, therefore, be exploited by an attacker.

b: EXAMPLE OF PUBLIC KEY EXPORT
Exporting a public key is a straightforward procedure. Listing 7 shows how to obtain the private record the corresponding certificate from Keystore. The certificate can be converted to a byte array or base64 string and sent to the recipient. Note that the private key record contains a private key field that refers to the private key. No sensitive information that could lead to key abuse is present in the private key record.

3) ENCRYPTION AND DECRYPTION USING AN ASYMMETRIC KEY
RSA in various modes and padding settings is the only asymmetric algorithm available on Android to encrypt and decrypt LISTING 6. Import of RSA private key. LISTING 7. Export of EC public key. data securely. As of this writing, no other asymmetric algorithm is supported. Listing 9 shows the process to encrypt data using an RSA key in ECB mode with PKCS1 padding. Cipher is initialized with an RSA / ECB / PKCS1 padding transformation that matches the RSA key. The encryption mode is set, and the encryption key is the RSA public key. Data input is in a byte array format. For simplicity, the doFinal(1) method is called to encrypt the data. The result is a byte array of encrypted data. Evidently, the RSA private key is used to decrypt the data. Listing 10 shows an approach to decryption. The Cipher instance is initialized with the same transformation as the encrypted data. Decryption mode is set, and the decryption key is the RSA private key.

4) DIGITAL SIGNATURE AND VERIFICATION OF SIGNATURE
RSA, EC, DSA can be used in different modes and padding settings. Table 4 shows all the different configurations that can be used for signing and verification. Listing 11 shows how to generate an elliptic curve key pair with SHA512 digest for signing and verification.

a: EXAMPLE OF ECDSA DATA SIGNING AND VERIFICATION
Listing 12 shows how to sign data with an EC key using the SHA512. The signature is initialized with the SHA512 with ECDSA transformation. The signing key is the EC private key. The data to be signed is passed to the update(1) method, and the sign() method is called to sign the data. The result is a byte array.
The EC public key certificate is used to verify the signature and Listing 13 shows the corresponding procedure. The Signature instance is initialized with the same transformation as for data signing. The verify(1) method is called to verify the VOLUME 9, 2021 signature passed to the method. The data is then passed to the update(1) method. The result is a Boolean value indicating whether the signature is valid.

5) IMPORT OF 'RAW' SYMMETRIC KEYS
Importing symmetric keys is much easier than importing asymmetric keys. The symmetric key is placed in SecretKeyEntry and imported directly into KeyStore.

6) ENCRYPTION AND DECRYPTION USING A SYMMETRIC KEY
AES in various modes and padding settings is the only symmetric algorithm used in Android, no other symmetric algorithm is supported. Table 3 lists all combinations of encryption and padding modes. In addition, all combinations support all AES key sizes that KeyGenerator generates (128, 192, 256 bits). Listing 15 shows how to encrypt data using an AES key in GCM mode without padding.
The Cipher is initialized with the AES / GRCM / No Padding transformation, which corresponds to AES's key purpose. Data input is in a byte array format. The doFinal(1) method is called to encrypt the data. The result is a byte array of encrypted data. To decrypt the cryptogram, the Cipher initialization vector must be saved for later use.

LISTING 15. Encrypt data with AES.
Listing 16 shows the decryption process. The Cipher instance is initialized with the same transformation as the encrypted data. Decryption mode is set, and GCMParam-eterSpec is initialized with an initialization vector and an authorization tag. The doFinal(1) method is called with the passed cryptogram as a parameter. The result is an array of bytes that can be converted to string format.

LISTING 16. Decrypt data with AES.
To summarize, the Android OS already has rich functionality to perform symmetric, asymmetric encryption, block cipher operations, signing, and verification, among others, however, many primitives are still missing, e.g., the post-quantum candidates already highlighted in the NIST final selection round. The examples of the available primitives show that their use is relatively straightforward, and the examples of code can be found in the repository. The next section compares the primitives on flagship and legacy mobile devices and discusses some of the interesting observations made during this benchmarking campaign.

III. CRYPTOGRAPHIC ALGORITHMS COMPARISON
This section uses information from the previous one to execute the tests for the listed cryptographic algorithms. The measurements campaign contained 280 tests each that measure the running time of cryptographic algorithms on 16 smartphones and 1 smartwatch, shown in Fig. 2. We attempted to cover such a broad variety of devices to cover both different hardware characteristics as well as different operating systems, which allows to understand which execution could be expected on a particular generation of devices. This section presents the results of the analysis of the measurements.

A. DATASET DESCRIPTOR
The dataset for the oncoming results is currently available at IEEE DataPort [13]. The primary data related to the collected data is located in folder Measurement and subfolder with the measurement file.
The developed application was designed to gather the data and then generate the dataset. The dataset consists of JSON files, each containing measurements of available devices' security primitives execution times. The data was gathered in a span of multiple 250 iterations. Each measurement was taken with a 50 repetitions interval for every primitive. We define the main components of the dataset in the following: 1) context[] -provides the details about the device and OS including device name, model, battery-related information, Software Development Kit (SDK) version, and basic technical specification.  Table 5).

2) benchmarks[] -provides entries per primitive, such as:
• name -the overall identification title of the primitive, including paddung and other optional fields; • params -additional parameters unilized for the execution if any; • totalRunTimeNs -the overall time of the primitive's execution time; The following subsections provide a discussion on the obtained results.

B. CREATION OF ASYMMETRIC KEY
Asymmetric key generation testing measurement data shows the time it takes to generate an asymmetric key. The measured key types are RSA and EC with different key size options. Evidently, as the key size increases, the algorithm's computational complexity also increases. Thus, a larger key size is expected to have a longer execution time than a smaller key size. The heatmap 3 shows the results of the asymmetric key generation. The results of the RSA algorithm support this assumption on all devices.
Using the EC-based algorithm, if we compare the runtimes of EC224 and EC256 on different devices, the results unexpectedly show that a larger key size on six devices results in shorter runtimes, which contradicts the assumption. This could be caused by a small difference between the key size used in the algorithms.

C. ENCRYPTION/DECRYPTION USING AN ASYMMETRIC KEY
Asymmetric key encryption is currently only supported for RSA, as mentioned in subsection II-B3. RSA for encryption can be used in eight different ways. The difference between the two is in the fill mode used. Android Keystore and generic java Keystore do not implement ECB mode for RSA, so encryption / decryption can only be used for data smaller than the key size. Interestingly, the encryption modes have ECB in their name, although it is not implemented.
The heatmaps 4 and 6 show that the most consistent runtimes across devices are achieved with the RSA with PKCS1 padding option. PKCS1 Padding adds the least overhead of all padding schemes supported (at least 11 bytes). Completing the OAEP adds even more overhead. The OAEP padding scheme requires two hash functions with different properties to work. One hash function must map an arbitrary size input to a fixed size output. Another hash function maps an arbitrary size input to an arbitrary size output.
Such a hash function is called the Mask Generation Function (MFG). OAEP adds at least 42 bytes, which is 31 bytes more than the minimum PKCS1 padding. The results support the assumptions, and the overall execution time for schemes using OAEP padding is longer than for PKCS1 schemes. Based on the measurements, we can conclude that the Samsung Galaxy S6 shows significantly slower encryption with a key size of 4096 bits on all encryption schemes. Huawei P9 Lite and Asus Zenphone 3 MAX, when used with OAEP padding, result in slower performance than other devices. From a security point of view, the OAEP padding scheme is recommended [12]. Comparing the algorithms' results using the OAEP padding scheme shows that RSA3072 / OAEP with SHA-512 and MGF1 padding yields the best result.
As far as decryption is concerned, the inevitable statement is that it should be slower. The advantage of encryption is that the public figure is usually relatively small. The private metric, decryption, is more extensive, so decrypting data is a slower operation. The results in heatmap 5 and 7 summarize the results of the assumption that data decryption in most RSA implementations is slower than encryption. Same algorithm for best overall execution time as encryption, RSA / ECB / PKCS1 Padding.

D. DIGITAL SIGNATURE
Unlike asymmetric key encryption and decryption, digital signatures are supported by RSA and EC. Benchmarks are categorized by the hash function that RSA or EC uses. The dataset provides RSA results without hashing with MD5, SHA1, SHA224, SHA256, SHA384, and SHA512. All results demonstrate similar behavior for each device, and, in order not to overload the reader with repetitive data, this article provides an example with the most complex of them, SHA512, shown in the heatmap 8.
Overall, measurement results show that the difference in runtime between signature algorithms using different numbers of bits in the SHA function is minimal. Comparing the algorithms' results using the SHA hash function shows that SHA512 with RSA2048 / PSS shows the best results.
A similar procedure was used to verify the digital signature, see example results for SHA512 in heatmap 9. The difference in the execution time of the verification algorithms using the SHA hash function is the same as in the digital signature, i.e., it is minimal. Evidently, the verification procedure on a smartwatch requires more than an order of  magnitude more time than the same procedure on a smartphone when the mouse cursor is hovering over a sufficiently low level, that is, tens of milliseconds.
More heatmaps for the utilization of MD5, SHA1, SHA224, SHA256, and SHA384 with RSA are available along with the dataset.

E. CREATION OF SYMMETRIC KEY
The symmetric key generation performance test measures the time it takes to generate a symmetric key. The measured key type is AES with different key sizes. As the key size increases, the computational complexity of the algorithm also increases. Thus, a larger key size is expected to have a longer execution time than a smaller key size. The histogram 10 summarizes the results of creating a symmetric key. The difference in battery life on Google Pixel 3A, Huawei P20 Lite, LG Nexus 5X, Google Pixel XL is expected in line with the previous trend. On other devices, the execution time is equal, or the execution time does not increase with increasing key size. It could be caused by hardware optimization or statistical error.

F. ENCRYPTION/DECRYPTION USING AN SYMMETRIC KEY
AES is the only algorithm that supports symmetric encryption with different key sizes and variations. Heatmaps 11 and 12 show that the runtime on the device remains the same for all variants and key sizes. Based on results and in terms of security, AES256 / GCM without padding provides the best value.
It is assumed that decryption and encryption should be approximately the same due to the same keys for both operations. Compared to RSA decryption, AES decryption should be faster. The results in the heatmap 12 show an overall slower execution time than the symmetric encryption execution time, which does not support the assumption of the same execution time. Compared to the RSA decryption execution time, the AES decryption execution time is faster, confirming the assumption.

G. ADDITIONAL DISCUSSION AND LESSONS LEARNED
Interestingly, one may question what are the effects of RAM, OS version, or CPU (given in Table 5) on the execution of the primitives. Let us consider Figure 3 as an example. On the one hand, the OS version does not provide any representative information at all, see, e.g., for RSA-512: HTC One M9 shows 72 ms, Huawei P9 Lite shows 30 ms, and Asus Zenphone 3 Max has 228 ms -there is no correlation with OS version. RAM impact has a similar pattern. On the other hand, the execution time is indeed mostly influenced by the CPU characteristics. However, it seems rather unfair to compare the execution of various processor types with different numbers of processors with completely different clock rates since engineers/researchers would not have any real knowledge about the processor's design and would only face the execution time in the end. Moreover,    most mobile processors cannot utilize all the cores at once (usually 4 for ''background task'' and 4 for ''CPU demanding tasks'').
One of the most significant evaluation challenges is the actual energy consumption evaluation on modern mass-produced devices. There are two significant limitations.
First, the only way to reliably measure energy consumption is to connect to the battery connectors physically. We have attempted to do that and faced two sub-challenges: (i) most of the devices have irremovable or shielded batteries that heavily limit the access to the connectors without physically damaging the device; (ii) even after accessing the connectors, VOLUME 9, 2021  we found out that the impact of the primitives execution on energy consumption is miserable compared to, e.g., wireless transmission module or the display (the primary energy consumer).
Previously, we also attempted to measure the energy consumption for the execution of various blockchain consensus algorithms in [24], where we have proven that the impact on battery is not measurable only if the device is in the saturation of the cryptography-related executions, which is not real in daily life. Therefore, this paper attempted to highlight the metric, which is the most accessible for the developers from a user perspective -the average execution time causing uncomfortable delays while a human is interacting with the smartphone. As it could be seen from the collected data, some devices can produce a few seconds-level delays just for one execution, which may be unacceptable for close-to-real-time applications.
Notably, there are approaches how to approach the conversion of the execution time into relative computational energy. 4 However, those may have relatively low accuracy since even the comparison of self-discharge rate compared to one under additional tasks do not provide the necessary granularity [8].

IV. SUMMARY
The development of modern technologies and the overall improvements in the computing power are pushing towards evaluating cryptographic primitives used on devices available on the market. This article closes this white spot on the roadmap for information security in the field of Android devices. Along with this, it provides source code examples suitable for a future re-evaluation of new devices. Specifically, it summarizes the results from a benchmark app that ran 280 tests on 16 smartphones and 1 smartwatch in terms of execution time (as the most convenient battery life convention).
The results were further processed and visualized on heat maps and a histogram. Based on the results, it was concluded that not all natural assumptions regarding the executions of primitives were fulfilled. Some older devices with older processors execute some cryptographic algorithms faster than newer devices with newer processors. It can be explained by hardware acceleration for specific cryptographic algorithms.
After analyzing the collected data, specific cryptographic algorithms were selected to implement an application using cryptographic operations, see Section III. Selected algorithms: AES256 / GCM / No Padding for symmetric encryption, SHA512 with RSA2048 / PSS for digital signature and RSA3072 / OAEP with SHA512 and MGF1 Padding for asymmetric encryption.