A New Hybrid (MD5 and RC4) Cryptography Algorithm Using Multi-Logic States

In cryptography, the MD5 and RC4 algorithms are used in many applications to provide security. Both depend on two-state operations, such as XOR, for manipulating bits, and these operations are considered a weakness that can be vulnerable to attack. This paper presents a new hybrid of MD5 and RC4 based on a multi-logic states operation, called (E#), instead of XOR to increase the security level. This new (E#) uses three keys instead of one for controlling the variable block bits sizes of (1, 2, 4, and 8) bits and for determining the table numbers, which are formed from the addition in the Galois Field GF (2ⁿ) based on the block bit size for increasing the complexity of the proposed algorithm. This new hybrid algorithm leverages (E#) in both MD5 and RC4 and takes the output of the modified MD5 for use as the input key to RC4 increasing the security of RC4. The results are evaluated based on complexity, National Institute of Standards and Technology (NIST), histogram analysis, number of pixels of change rate (NPCR) and unified average changing intensity (UACI) and demonstrate that our modifications to both algorithms increase their complexity and security.