Cryptanalysis of a cryptosystem with non-commutative platform groups

A cryptosystem with non-commutative platform groups based on conjugator search problem was recently introduced at Neural Computing and Applications 2016. Its versatility was illustrated by building a public-key encryption scheme. We propose an algebraic key-recovery attack in the polynomial computational complexity. Furthermore, we peel off the encryption and decryption process and propose attack methods for solving the conjugator search problem over the given non-abelian group. Finally, we provide corresponding practical attack examples to illustrate the attack methods in our cryptanalysis, and provide some improved suggestions.