Behavior and system based backdoor detection focusing on CMD phase

Backdoor as a mechanism surreptitiously introduced into a computer system is widely used in performing network attacks. In this article, it is considered to detect its presence while helping an attacker to bypass normal authentication methods of a computer to maintain the access gained. In the latest researches have been done on this field so far, it is emphasized on analyzing only the behavior of backdoors. However, in this paper we propose a novel approach, combining systemic and behavioral features focusing on the "CMD" phase that the attacker sends commands to the victim. Through the detection method driven in this article, at first we gather the systemic and behavioral alerts produced while the attacker is installing and utilizing the backdoor interactively and then categorize them by specific features selected to give scores to the both aspects seen. Scores are given in two steps. The first step based on the prominent systemic alerts selected which are specified to backdoors and in the second step we give scores to the behavior it has in the command phase by creating and running a Markov Model. Literally, the scores are normalized and aggregated to determine the probability of backdoor residence on the computer monitored. We evaluated the algorithm in six different scenarios and by a group of well-known backdoors to make distinction between the proposed method and prior works.