Unwanted communications such as SPIT (SPam over Internet Telephony) and number harvesting are expected to become serious problems as the use of VoIP (Voice over Internet Protocol) grows. Having call rates that are approaching zero, VoIP is becoming an attractive alternative media for delivering spam. Existing solutions to detect unwanted communications have problems, such as the requirement of modifying the SIP (Session Initiation Protocol) server, significant inconvenience to users, and the possibility of the unwanted communication avoiding detection. We propose a novel detection method using call pattern analysis. We express call patterns with parameters such as user relationships, average call duration, and the rate of unsuccessful calls.The proposed method does not require any SIP server modification and can be deployed in a non-intrusive manner to users. Furthermore, we have made detection avoidance difficult for malicious users. We also describe in detail an evaluation of our proposed method and comparative methods and the favorable results we have obtained in various experiment scenarios.