In this correspondence, we analyze the vulnerabilities of biometric authentication protocols with respect to user and data privacy. The goal of an adversary in such context is not to bypass the authentication but to learn information either on biometric data or on users that are in the system. We elaborate our analysis on a general system model involving four logical entities (sensor, server, database, and matcher), and we focus on internal adversaries to encompass the situation where one or a combination of these entities would be malicious. Our goal is to emphasize that when going beyond the usual honest-but-curious assumption much more complex attacks can affect the privacy of data and users. On the one hand, we introduce a new comprehensive framework that encompasses the various schemes we want to look at. It presents a system model in which each internal entity or combination of entities is a potential attacker. Different attack goals are considered and resulting requirements on data flows are discussed. On the other hand, we develop different generic attacks. We follow a blackbox approach in which we consider components that perform operations on biometric data but where only the input/output behavior is analyzed. These attack strategies are exhibited on recent schemes such as the distributed protocol of Bringer (ACISP 2007), which is based on the Goldwasser-Micali cryptosystem, the related protocol of Barbosa (ACISP 2008), which uses the Paillier cryptosystem, and the scheme of Stoianov (SPIE 2010), that features the Blum-Goldwasser cryptosystem. All these schemes have been developed in the honest-but-curious adversary model and show potential weaknesses when considered in our malicious insider attack model.