F2Attack: Two-Factors Scoring Method for Query-Efficient Hard-Label Black-Box Textual Adversarial Attacks

In the hard-label black-box setting, existing attack methods randomly select words for perturbation, generating invalid word replacement operations, resulting in low attack success rate. Recent works alleviate this problem by evaluating the impact of words on model predictions, but they can only evaluate the impact of words on model predictions, not the impact of words on attack. If the attacker replaces too many words that have significant impact on the text semantics during the attack process, the adversarial example has poor semantics and the attack behavior is easily detected. To address the above issues, this paper proposes a two-factors word scoring method, which uses the attention score output by the pre-attack model and the semantic similarity after word replacement to evaluate the impact of the word on attack. Based on the scoring method, this paper proposes a query-efficient hard-label black-box adversarial attack method called F2Attack. F2Attack uses the two-factors method to score words, and then replaces words have great impact on the model predictions but small impact on text semantics based on scoring results to generate the initialized adversarial example. Then, F2Attack adopts the simulated annealing algorithm to optimize the semantic similarity of the adversarial example. We conduct experiments on four representative natural language models, seven text classification datasets, two natural language inference datasets, and four commercial APIs, and compare them with baseline methods. Taking text classification as an example, when the number of queries is limited to 100, F2Attack increases the attack success rate by an average of 15.165%, and the semantic similarity by 0.067, which is significantly better than the baseline methods.