FuzzDocs: An Automated Security Evaluation Framework for IoT

As Internet of Things (IoT) devices have rooted themselves in the daily life of billions of people, security threats targeting IoT devices are emerging rapidly. Thus, IoT vendors have employed security testing frameworks to examine IoT devices before releasing them. However, existing frameworks have difficulty providing automated testing, as they require a lot of manual effort to support new devices due to the lack of information about the input formats of the new devices. To address this challenge, we introduce FuzzDocs, a document-based black-box IoT testing framework designed to automatically analyze publicly accessible API documents about target IoT devices and extract information, including valid inputs used to call each functionality of the target devices. Based on the extracted information, it generates valid-enough test inputs that are not easily rejected by target devices but can trigger vulnerabilities deep inside them. This document-based input generation allows FuzzDocs to support new devices without manual work, as well as provide effective security testing. To prove its feasibility, we evaluated FuzzDocs in a real-world IoT environment, and the results showed that FuzzDocs extracted input formats with 93% accuracy from hundreds of pages of documents. Also, it outperformed the existing frameworks in testing coverage and found 35 potential vulnerabilities, including two unexpected system failures in five popular IoT devices.

Thus, IoT devices are now capable of more advanced and 23 complicated functions than in their early stages, and they have 24 become an essential component of people's daily lives. 25 Now that IoT devices can manage and store much more 26 data, including some sensitive information, IoT security has 27 The associate editor coordinating the review of this manuscript and approving it for publication was Barbara Masucci . become a major issue. We can easily see many real-world 28 cases showing how IoT devices are vulnerable [1], [2], [3]. 29 Worse, the overheated IoT market encourages IoT vendors 30 to aggressively release new devices, but a lack of budget 31 discourages them from conducting sufficient security checks. 32 For the same reason, even if vulnerabilities are found in 33 already-released devices, IoT vendors are unlikely to have the 34 chance or capability to fix the vulnerabilities properly. 35 One practical solution to address this problem is to iden-36 tify vulnerabilities in IoT devices using automated tools and 37 take appropriate security precautions. Several researchers 38 have proposed a series of solutions to test the security of 39 IoT devices, ranging from fuzzing device firmware through 40 emulation [4], [ [13]. 42 FuzzDocs extracted correct input formats with 93% accuracy   In summary, our contributions are as follows: 96 • We present a test input generation model leveraging 97 API documents for automatically identifying potential 98 vulnerabilities in target IoT devices.

99
• We present the design and implementation of a new 100 document-based testing framework for IoT, called Fuz-101 zDocs, which is capable of automatically generating the 102 test inputs from the API documents and detecting the 103 vulnerabilities in the target devices.

104
• We evaluate FuzzDocs in real-world environments, 105 demonstrating that the effectiveness of FuzzDocs is 106 superior to existing solutions and discovering 35 poten-107 tial vulnerabilities in five IoT devices. The rest of this paper is organized as follows. Section II 111 gives the required background and our insights of FuzzDocs 112 with a running example. Section III reviews the previous 113 studies and their limitations. The overall system design is 114 presented in Section IV. Section V and VI detail two key 115 modules of FuzzDocs, respectively. The evaluation results are 116 summarized in Section VII, and Section VIII discusses the 117 limitations of the current design. Finally, we conclude this 118 paper in Section IX. Unlike traditional embedded devices, most IoT devices inter-122 act with other endpoints (i.e., devices or users) over the 123 network. As IoT devices become more diverse and com-124 plex, interoperability between these devices gains impor-125 tance. Thus, most IoT vendors open application programming 126 interfaces (APIs) to effectively interact with their IoT devices. 127 The APIs allow developers to build various IoT ecosystems 128 (e.g., smart homes and smart factories) by connecting multi-129 ple IoT devices, regardless of vendor.
130 Figure 1 shows an overview of the communication model 131 of modern IoT devices. APIs and the API module enable the 132 endpoints to invoke the IoT device's functionality by send-133 ing request messages [9], [10], [11], [16]. For example, the 134 endpoints transmit a request message to the device through 135 the network (e.g., using Wi-Fi). The device then parses the 136 FIGURE 2. Example of partial API document for a popular IoT device [15]. The API document is publicly accessible and provides detailed information (i.e., API specifications) needed to create a valid request message to call the API.   There are many ways to implement the APIs in the IoT 149 devices, but most IoT vendors adopt HTTP-based APIs (e.g.,

150
REST API) due to their flexibility, scalability, and simplic-   specification by splitting it into sections. The tables or lists in 173 each section are employed to describe API specifications that 174 contain multiple descriptions (e.g., parameters).

176
As discussed earlier, the API document guides the user 177 through the process of creating an valid request message 178 to execute a specific function (API) of the IoT device. For 179 example, when calling the 'settings' API, we must set the 180 'default_state' parameter to one of the three predefined 181 strings ('off', 'on', or 'last') as described in the 'Param-182 eters' section in Figure 2. When we send a valid request 183 message created according to the requirements, the device 184 sends the response message containing the API execution 185 results, as shown in Figure 3a. 186 Let us consider an example in which a user intentionally 187 builds an invalid request message with parameters that vio-188 late the requirements from the API document in Figure 2. 189 For instance, we set the 'default_state' parameter to 190 a random string rather than the predefined values. If the 191 device appropriately handles the invalid request message, 192 it sends the response message indicating that the message 193 includes invalid parameters (e.g., HTTP 400 bad requests), 194 as shown in Figure 3b. However, some invalid request mes-195 sages cause unexpected errors that the device cannot handle. 196 If we set 'default_state' to a very long random string, 197 unexpected errors such as buffer overflows may occur when 198 the device processes the parameter. In this case, the device 199 sends response messages indicating the unhandled errors 200 (e.g., HTTP 500 internal errors), which could lead to severe 201 security vulnerabilities. Figure 3c shows the result of this 202 example, which makes the device unable to respond to any 203 further request messages until it is rebooted (i.e., DoS).  [16], [23], [24], [25], [26], 218 [27], [28], [29] have emerged to discover security bugs in IoT 219 devices, and most of them adopt black-box fuzzing due to the 220 closed source of target devices and the ease of testing.  [25], [26], [27], [28], and 225 companion-app-based fuzzing [9], [11], [16]. However, these 226 prior studies have two limitations: (i) the amount of manual 227 effort required (i.e., seeds or settings), and (ii) restricted test-228 ing coverage. These limitations make these prior studies hard 229 to automate and also reduce their efficiency. We summarize 230 the prior studies in Table 1  In order to test IoT devices without firmware, researchers 260 have developed network-based fuzzing [10], [25], [26], [27], 261 [28] that tests the security of IoT devices by sending mutated 262 messages directly to the devices over networks. However, 263 most IoT devices have strict message formats in network 264 communication and reject invalid messages. That is, input 265 messages created in a brute-force manner are easily rejected 266 by the devices, reducing the effectiveness of fuzzing tests [9], 267 [11]. Thus, network-based fuzzing first takes a set of seed 268 messages (or grammar) from security experts and mutates the 269 seed messages to create input messages for testing. After this 270 method sends the mutated input messages to the IoT devices, 271 it analyzes the response messages to detect potential vulnera-272 bilities. Some researchers [10] use the response messages as 273 feedback to create subsequent input messages, increasing the 274 possibility of triggering more unusual responses.

275
Network-based fuzzing does not require firmware analysis, 276 but well-formed seed messages are essential, which means its 277 success depends strongly on the quality of the seed messages. 278 Also, if the seed messages cannot invoke a specific API of 279 the target devices, the API cannot be tested by network-based 280 fuzzing, which significantly reduces the coverage of testing. 281 Moreover, creating high-quality seed messages requires a lot 282 of manual effort by security experts, which is not suitable for 283 automated testing. In a parallel line of work, researchers [9], [11], [16] leverage a 286 companion application, which is a mobile app for IoT devices 287 instead of firmware. This method creates input messages by 288 analyzing companion apps. Technically, it retrieves message 289 creation functions of companion apps using reverse engineer-290 ing. Then, it mutates the retrieved functions' parameters at 291 runtime to send mutated input messages to IoT devices.

292
However, this method suffers from two fundamental prob-293 lems. The first is the limited fuzzing coverage; this method 294 cannot guarantee that all the device's APIs (functionalities) 295 are fully tested because there still remain the APIs not invoked 296 during the automated companion app execution. In this case, 297 security experts need to manually click all the UIs in the app 298 to trigger the remaining APIs, which takes a lot of manual 299 effort and is not suitable for automated testing. Moreover, this 300 method is unable to test APIs that are not called through the 301 app's UI, such as APIs that change internal device settings 302 (see Section VII-C1). Second, if the companion app is obfus-303 cated, this method is impractical because code obfuscation 304 prevents app reverse engineering. Such code obfuscation has 305 become very common in mobile environments due to the 306 surge in mobile app security issues [33].

308
Firmware-based fuzzing is not practical for automated test-309 ing. Other fuzzing methods still require a lot of manual work 310 and have limited test coverage due to the lack of information 311 about IoT devices' input message formats. FuzzDocs can 312 TABLE 1. Summary of related work on IoT device fuzzing. While existing solutions require firmware analysis or a lot of manual work by security experts, such as generating seed messages, FuzzDocs can automatically test the security of IoT devices without such a burden.   also extract API specifications from the document manu-337 ally. To resolve these problems, our framework centers on 338 automating the API specification extraction process. In general, IoT devices reject invalid API request messages 341 at early stages (i.e., message parsing phases) that violate their 342 predefined message format. If we know the detailed message 343 formats for each target device, we can easily create input 344 (request) messages that are valid enough not to be easily 345 rejected by the devices but can trigger unexpected internal 346 errors beyond the parsing state. However, manually creating 347 such detailed message formats for target devices is time-348 consuming. Therefore, our framework needs to automatically 349 generate effective and detailed message formats for each API 350 to create valid enough input messages for testing. It is important to recognize if the target IoT device is entering 353 an abnormal state during security testing. The best way to 354 know the internal states of the device is to directly examine 355 state information through its firmware. In other words, with-356 out direct access to the firmware, it is challenging to identify 357 the device's internal state. This lack of state information 358 makes it hard to determine whether the input messages sent 359 by our framework are triggering potential security bugs on 360 the device. Thus, our framework should provide a remote and 361 automated mechanism to recognize the internal states of the 362 target device.  Figure 4 illustrates the key modules of FuzzDocs and its over-365 all workflow. There are two main modules: the information 366 extraction module (top) and the fuzzing module (bottom). 367 The information extraction module takes and parses API 368 documents in various formats to create a layer of abstrac-369 tion around the target IoT device's API request message 370 formats. (1) First, it extracts API specifications (e.g., URLs, 371 parameters) required for creating valid API request messages 372 from the input API document. When doing so, it uses text-373 based document parsing to process API documents regard-374 less of their format (R1). (2) Next, it utilizes natural lan-375 guage processing (NLP) techniques to analyze the extracted 376 information and create a message format for each API. Each

393
The key idea of the information extraction module is to unnecessary parts and improve parsing efficiency. First, 430 it removes the optional areas (e.g., sidebars and naviga-431 tion bars) in the API document, which have no meaning-432 ful text for the API specifications. Next, it replaces the 433 tags used to emphasize keywords (e.g., <span>, <strong>, 434 <italic>) with double quotes when these tags are within 435 the text. For instance, 'The <span>brightness</span> from' 436 is replaced with 'The ''brightness'' from.' Normally, such 437 tags are employed to highlight the keywords (e.g., parameter 438 names), so this tag replacement helps the module extract 439 important keywords for the API.

440
After the above preprocessing, the module builds the 441 section tree as described in Algorithm 1. It starts (line 6) by 442 traversing all the HTML tags in the API document (T ) in a 443 depth-first search (DFS) order to read the tags in the order 444 they appeared in the API document. For each tag, it checks to 445 see if there is text between the opening (<tag>) and closing 446 (</tag>) pairs of the current tag (t i ) (line 7). Note that the 447 module does not consider the text inside a tag's attributes 448 (e.g., href attribute). If the current tag has text, it creates a 449 new node for the current tag, which contains a type, the found 450 text, and an insertion index (idx). The insertion position of the 451 new node depends on whether the current tag is a title tag (H ). 452 In the case of the title tag, the module compares the priority 453 of the last inserted title tag (l t ) with the current tag (line 10). 454 VOLUME 10, 2022 FIGURE 5. Each section in the API document (a) and a partial section tree originated from it (b) and processed by the information extraction module. Regardless of the document format, the section tree captures the hierarchical relationship between the texts in the API document and identifies candidate nodes for each API specification. The lower the level of the title, the higher the priority (i.e., 455 h1 > . . . > h6). If the current tag has a lower priority than 456 the last title tag (line 11), the new node becomes a child 457 node of the node for the last title tag (i.e., the last title node).  expression-based method [35] to easily discover them. In gen-480 eral, the combination of the URL and HTTP methods is a 481 unique identifier for APIs, meaning that they usually appear 482 at the beginning of the API document. These characteristics 483 are also reflected in the section tree. Thus, if the module 484 finds multiple candidate nodes that include the URL/HTTP 485 method, it selects the node with the lowest index for URL and 486 HTTP methods. It then inserts the selected candidate nodes 487 into the node pool.

488
Next, the module should find candidate nodes for the 489 parameters, but the parameters have no unique patterns, 490 unlike the URL and HTTP methods. Also, parameters are 491 divided into various types according to their position in 492 an input message (e.g., path or body). These characteris-493 tics prevent the module from determining which nodes are 494 associated with which type of parameters. To resolve this 495 problem, the module leverages title nodes as an indicator 496 for locating the candidate nodes for parameters. For this, 497 we build a dictionary for each type of parameter by collecting 498 the keywords used as titles in the parameter sections from 499 about 100 API documents for 10 popular IoT vendors [15], 500 [36], [37], [38], [39], [40], [41], [42], [43], as shown 501 in Table 2.

502
The module leverages the dictionary for the parameters as 503 follows. It first selects title nodes containing any combination 504 of suffix and prefix. In the section tree shown in Figure 5 505 (b), the ''Path Params'' and ''Body Params'' are selected. 506 Then the module designates the child nodes (only non-title 507 nodes) of the selected title nodes as the candidate nodes for 508 corresponding parameters (i.e., path parameters and body 509 parameters) and pushes them into the node pool. In Figure 5 510 (b), the nodes from index 6 to 8 become candidate nodes for 511 path parameters. Identifying examples (i.e., 517 specific code) from the API document is not a new topic, 518 so we can just adopt a regular expression-based approach 519 that is similar to prior work in this field [44]. The module 520   The basic block includes the essential parts of the valid input 533 messages, so the module begins by analyzing the candidate 534 nodes for the URL and HTTP methods from the node pool.

535
Filling the HTTP methods is straightforward because we just 536 adopt the value from the HTTP method candidate nodes (e.g.,

537
PUT, DELETE). In contrast, the URL part typically uses a 538 local IP address of the device as a hostname to call the API.

539
However, the hostname are not included in API documents 540 for security reasons, meaning that they are represented as 541 symbolic values (e.g., <IP address>/api/light). For this rea-542 son, the module receives the hostname from the administrator 543 and replaces the symbolic values in the URL. In general, all 544 the APIs of one IoT device use the same hostname, so it 545 receives the hostname only once from the administrator dur-546 ing the entire testing process. Note that FuzzDocs performs 547 a ping test to verify whether the hostname is valid and sends 548 alerts if the test fails.

549
Other symbolic values inside the URL could be the value 550 of a path or query parameter, and these values are typically 551 denoted via the following syntactic symbols [45].  Next, the module analyzes the parameter candidate nodes 561 to find the names of each parameter used for the input mes-562 sages calling the API. As shown in the upper part of Figure 6, 563 the parameter candidate nodes offer plenty of information 564 about what the names and value types are, but we have no 565 idea yet which node is related to which parameter names. 566 To address this problem, the module leverages the linguistic 567 units in a node's text. As shown in Table 3, the linguistic 568 units in the text of the node vary depending on what kind of 569 parameter information the text represents. For example, if a 570 node's text is a pre-defined keyword used for data types (e.g., 571 string, integer, or array), it indicates a parameter's value type. 572 Similarly, if a node is a noun and not a pre-defined keyword, 573 the node is the name of a specific parameter. Based on these 574 findings, the module locates nodes containing a parameter's 575 name and feeds the names to the basic block. The elements in the advanced block are usually the value 578 types, required flags (e.g., optional or mandatory), and a 579 range of allowed values that are used in the parameters. To fill 580 this block, the module revisits the parameter candidate nodes. 581 In the pool, we have marked the nodes with a parameter 582 name, but we should still determine which parameter name 583 the remaining nodes belong to. In general, when writing an 584 API document, the author first gives a parameter's name 585 and then arranges other information about that parameter. 586 Thus, we leverage the index (idx) of the node we assigned 587 when building the section tree in the previous step. As shown 588 in Figure 6, the nodes '11: power' and '14: color' are the 589 names of body parameters. The nodes between them (i.e., '12: 590 string' and '13: The power state you. . . ') belong to the node 591 '11: power' because they are not the name and have a lower 592 index than the next parameter name node, '14: color'. In this 593 manner, the module inserts each parameter's information 594 (e.g., value types and descriptions) into the advanced block. 595 VOLUME 10, 2022  Table 4. names and value types. This information is used when the 641 module fails to identify or analyze body parameter candidate 642 nodes. 644 We have discussed creating message formats by parsing the 645 API documents of the IoT device. With these message for-646 mats, the fuzzing module can examine security vulnerabilities 647 in the target IoT device by conducting the following two 648 steps as shown in Figure 7: (i) input message creation and 649 (ii) response analysis.

651
The left part of Figure 7 shows input message creation. 652 Note that each message format received from the information 653 extraction module is matched one-to-one with an API of the 654 target IoT device. The fuzzing module first selects one of the 655 message formats according to a user-configurable scheduling 656 algorithm (e.g., a round robin). Then, the module uses the 657 message format as a template and fills the input message with 658 the basic block (e.g., URL, parameter names) without specific 659 values.  After transmitting the created input message, the module 719 listens to the response from the target device to know whether 720 the message causes unexpected errors (e.g., crashes) that can 721 lead to security vulnerabilities. Given that we cannot instru-722 ment the device's internal state locally, the fuzzing module 723 automatically analyzes the response message to the corre-724 sponding input message remotely and determines whether the 725 input message caused errors. The right of Figure 7 shows the 726 response analysis step. In particular, the module considers 727 an input message to be potentially error-prone if any of the 728 following conditions are met.

729
• Timeout: Before testing an API, the module trans-730 mits a normal request message (i.e., without mutation) 731 10 times with an interval of 1 second, then calcu-732 lates the average of the response time to set a time-733 out for that API. Next, when conducting the actual 734 testing for the API, it sends an input message three 735 times because the network environment can affect the 736 timeout. When the timer expires three times, the mod-737 ule gives an alert that a DoS vulnerability has been 738 triggered.

739
• Connection Lost: If the device abruptly closes an active 740 connection, the module considers it a sign that the device 741 has fallen into the error state. The module monitors 742 for cases where the device sends TCP RST packets in 743 response to the input message.

744
• HTTP Internal Server Error (500): If the device 745 returns a response message with Internal Server Error 746 (500), it indicates that the device has entered an error 747 state due to the input message.

748
Coverage Feedback Mechanism. Generally, a fuzzer 749 optimizes the input mutation process based on the feedback 750 of executions to find security vulnerabilities more effectively. 751 In the absence of a feedback mechanism, fuzz testing could be 752 blind during input mutation (creation). However, the fuzzing 753 module in FuzzDocs optimizes the input message creation 754 step by using the response message from the target device as 755 feedback, which improves the effectiveness of vulnerability 756 discovery. As shown in Figure 7, the module maintains a 757 response dictionary that stores the contents of the response 758 messages in previous input message transmissions. Thus, 759 whenever the module sends an input message, it stores the 760 contents of the corresponding response message (e.g., status 761 code, keys of a JSON body) in the dictionary. If an input mes-762 sage causes a new response message that does not match the 763 response dictionary, this new response message indicates that 764 a new code block has been executed in the device firmware. 765 Then, the module inserts all the values of parameters in the 766 input message into the parameter value dictionary as well. 767 The inserted values are utilized as starting points for the next 768 input message creation, which could directly explore more 769 code coverage.

771
In this section, we present the prototype implementation 772 of FuzzDocs and evaluate it in real-world environments to 773 demonstrate its effectiveness and efficiency.  Wi-Fi or Bluetooth, but for convenience, we will only test 804 the devices that communicate via Wi-Fi in this experiment.

805
Also, the API documents of the device to be tested should be 806 publicly available. Thus, as shown in Table 5 network-based fuzzers [26], [27]  As IoT devices typically operate on limited hardware 846 resources, sending extremely input messages to the device 847 can make the experimental results worthless. Taking this into 848 account, we limited the maximum number of input mes-849 sages the three frameworks send to IoT devices to 10 per 850 second during the entire experiment process, and then we 851 inspect each API for an hour. As a result of our experiments, 852 FuzzDocs discovered 35 crashes in a total of nine APIs, 853 including two DoS vulnerabilities (Shelly Plugs and Duo) as 854 summarized in the right part of Table 5.

855
In particular, FuzzDocs discovered at least one crash on 856 every single device used in the experiment, and in some 857 cases, it found multiple crashes in one API (e.g., Shelly 858 PlugS, shown in Table 5). This means that based on the 859 created message formats,FuzzDocs successfully generated 860 multiple input messages with different combinations of the 861 parameters, which led to causing such distinct crashes in each 862 API. Also, FuzzDocs only spent about 10 minutes on average 863 before finding a crash in each vulnerable API. In contrast, 864 Ffuf and Doona did not find any crashes other than the one 865 found on the Hue bridge. This is because, unlike FuzzDocs, 866 these frameworks create input messages without consider-867 ing the message formats of IoT devices. Most of the input 868 messages created by them are filtered during the message 869 parsing stage of the IoT devices before triggering bugs deep 870 inside the IoT devices. As a result, FuzzDocs can provide 871 more comprehensive and broad test coverage than existing 872 network-based fuzzers by automatically creating message 873 formats for IoT devices. 874 2) CASE STUDY 875 One of the interesting results is a crash that led to a denial 876 of service vulnerability discovered in Shelly PlugS, which 877 is a smart plug device. The smart plug is a key device in 878 home automation services and provides residential energy 879  Next, we evaluated the coverage of the API, which is impor-933 tant for meeting the requirements of an intelligent assess-934 ment system. To this end, we compared the API coverage of 935 FuzzDocs with the ones of existing studies [9], [11], which 936 create input messages using companion apps. One especially 937 noticeable point is that these previous studies only tested APIs 938 that could be called through the UI of companion apps and 939 required manual interaction with the UI at the initial stage.

940
For the comparison, we calculated the number of APIs that 941 could be called from companion apps by first referring to 942 the previous studies. We installed each device's companion 943 app and a traffic monitoring tool on our test mobile phone 944 (Android OS). The monitoring tool started to collect all traffic 945 sent from the mobile phone to the devices as soon as the 946 companion apps launched. We manually simulated all UI 947 inside each companion app according to the vendor-provided 948 manuals, as in the previous studies. After the simulation, 949 we obtained the number of APIs by analyzing the captured 950 traffic. Shelly devices provide additional web-based UI with 951 the companion app, so we conducted the same simulation for 952 the web UI and merged the results. 953 Figure 9 shows the experimental results. Overall, the com-954 panion apps were able to call fewer APIs than listed on the 955 API documents (see Figure 9 (a)) because most of the APIs 956 not available in companion apps are developer-only APIs. 957 Developer-only APIs are used to modify the internal state of 958 devices, so they are not invoked through the companion apps 959 to prevent user error.

960
In addition to the available APIs, the number of parameters 961 that the companion apps use for each API is also restricted, 962 as shown in Figure 9 (b). For example, when calling the 963 '/light/' API of the Shelly Duo, the companion apps 964 cannot use two of the seven parameters listed in its API docu-965 ment, 'effect' and 'transition'. This lack of available 966 APIs and their parameters significantly degrades the coverage 967 of the companion-app-based fuzzing. In contrast, FuzzDocs 968 can test any API in the API documents without parameter 969 restrictions. 970 We did attempt to analyze the Hue companion app's traffic, 971 but we could not obtain correct results because the traffic 972 was encrypted and we were unable to decrypt it. Instead, 973 we manually compared all the UIs in the Hue companion 974 app with the APIs described in the API document. Similar 975 to other devices, we could not modify the device's internal 976 settings through the app's UI. For example, the configuration 977 API [53] of Hue devices can change various internal settings, 978 such as the device's IP address or proxy server, but we could 979 VOLUME 10, 2022 TABLE 5. Summary of IoT devices under testing and the crashes identified by FuzzDocs and existing network-based fuzzing frameworks (Ffuf [26], Doona [27]). The number in parentheses on the crash field indicates the number of APIs for which a crash was found.

COAP [56]
, are all similar in that they operate by exchanging 1013 request/response messages with various parameters, except 1014 for the underlying network protocols and structures for trans-1015 mitting messages. This means that FuzzDocs can be easily 1016 extended to other API implementations by modifying the 1017 method for assembling messages.

1029
In this work, we have proposed FuzzDocs, the first document-1030 based black-box IoT testing framework. To achieve more 1031 automatic and effective security testing for IoT devices, 1032 section-tree-based document parsing enables FuzzDocs to 1033 extract API specifications from human-readable API doc-1034 uments and creates valid-enough input messages based 1035 on them. Implementing the prototype of FuzzDocs and evaluating it in real-world IoT environments showed that FuzzDocs outperformed existing frameworks in testing coverage and