Students’ Use and Non-Use of GAI tools in Revising Penetration Testing Reports

Penetration testing reports document cybersecurity vulnerabilities discovered and recommendations to remediate them. Thus, the reports need to be written clearly and structured for replication of test results, decision-making and action by a company’s senior management, IT management and IT technical staff. This study aims to explore students’ decision to use or not use genAI tools by examining their perceptions on using genAI tools, reasons for the use or non-use of genAI to revise their drafts for clarity and conciseness, cohesion and coherence, relevance to the audience, and professionalism and tone, their learning outcomes and the usefulness of the workshop. The students participated in a three-hour workshop embedded in a regular module delivered via zoom. During the workshop, the students were given the option to use or not use any genAI tools to help them perform the revision tasks in the workshop. The students performed the revision tasks in lab group teams and uploaded their revised drafts to the discussion forum in the learning management system for further feedback by the faculty. There was no advance support to students regarding any genAI tool as the aim of the study was to explore the above-mentioned aspects of their current use or non-use of genAI tools. They responded to an online survey at the end of the workshop. The findings revealed that the students were positive about the benefits of using genAI in helping them revise their penetration testing reports. This awareness, however, did not sway some students who chose not to use genAI tools as they wanted to understand and perform the task using their own thinking process. The genAI users mainly used ChatGPT and Copilot to perform the revision tasks. They found these tools helpful in assisting them to generate ideas, structure the report and perform the four revision tasks. Furthermore, they used these tools even though they were aware that moderate levels of editing on genAI’s output were required. The findings also showed that both groups of students mentioned similar learning outcomes from the workshop. However, the emphasis in the learning outcomes mentioned differed for both groups. This reflects that the critical elements in writing pedagogy with or without genAI tools might be students’ content and language proficiencies besides the ownership of their writing process.