Case study: using AI-assisted code generation in mobile teams

We evaluate the performance of AI-assisted programming in mobile development teams that are focused on native mobile languages like Kotlin and Swift. The case study involves 16 participants and 2 technical reviewers, from a software development department and it is designed to understand the impact of using Large Language Models trained for code generation in particular phases of the team, more specifically, technical onboarding and technical stack switch. The study uses technical problems dedicated to each phase and requests solutions from the participants with and without using AI-Code generators. It measures time, correctness, and ‘technical integration’ using a new proposed metric ReviewerScore, extracted from industry standards, the code reviewer of merge/pull requests.The output is converted and analyzed together with feedback from the participants in an attempt to determine if using AI-assisted programming tools will have an impact on getting developers onboard in a project or helping them with a smooth transition between the two native development environments of mobile development, Android and iOS. The study was performed between May and June 2023 with members of the mobile department of a software development company based in Cluj-Napoca, with Romanian ownership and management.