Thesis: Can AI Agents Replace the Tester? Comparing LLM Browser Agents and Scripted End-to-End Tests

Scripted end-to-end tests can be costly to develop and maintain, while browser-controlling AI agents offer a different way to explore web applications for defects. This thesis will evaluate both approaches using open-source applications with intentionally introduced faults. The aim is to compare defect detection, reliability and overall cost, including model usage, maintenance and human review.

The assignment

• Set up one or two open-source web applications, preferably in a regulated domain with important workflows, access roles and data-integrity requirements.
• Create and reproducibly inject 30–50 realistic defects covering functional logic, input validation, authorization, data integrity and usability.
• Develop a scripted E2E baseline within a fixed time budget and compare it with unguided, persona-based and specification-guided agent exploration, as well as regression scripts generated by agents.
• Connect at least two LLMs to a real browser, comparing a larger, more capable model with a smaller, less expensive alternative.
• Measure detection rates by defect category, false positives, review effort, variation between runs and the reproducibility of reported issues.
• Introduce realistic interface changes to evaluate robustness and maintenance needs.
• Track implementation effort, model and compute costs, execution time and human review to estimate cost per run, cost per detected defect and long-term break-even points under different testing schedules.

Who we are looking for

• Studying towards a bachelor's or master's degree in computer science, software engineering or a related discipline.
• Expecting to graduate in June 2027.
• Fluent in both Swedish and English, in speech and writing.

Practical details

Based at Knightec Group's Sundsvall office. Start in January or March 2027. Apply through the company's careers page as soon as possible, no later than 30 November 2026. Questions can be sent to johanna.edstrom@knightec.se.