Background
Multi-modal large language models (M-LLMs) represent a cutting-edge type of artificial intelligence capable of processing, understanding, and generating multiple types of data, or "modalities," such as text and images. This versatility makes them highly promising for GUI testing, an area where traditional methods can be time-consuming and costly to maintain. Previous research into Large Language Models (LLMs) applicability for testing has primarily focused on low-level testing and unit tests, where this would be the next step in using artificial intelligence to raise and preserve quality in software.
This master thesis is done in collaboration with the Tactical Environment Simulation section (TES) which develop and maintain multiple simulation tools for a wide range of users. With multiple GUIs to maintain, the current GUI test strategy is manual verification at regular intervals, which is time-consuming and error-prone. By enabling more automatic test flows, the expectation is that the time to find and mitigate bugs would be decreased and the quality of the software would be raised.
Description of the Thesis
This master thesis aims to implement and evaluate the accuracy of M-LLMs in GUI testing. The work will encompass the entire process, from receiving prompts with test instructions to executing these instructions through a Python input control library (e.g., PyAutoGUI) and interpreting the results. The thesis will assess the effectiveness and reliability of M-LLMs in automating and enhancing GUI testing procedures. The GUI under test is not required to be the one of the TES tools, which enables more generic testing and control of the test environment.
This position requires that you pass a security vetting based on the current regulations around/of security protection. For positions requiring security clearance additional obligations on citizenship may apply.