Visual Software Quality Assurance using Vision Language Models

In this edition of our weekly SERG seminar, we will have a talk by a visiting faculty member from the University of Alberta, Cor-Paul Bezemer:

Abstract: Visual software quality assurance (QA) ensures that everything displayed on the screen looks as expected. Because visual QA is challenging to automate it is mostly done manually or through snapshot testing by comparing screenshots. However, manual testing does not scale well, and snapshot testing leads to many false positives for modern intelligent software due to its non-determinism. For example, AI-driven non- player characters (NPCs) in a game or personalized experiences on a website may cause differences between screenshots. To complicate matters, many visual anomalies (such as stylistic issues) require commonsense reasoning to detect, which traditional testing techniques cannot do. In my lab, we focus on visual quality assurance using vision language models. In my talk, I will give an overview of some of our most recent work on using vision language models for visual bug analysis for games.

Bio: Cor-Paul Bezemer (he/his) is an Associate Professor in the Electrical and Computer Engineering department at the University of Alberta. He is the Canada Research Chair (CRC) in Quality Assurance of Intelligent Systems. He heads the Analytics of Software, GAmes And Repository Data (ASGAARD) lab. Before that, he was a postdoctoral research fellow in the Software Analysis and Intelligence Lab (SAIL) at Queen’s University in Kingston, Canada. His research interests cover a wide variety of software engineering and performance engineering-related topics. His work has been published at premier software engineering and machine learning venues such as the TSE and EMSE journals and the ICSE, ESEC-FSE, CVPR and NeurIPS conferences. Before moving to Canada, he studied at Delft University of Technology in the Netherlands, where he received his BSc (2007), MSc (2009) and PhD (2014) degrees in Computer Science.