CSE1110 - Software Quality and Testing

HomeTeaching TeamContactScheduleJPacman labworkGradingFAQ

This course covers the most important testing techniques needed to build high quality software systems. Specific topics covered are quality attributes, maintainability and testability, manual and exploratory testing, automated testing, devops, test adequacy, model-based testing, state-based testing, decision tables, reviews and inspections, design-by-contract, test-driven design, unit versus integration testing, mocks and stubs.

Our learning goals are:

  • Participants are able to create unit, integration, and system tests using current existing tools (i.e., JUnit, Mockito, and JaCoCo) that successfully test complex software systems.

  • Participants are able to derive test cases that deal with exceptional, corner, and bad weather cases by performing several different techniques (i.e., boundary analysis, state-based testing, decision tables).

  • Participants are able to measure and reflect about the efficiency of the developed test suites by means of different test adequacy metrics (i.e., line, branch, condition, MC/DC coverage).

  • Participants are able to reflect about limitations of current testing techniques, when and when not to apply them in a given context, and to design testable software systems.

  • Participants are able to write maintainable test code by avoiding well-known test code smells (e.g., Assertion Roulette, Slow or Obscure Tests).

A lot of the material of this course is also available in our EdX MOOCs on Automated Testing (Practical and Advanced skills for Java Developers). While these MOOCs are archived, if you’re interested you can still watch the videos and the exercises.

Teaching team

Frank Mulder
Casper Boone
Head TA
Max Lopes Cunha
Head TA
Cedric Willekens
Danny Plenge
David Moolenaar
Denise Toledo
Dylan Franken
Issa Hanou
Jasper Teunissen
Jordi Smit
Khalid El Haji
Leonard in 't Veen
Mirco Kroon
Nima Salami
Otto Kaaij
Rembrandt Klazinga
Rico van Buren
Robin Kouwenhoven
Shaad Alaka
Sunwei Wang
Wouter Polet
Yana Angelova
Yoshi van den Akker
Felix Dekker


  • Questions about the content: Brightspace forum, TAs during labwork, teachers during the break (no e-mail to the teachers).
  • Infrastructure problems: contact Casper Boone or Max Lopes Cunha.
  • TA availability: lab sessions. No direct Mattermost messages.
  • Personal questions, mail Maurício and Arie.


Welcome to SQTE and Revisting JUnit 23/April, 8:45am, Arie van Deursen & Maurício Aniche
  • Welcome to SQTE
  • The JPacman Labwork
  • Midterm and exam
  • Revisiting the JUnit framework
  • Introduction: (pdf)
  • Labwork slides: (pdf)
  • (Automated) software testing (automation): (pdf)
  • Pragmatic software testing at SIGCSE 2019: (pdf)
Important actions for after/before the lecture: Additional material: Watch our summary videos:
Introduction to testing and testing principles 26/April, 10:45am, Arie van Deursen
  • Chapters 1, 2, and 3 of our book
  • Faults and failures
  • Principles of testing
  • Testing in the software development life cycle
  • Validation and verification
  • The V-Model
  • Static analysis
  • Foundations, ch. 1-3 (pdf)
Additional material: Watch our summary videos on the topic:
Specification-based and Boundary Testing 30/April, 8:45am, Maurício Aniche
  • Chapter 4 of our book
  • Functional testing (testing from requirements)
  • Boundary Testing
  • On and off points
  • Specification-based and boundary testing: (pdf)
Additional material: Watch our summary videos:
Model- and State-based testing 1/May, 10:45am, Arie van Deursen
  • Chapter 4 of our book
  • Model-based testing
  • Decision tables
  • State-based testing
  • Model-based testing and Decision Tables: (pdf)
  • Model-based testing and State Machines: (pdf)
Additional Material:
  • Arie van Deursen. Beyond Page Objects: Testing Web Applications with State Objects. ACM Queue, 2015. (html).
Watch our summary videos:
  1. Decision Tables
  2. State Machines
  3. Interview with Lionel Briand on Model-Based Testing in Practice.
Structural testing 3/May, 10:45am, Maurício Aniche
  • Chapter 4 of our book
  • Structural testing
  • Line, branch, decision, condition coverage
  • MC/DC coverage criteria
  • Coverage subsumption
  • An overview of passive learning: (pdf)
  • Structural testing: (pdf)
Watch our summary videos:
3/May Deadline: Part 0
You should deliver part 0 of the assignment. See instructions on how to deliver it. Deadline: 6pm.
Pragmatic exploratory testing 7/May, 8:45am, Jan Jaap Cannegieter (Squerist)

Jan Jaap Cannegieter is a leading test, QA and requirements expert from the Netherlands. He had over 25 years of experience in ICT. Jan Jaap did assignments in testing, test automation, quality assurance, process improvement, requirements, business – ICT alignment, Agile and digitalization. Jan Jaap is Principal Consultant at Squerist, a consultancy company of 100 employees specialized in process management and testing. Within Squerist Jan Jaap is, beside his assignments, responsible for coaching, knowledge management and product development. Jan Jaap is the well-known author of several articles and books in the Netherlands and is a well-known (keynote) speaker on international conferences.

  • Exploratory testing: (pdf)
Additional material:
  • Our interview with Jan Jaap on Exploratory testing: Part I and Part II
10/May Deadline: Review Part 0
You should deliver your peer review of part 0. See instructions on how to deliver it. Deadline: 6pm.
Testability and Mock Objects 10/May, 10:45am, Maurício Aniche
  • What's testability?
  • Design for testability
  • Mock objects
Watch our summary videos: Slides: Additional material:
Security Testing, Part I 13/May, 3:45pm, Azqa Nadeem
  • Classifications of security testing
  • Java SAST tools and techniques
  • SAST under the hood
  • Security testing: (pdf)
Security Testing, Part II 15/May, 8:45am, Sicco Verwer
  • Dynamic Application Security Testing (DAST)
  • Fuzzing
  • Fuzzing for vulnerability hunting
Test-Driven Development (TDD) 17/May, 10:45am, Maurício Aniche
  • Short talk: How the solar team tests their car (Pieter Tolsma)
  • Test-Driven Development (TDD)
  • Advantages and disadvantages
  • Pragmatic TDD
Slides: Suggested reading:
  • Growing Object-Oriented Software Guided by Tests, by Freeman and Pryce. Best book on the topic.
20/May Deadline: Part 1
You should deliver part 1 of the assignment. See instructions on how to deliver it. Deadline: 6pm.
24/May: Midterm
Check MyTimeTable for location.
28/May Deadline: Review Part 1
You should deliver your peer review of part 1. See instructions on how to deliver it. Deadline: 6pm.
Web testing 29/May, 3:45pm, Frank Mulder
  • JavaScript unit testing
  • UI component testing
  • Web end-to-end testing
Design by Contract and Property-based testing 7/June, 8:45am, Arie van Deursen
  • Design by contract
  • Pre-conditions, postconditions, and invariants
  • The Liskov Substitution Principle and Testing Class Hierarchies
  • Property-based testing
  • Selected questions from the mid-term exam
Slides: Additional material: Watch our summary videos:
Search-based software testing 11/June, 8:45am, Annibale Panichella
  • Search-based software testing
  • Genetic algorithms
  • Fitness function
  • Automated test case generation
Slides: Watch our summary videos: Readings:
5/June Deadline: Part 2
You should deliver part 2 of the assignment. See instructions on how to deliver it. Deadline: 6pm.
The triangle problem. How to deal with breaking changes in heavily-used (open-source) projects 14/June, 8:45am, Tim van der Lippe (Google, former TU Delft student)

Tim van der Lippe is a software engineer at Google in London. He is a former student of Delft University of Technology and former teaching assistant of this course. During his study and in his current job, he is the maintainer of two open source projects: web components library Polymer and Java mocking framework Mockito. In this guest lecture, he will give an overview of his daily work at Google, the problems of working at scale and what solutions were necessary to migrate Google to the latest version of Mockito.

Additional material:
14/June Deadline: Review Part 2
You should deliver your peer review of part 2. See instructions on how to deliver it. Deadline: 6pm.
Testing the Spring Framework 20/June, 3:45pm, Stéphane Nicoll (Pivotal)
Stéphane Nicoll has a thing for code quality and robustness. He's been spreading the word for more than ten years while developing large scale Java enterprise applications in the geospatial, financial, or logistics sectors. An Apache Maven PMC member since 2006, he joined the core Spring Framework development team early 2014, being one of the main contributors to both Spring Framework and Spring Boot since.
21/June Deadline: Part 3
You should deliver part 3 of the assignment. See instructions on how to deliver it. Deadline: 6pm.
27/June Deadline: Review Part 3
You should deliver your peer review of part 3. See instructions on how to deliver it. Deadline: 6pm.
2/July: Final exam
Check MyTimeTable for location.


Your final grade is calculated as follows:

Grade = (L + 2 * max(M, E) + 2 * E) / 5


  • L is the labwork (L being the average grade of the three parts of the labwork) + peer review.
  • M is the midterm
  • E is the exam

The midterm and the exam both contain 40 multiple-choice question, with 4 options each. Their final grade is calculated as:

Grade = (C - W/3) / 4


C is the number of correct answers and W is the number of wrong answers. You are allowed to not answer a question; in this case, it won’t count as a wrong answer, and thus, not part of the guessing adjustment. With such a formula, we suggest you to not answer a question if you have no knowledge of the question; on the other hand, if you have partial knowledge (i.e., you can discard 1, 2, or 3 choices out of the given 4), it is therefore best to guess.

Note that you are required to have L >= 5.75 and E >= 5.75 to pass. If you do not pass both the labwork and exam, your final grade is calculated as min(E, 4.0).


  • Will there be collegerama recordings of this course? In the schedule above, we provide you with summary videos for all our lectures. For that reason we will not make TU Delft collegerama recordings.

  • Are the deadlines strict? Yes, deadlines are strict. If you miss the deadline, you get a 0. If you had an issue that prevented you from delivering, please talk to your student counselor.

  • Can I skip the midterm? Yes, you can. The midterm is not compulsory. However, note that the midterm can count as 40% of your final grade. You have less material to study for the midterm too. So, our suggestion is for you to study hard and take the midterm!

  • My friend is from group Beta, and I’m from group Alpha. Can we still be a team? Yes, you can. Keep in mind that your lab times will still be different. In moments where the lab is busy, TAs will prioritize students from that lab; on the other hand, in times that the lab isn’t too busy, you can work together there.

  • I did the JPacman assignment last year. Can I reuse the grade? If you did the JPacman assignment in 2018 and you got a passing grade, you do not need to do it again. Go to Brightspace -> Assignments -> Reuse 2018 JPacman grade, and reply it with a “Yes”.

  • Can I use my devices in class? Educational research clearly shows that using devices on non-class related activities can harm your learning. Moreover, it can also harm your colleague’s learning. Therefore, you are not allowed to use any devices during the lecture. Exception: activities that require devices. I will help you by making announcements during the lecture about the times you should use your device.

  • How did you do this website? This website is done with Jekyll and some straightforward HTML. The main picture was taken by Agence Olloweb.

  • Can I use your material? Attribution-NonCommercial-ShareAlike CC BY-NC-SA This website as well as all the content created by me are licensed through Attribution-NonCommercial-ShareAlike CC BY-NC-SA. This license lets others remix, tweak, and build upon your work non-commercially, as long as they credit you and license their new creations under the identical terms. The content from others that I link are subject the own authors’ licenses.