Improving ING's A/B testing pipeline
ING features an extensive A/B testing pipeline, that they use to run various experiments with, mostly for their web applications. ING is interested to improve the A/B testing pipeline and also to extend it to traditional applications (e.g., mobile or desktop).
Potential project ideas:
Analyzing the existing state: Why do some teams make extensive use of A/B testing while others shy away? What are the organizational hurdles that need to be removed? How can we define success for an A/B experiment? How can we statistically test the cummulative effect of multiple A/B experiments?
Tooling for A/B testing at the release pipeline: How can we build an effiecient A/B testing pipeline for traditional applications? What are the implications of using A/B testing feature toggles on the code?
Contacts about the project:
- Georgios Gousios (TU Delft)
- Arie van Deursen (TU Delft)
- Hennie Huijgens (ING)
- Kevic, K., Murphy, B., Williams, L., & Beckmann, J. (2017, May). Characterizing experimentation in continuous deployment: a case study on bing. In Proceedings of the 39th International Conference on Software Engineering: Software Engineering in Practice Track (pp. 123-132).