Research Track "Trustworthy AI"

PhD Candidate: Patrick Altmeyer
Track leader: Cynthia Liem

More and more software services in the banking domain rely on machine learning. This makes it crucial that the outcomes of the machine learning procedures in these services can be trusted. To be able to assess this, the 7 key requirements of the EU’s Ethics Guidelines for Trustworthy Artificial Intelligence (Human agency and oversight; Technical Robustness and safety; Privacy and data governance; Transparency; Diversity, non-discrimination and fairness; Societal and environmental well-being; Accountability) will need concrete operationalization for the Fintech domain.

In several previous applications of machine learning (and statistical modeling), it has been found that concepts of ‘trust in’ and ‘correctness of’ models are not always clear-cut. Models that seem to perform well according to common performance metrics, may show unexpected behavior in the wild. Seemingly minor researcher degrees of freedom may have major outcomes on final results, and model outcomes may be misinterpreted, even by data scientists.

In this track, we approach trustworthy and explainable artificial intelligence from the perspective of counterfactuals: small perturbations to input values that lead to different model outcomes. Such counterfactuals can provide recourse, offering people means to strategically alter their behavior so that models do give desired outcomes.

Selected Publications

Patrick Altmeyer, Mojtaba Farmanbar, Arie van Deursen, Cynthia CS Liem. Faithful Model Explanations through Energy-Constrained Conformal Counterfactuals. Proceedings of the AAAI Conference on Artificial Intelligence, 2024. Preprint.
Floris Hengst, Ralf Wolter, Patrick Altmeyer, Arda Kaygan. Conformal Intent Classification and Clarification for Fast and Accurate Intent Recognition. Findings of the Association for Computational Linguistics: NAACL 2024. Publication
Patrick Altmeyer, Andrew M. Demetriou, Antony Bartlett, Cynthia C. S. Liem. Position: Stop Making Unscientific AGI Performance Claims. Proceedings of the 41st International Conference on Machine Learning. Preprint
Patrick Altmeyer, Arie van Deursen, Cynthia C. S. Liem. Explaining Black-Box Models through Counterfactuals. JuliaCon Proceedings, 1(1), 130, 2023 (preprint).
Endogenous Macrodynamics in Algorithmic Recourse. P Altmeyer, G Angela, A Buszydlik, K Dobiczek, A van Deursen, Cynthia S. Liem. IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), 418-431, 2023 (preprint)
Cynthia C. S. Liem and Chris Mostert, “Can’t Trust the Feeling? How Open Data Reveals Unexpected Behavior of High-level Music Descriptors,” in ISMIR 2020.

Other Activities

Award. Patrick Altmeyer (2023). 2nd prize in Pluto Notebook Competition, JuliaCon2023.
Award. Patrick Altmeyer (2023). 1st Prize Winner at ING Experiment Week 2023.
Toolset. Trustworthy Artificial Intelligence in Julia (Taija). This track has resulted into a number of packages in Julia, available at https://www.taija.org.
Tool. Patrick Altmeyer et al. (2023–). ConformalPrediction. https://github.com/JuliaTrustworthyAI/ConformalPrediction.jl (Note: over 100 stars on Github)
Tool. Patrick Altmeyer et al. (2023–). CounterfactualExplanations. https://github.com/JuliaTrustworthyAI/CounterfactualExplanations.jl (Note: 100+ stars and 10+ contributors on Github)
Talk. Patrick Altmeyer (2024). Multiple talks at JuliaCon 2024. https://pretalx.com/juliacon2024/speaker/8DGYCX/