David Hartmann

David Hartmann

PhD researcher, Weizenbaum Institute

Lecturer, Technische Universität Berlin

About

Hello! I am David, a PhD researcher at the Weizenbaum Institute and a lecturer at Technische Universität Berlin.

My research asks who evaluates AI systems, how, and for whom, examining why independent AI auditing remains structurally constrained despite growing regulatory demand. Working across the FAccT, HCI, and NLP communities, I combine technical and empirical methods with critical data studies, with recent work on the measurement infrastructure behind NLP and LLM evaluation.

I have an MSc in Computer Science from TU Berlin, with a major in cognitive systems and ML and a minor in HCI. Additionally, I have a background in philosophy, focusing on science and technology studies.

Research

I am motivated by a desire to clarify the misconceptions surrounding the “magic” of AI, and to this end, I aim to develop reverse-engineering methods for black-box audits. Finally, through my research at Weizenbaum, I want to better understand and support the ecosystem of organisations that analyse and challenge AI systems, particularly their often problematic integration into social and organisational processes.

At its core, through my work, I aim to build infrastructures and methodological frameworks that work towards measuring complex socio-technical systems (including ML systems), asking what can be measured, how, for whom, from which perspective(s), to what end and with what limitations.

I have collaborated with smart, interdisciplinary researchers on projects such as:

Publications

  1. Bye Bye Perspective API: Lessons for Measurement Infrastructure in NLP, CSS and LLM Evaluation
    D. Hartmann, M. Tonneau, A. Kraft, LK Seiling, D. Staufer, P. Delobelle, J. Fillies, A. R. Luther, J. Batzner, M. Lisker
    Findings of EMNLP 2026[arXiv]
  2. Audit Me If You Can: Query-Efficient Active Fairness Auditing of Black-Box LLMs
    D. Hartmann, L. Pohlmann, L. Hanslik, N. Gießing, B. Berendt, P. Delobelle
    Findings of ACL 2026[paper][arXiv]
  3. Lost in Moderation: How Commercial Content Moderation APIs Over- and Under-Moderate Group-Targeted Hate Speech and Linguistic Variations
    D. Hartmann, A. Oueslati, D. Staufer, L. Pohlmann, S. Munzert, H. Heuer
    CHI 2025[paper][arXiv]
  4. A Systematic Review of Echo Chamber Research: Comparative Analysis of Conceptualizations, Operationalizations, and Varying Outcomes
    D. Hartmann, S. M. Wang, L. Pohlmann, B. Berendt
    Journal of Computational Social Science, 2025[paper][arXiv]
  5. Addressing the Regulatory Gap: Moving Towards an EU AI Audit Ecosystem Beyond the AI Act by Including Civil Society
    D. Hartmann, J. R. Laranjeira de Pereira, C. Streitbörger, B. Berendt
    AI and Ethics, 2024[paper][arXiv]
  6. Watching the Watchers: A Comparative Fairness Audit of Cloud-based Content Moderation Services
    D. Hartmann, A. Oueslati, D. Staufer
    European Workshop on Algorithmic Fairness (EWAF), 2024[paper][arXiv]

Full publication list on Google Scholar.

Teaching

Reviewing & community engagement
  • Reviewer: NeurIPS, FAccT, CHI
  • Member: IAAA, EvalEval, FIfF
  • Organiser of workshops at FAccT (rigorous AI evaluation), MuC (content moderation), and TD-Lab-funded civil society audit workshops

News & Talks

Sep '26🛠️ Co-organized the workshop “What Does This Number Even Mean?” on AI benchmarking at the BIFOLD Summer School, with Angelie Kraft.
Aug '26🛠️ Organized the workshop “Human-Centered Content Moderation: Expertise, Context & Evaluation” at Mensch und Computer 2026 in Duisburg, with Anna Ricarda Luther, Hendrik Heuer and Yixin Zou.
Aug '26📜 “Bye Bye Perspective API”, with Manuel Tonneau and colleagues, accepted to Findings of EMNLP 2026.
Jul '26📜 “Audit Me If You Can” presented at ACL 2026 (Findings).
Jun '26🛠️ Co-organized the tutorial “Every Eval Ever” with the EvalEval coalition at FAccT 2026.
Jun '26🎤 Presented “Bye Bye Perspective API” at the Weizenbaum Conference and our project “Evaluating GenAI Evaluations” at the Weizenbaum Research Day.
May '26🗞️ Spoke to Tagesspiegel Background about what the shutdown of Google’s Perspective API means.
Apr '26🎓 Started co-organising “Fairness, Accountability, and Transparency in Machine Learning” (FAccT) at TU Berlin.
Apr '26📜 New preprint on the closure of Perspective API and its lessons for measurement infrastructure in NLP, CSS and LLM evaluation.
Apr '26📜 Presented “From Method to Interface: Human-Centred Auditing Under Resource and Access Constraints” at the HEAL workshop at CHI 2026.
Feb '26🔬 Started co-leading the Weizenbaum project “Evaluating GenAI Evaluations: Epistemics, Stakeholders, Technical Infrastructure and Policy” with Jan Batzner, Rainer Rehak and Anne Krüger.
Jan '26📜 New preprint on query-efficient active fairness auditing of black-box LLMs.
Jan '26💰 Our Civil Society Co-Audits project was awarded TD-Lab funding.
Nov '25📜 Co-authored Weizenbaum Policy Paper #17 on harmonised standards and conformity assessments in the AI Act.
Oct '25🤝 Hosted a participatory co-design workshop with eight civil society organisations on auditing AI systems.
Oct '25🎤 Guest lecture “AI Accountability & the AI Act” at Vilnius University.
Sep '25🛠️ Taught a workshop on AI & Ethics at the Summer School Potsdam, with Laurenz Sachenbacher.
Aug '25📜 New position paper with ECAT colleagues: “Generative AI and the Future of the Digital Commons: Five Open Questions and Knowledge Gaps”.
Jul '25🎤 Talk “Auditing the Stack: Independent Evaluations of the GenAI/LLM Pipeline from Moderation to Consent” at the AlgorithmWatch Tech Research Colloquium.
Jul '25🗞️ Featured in Die Zeit.
Jun '25🎙️ Guest on the CAISZeit podcast about who moderates our timelines.
Jun '25🎤 Invited talk “Audit Me If You Can: Query-Efficient Active Fairness Auditing for Black-Box LMs” at the workshop Responsible Data Science for Safe and Socially Aligned AI Applications, TUM.
Jun '25💬 Moderated the Weizenbaum Forum “Ein Ökosystem für KI-Rechenschaftspflicht?” on AI auditing ecosystems in Germany (video).
May '25💬 Panel “Taking Care of the Internet: Wie Milliarden Posts moderieren? Und wo kann KI (nicht) helfen?” at re:publica 2025, with Hendrik Heuer and Anna Ricarda Luther.
Apr '25🏛️ Became doctoral representative at the Weizenbaum Institute.
Apr '25📜 Presented “Lost in Moderation” at CHI 2025 in Yokohama.
Apr '25📜 “A systematic review of echo chamber research” published in the Journal of Computational Social Science.
Apr '25🎤 First presentation of “Audit Me If You Can” at the Social Science and Language Models workshop, with Lena Pohlmann.
Mar '25🇧🇪 Research stay at the European Centre for Algorithmic Transparency (ECAT) in Brussels.
Jan '25🎙️ Guest on the Internet Governance Explained podcast, talking about content moderation and AI.
Nov '24📜 “Addressing the Regulatory Gap” published in AI and Ethics.
Oct '24🎓 Started teaching “Introduction to Critical Data Studies” at TU Berlin, together with Milagros Miceli.
Jul '24🎤 Guest lecture “Impact Assessments and Accountability” at TU Berlin.
Jul '24🎤 Presented our work on fairness audits at the Center for Advanced Internet Studies (CAIS).
Jul '24🎤 Presented “Watching the Watchers” at the European Workshop on Algorithmic Fairness (EWAF 2024), with Dimitri Staufer.
Jun '24🎓 Presented my PhD project at the FAccT 2024 Doctoral Colloquium.
May '24💬 Organized and moderated the book presentation of “Algorithmic Regimes: Methods, Interactions, and Politics” at the Weizenbaum Institute.
May '24🎤 Presented “Methods from the Past”, on impact assessment case studies to advise the AI Act, at STS Graz, with Lena Pohlmann.
Mar '24🤝 Organized a Beyond AI Collective event at the Weizenbaum Institute.
Jan '24🛠️ Workshop “Artificial Amnesia: AI and Data Work” at UdK Berlin, with Laurenz Sachenbacher.
Jan '24📚 Started the Data Feminism Reading Group at the Weizenbaum Institute.
Dec '23🎤 Guest lecture “Digital Services Act (DSA)” at TU Berlin.
Dec '23🎤 Guest lecture “Echokammern in sozialen Netzwerken” at TU Berlin.
Nov '23🛠️ Workshop “In Math We Trust: KI und Ethik” at the Evangelisches Studienwerk, Essen, with Laurenz Sachenbacher.
Jun '23🎤 Guest lecture “Fairness und Machine Learning” at HTW Berlin.