{"id":13052,"title":"Why AI foundation models can pose risks to the common good","link":"https:\/\/www.reframetech.de\/en\/2025\/09\/25\/why-ai-foundation-models-can-pose-risks-to-the-common-good\/","date":"09\/25\/2025","date_unix":1758780671,"date_modified_unix":1759330632,"date_iso":"2025-09-25T06:11:11+00:00","content":"

Foundation models form the backbone of generative artificial intelligence \u2014 and thus of numerous digital tools such as ChatGPT and Gemini. However, their use carries risks, whether due to randomly compiled training data, profit-driven business models, or limited transparency. A new report by the Bertelsmann Stiftung highlights what mission driven organizations in particular should pay attention to and explores more responsible alternatives.\u00a0<\/em><\/p>\n

Creating a presentation with ChatGPT, translating a text with Gemini, or planning the next event with Copilot: artificial intelligence (AI) is becoming increasingly common in everyday (working) life. It is designed to provide quick answers, give food for thought, or take over routine tasks, freeing up more time for more complex work. These digital assistants rely on foundation models \u2013 large-scale AI systems trained on massive datasets that are adaptable to a wide range of uses. Whether an AI system produces accurate or misleading, balanced or biased results depends directly on the training data of its underlying foundation modelBasismodell<\/span> Ein gro\u00dfes, auf umfangreichen Datens\u00e4tzen trainiertes KI-Modell<\/strong>, das als Grundlage f\u00fcr verschiedene spezifische Anwendungen dient. Foundation Models\/Basismodelle k\u00f6nnen f\u00fcr eine Vielzahl von Aufgaben in verschiedenen Anwendungsgebieten feinabgestimmt werden.<\/span><\/span><\/span>.<\/span>\u00a0<\/span><\/p>\n

The Bertelsmann Stiftung has examined the potential disadvantages of these models in its new study, <\/span>Fragile Foundations: The Hidden Risks of Generative AI. <\/span><\/i>Author Anne L. Washington, Associate Professor of Technology Policy at Duke University’s Sanford School of Public Policy, analyzes the systemic weaknesses of current foundation models. Drawing on expert interviews, comparative model analysis, and recent research, the study shows that many problems stem not only from the applications but from the models themselves.<\/span>\u00a0<\/span><\/p>\n

Significant risks for vulnerable groups<\/span><\/b>\u00a0<\/span><\/h2>\n

Although marketed as \u201cgeneral purpose models,\u201d foundation models often prove unsuitable for broad use. For organizations with a social mission \u2013 whether in social work, environmental protection, or support for disadvantaged groups \u2013 the risks can be particularly serious. Precisely in contexts where people in vulnerable situations rely on assistance, faulty or discriminatory AI outputs can cause real harm.<\/span>\u00a0<\/span><\/p>\n

These risks are not hypothetical. A <\/span>chatbot designed for people with eating disorders<\/span><\/a> offered dieting tips, a <\/span>system used by the Austrian employment agency<\/span><\/a> recommended cooking or nursing jobs to women and IT jobs to men, and in California, a <\/span>chatbot encouraged suicidal thoughts<\/span><\/a> in a teenager instead of pointing him to sources of help. Such cases show how the supposed neutrality of AI can prove illusory. Organizations that rely on AI that produces distorted or discriminatory results risk eroding the trust on which their work depends.<\/span>\u00a0<\/span><\/p>\n

Critically questioning the fundamentals of AI<\/span><\/b>\u00a0<\/span><\/h2>\n

\u201cAI foundation models carry the risk of exacerbating existing injustices. Our report invites us to critically question and rethink the fundamentals of generative AI. Mission-driven organizations in particular should make a conscious decision about whether to use a foundation model and, if so, for what purpose,\u201d says Teresa Staiger, digital expert at the Bertelsmann Stiftung.<\/span>\u00a0<\/span><\/p>\n

A key concern is the reliance on uncurated training data \u2013 datasets gathered largely through automated web scraping. Such data often reflect historical biasBias<\/span> In der KI bezieht sich Bias auf Verzerrungen<\/strong> in Modellen oder Datens\u00e4tzen. Es gibt zwei Arten:\r\rEthischer Bias<\/em>:<\/strong> systematische Voreingenommenheit, die zu unfairen oder diskriminierenden Ergebnissen f\u00fchrt, basierend auf Faktoren wie Geschlecht, Ethnie oder Alter.\r\rMathematischer Bias<\/em>:<\/strong> eine technische Abweichung in statistischen Modellen, die zu Ungenauigkeiten f\u00fchren kann, aber nicht notwendigerweise ethische Probleme verursacht.<\/span><\/span><\/span> and overlook the perspectives of certain social groups. The business models of AI providers also play a role: When profit maximization overrides data quality, external evaluations of the data becomes impossible and accountability requirements are absent. In addition, systemic conditions further compound the problem: without consistent tools for systematic evaluation and review, errors in in foundation models ripple into downstream applications.<\/span>\u00a0<\/span><\/p>\n

Other paths are possible<\/span><\/b>\u00a0<\/span><\/h2>\n

In addition to identifying structural weaknesses, the study outlines potential alternatives. These include technical, participatory, data-related, and collaborative. Key recommendations include:<\/span>\u00a0<\/span><\/p>\n