{"id":4778,"title":"The Urgent Need for Robust Trust – Cultivating an environment in which algorithmic decision-making serves society","link":"https:\/\/www.reframetech.de\/en\/2020\/06\/11\/the-urgent-need-for-robust-trust\/","date":"06\/11\/2020","date_unix":1591857902,"date_modified_unix":1723455310,"date_iso":"2020-06-11T06:45:02+00:00","content":"

On February 19, 2020, the European Commission published its \u201c<\/span><\/i>White Paper on Artificial Intelligence \u2013 A European approach to excellence and trust<\/i><\/a><\/strong>\u201d<\/strong> and launched a consultation.\u00a0<\/span><\/i>We welcome the European Commission\u2019s effort to harmonize AI regulation and create an ecosystem in which algorithmic decision-making systems work for people and become a force for good in society.<\/span><\/i>\u00a0In this blogpost<\/span><\/i>\u00a0we want to share our\u00a0<\/span><\/i>contribution to the consultation, which is based on our\u00a0<\/span><\/i>findings from the\u00a0<\/span><\/i>\u201cEthics of Algorithms project\u201d<\/strong><\/span><\/i><\/a>.<\/span><\/i>\u00a0<\/span><\/p>\n


\nAlgorithmic decision-making (ADM) systems are having a profound impact on society. The use of<\/span> ADM systems has the capacity to unlock enormous societal, political, economic and cultural\u00a0<\/span>potential<\/span>. However, if not used in the right way,\u00a0<\/span>such systems<\/span>\u00a0<\/span>could<\/span>\u00a0also exacerbate existing\u00a0<\/span>inequ<\/span>i<\/span>ties<\/span>\u00a0or trigger unexpected new ethical issues with large-scale impact. In recent years, the Bertelsmann Stiftung\u2019s\u00a0<\/span>\u201c<\/span>Ethics of Algorithms<\/span>\u201d project<\/span>\u00a0has\u00a0<\/span>contributed to\u00a0<\/span>the\u00a0<\/span>debate<\/span>\u00a0on the increased use of\u00a0<\/span>so<\/span>-called\u00a0<\/span>AI and other ADM processes<\/span>.\u00a0<\/span>The project conducts<\/span>\u00a0research on these issues and\u00a0<\/span>promot<\/span>es<\/span>\u00a0dialogue on the societal impact of ADM technology and regulatory needs<\/span>;<\/span>\u00a0<\/span>in so doing, it seeks<\/span>\u00a0<\/span>levers able to shape\u00a0<\/span>the<\/span>\u00a0sociopolitical<\/span>\u00a0and economic environment\u00a0<\/span>in such a way as to optimize<\/span>\u00a0algorithmic decision-<\/span>making<\/span>\u2019s potential\u00a0<\/span>for social good<\/span>,<\/span>\u00a0while mitigating its risks.<\/span>\u00a0<\/span><\/p>\n

We believe that this issue\u00a0<\/span>cannot<\/span>\u00a0be addressed\u00a0<\/span>exclusively\u00a0<\/span>on a national level<\/span>.<\/span>\u00a0<\/span>We<\/span>\u00a0therefore welcome the European Commission\u2019s efforts to establish a single regulatory approach that guarantees a level playing field for all vendors, regardless of\u00a0<\/span>their<\/span>\u00a0country of origin<\/span>\u00a0or<\/span>\u00a0the member state\u00a0<\/span>in which the<\/span>\u00a0system<\/span>\u00a0is operating, and\u00a0<\/span>of whether\u00a0<\/span>the<\/span>\u00a0system is\u00a0<\/span>being\u00a0<\/span>operated from within or outside the EU<\/span>,<\/span>\u00a0<\/span>or<\/span>\u00a0involves decisions made regarding EU citizens.\u00a0<\/span>It seems\u00a0<\/span>essential for EU citizens and\u00a0<\/span>for<\/span>\u00a0the Digital Single Market policy\u00a0<\/span>to harmonize<\/span>\u00a0<\/span>data-<\/span>subject<\/span>s\u2019<\/span>\u00a0rights<\/span>\u00a0<\/span>at the\u00a0<\/span>European\u00a0<\/span>level. This includes aspects<\/span>\u00a0such as the right to human intervention, the right to receive an explanation of a decision<\/span>,<\/span>\u00a0and the right to challenge or contest a decision<\/span>. Our recommendations<\/span>\u00a01-3\u00a0<\/span>below\u00a0<\/span>comment on the nature and scope of this\u00a0<\/span>endeavor<\/span>.<\/span>\u00a0<\/span><\/p>\n

Discussions\u00a0<\/span>addressing<\/span>\u00a0the regulation of ADM systems\u00a0<\/span>often suggest that we are<\/span>\u00a0starting with a clean slate.\u00a0<\/span>In reality<\/span>, existing<\/span>\u00a0<\/span>legislation<\/span>\u00a0needs to be interpreted in a new light,<\/span>\u00a0and<\/span>\u00a0underlying principles have to be rearticulated and applied to new contexts. Even though this may take significant effort and expertise, it is\u00a0<\/span>necessary<\/span>\u00a0in order to\u00a0<\/span>avoid<\/span>\u00a0undercut<\/span>ting<\/span>\u00a0and\u00a0<\/span>delegitimiz<\/span>ing<\/span>\u00a0existing regulations<\/span>,<\/span>\u00a0and to properly focus the current discussions. Recommendations 4-7 therefore\u00a0<\/span>analyze<\/span>\u00a0relevant existing\u00a0<\/span>regulation<\/span>s while highlighting<\/span>\u00a0their deficiencies and necessary revisions.\u00a0<\/span>\u00a0<\/span><\/p>\n

The measures taken to ensure that ADM serves society must go beyond the establishment of new legal requirements and the revision of existing laws.\u00a0<\/span>We<\/span>\u00a0need\u00a0<\/span>a number of<\/span>\u00a0interlinked policy approaches that focus on the organizations involved in the development and deployment of AI<\/span>,<\/span>\u00a0<\/span>as well as on<\/span>\u00a0the mechanisms through which they interact. The EU should also help strengthen, promote and financially support approaches to ADM that demonstrate an exemplary best<\/span>–<\/span>practice implementation of the technology. Recommendations 8-12 thus present necessary policy measures that go beyond a revision of the legal framework.\u00a0<\/span>\u00a0<\/span><\/p>\n

Drawing on the findings of our\u00a0<\/span>research<\/span>, we suggest the following steps for\u00a0<\/span>the creation of<\/span>\u00a0a European approach to AI,\u00a0<\/span>under which<\/span>\u00a0technology\u00a0<\/span>would work<\/span>\u00a0for people and\u00a0<\/span>become<\/span>\u00a0a force for good in society:<\/span>\u00a0<\/span><\/p>\n


\nNature and scope of the European approach to AI\u00a0<\/span><\/b>\u00a0<\/span><\/p>\n

(1) Include rule-based algorithmic decision-making systems<\/span><\/b>\u00a0in the approach<\/span><\/b>:\u00a0<\/span><\/b>In addition to its focus on machine-learning technologies,\u00a0<\/span>t<\/span>he<\/span>\u00a0European\u00a0<\/span>a<\/span>pproach<\/span>\u00a0to AI must also address the use of rule-based decision-making systems that do not rely on machine-learning methods. One of the Bertelsmann Stiftung\u2019s key findings is that the impact of ADM systems depends on more than the actual technology itself (i.e<\/span>.,<\/span>\u00a0deep learning, machine learningMachine Learning<\/span> Ein Teilbereich der KI, bei dem Systeme aus Daten lernen und sich verbessern, ohne explizit programmiert<\/strong> zu werden.<\/span><\/span><\/span> or statistical analysis).\u00a0<\/span>\u00a0<\/span>Even technically simple ADM systems can have a strong impact on people\u2019s lives<\/span><\/b>, while technically sophisticated ADM may be used without having any impact on consumers and citizens (e.g<\/span>.,<\/span>\u00a0when used\u00a0<\/span>only\u00a0<\/span>for quality assurance\u00a0<\/span>purposes on<\/span>\u00a0production lines). Nevertheless, the specific characteristics of\u00a0<\/span>machine-<\/span>learning systems (AI) should also be translated into additional requirements (i.e<\/span>.,\u00a0<\/span>an<\/span>\u00a0obligation\u00a0<\/span>that<\/span>\u00a0changes made in the code<\/span>\u00a0be documented<\/span>,\u00a0<\/span>or<\/span>\u00a0specific requirements\u00a0<\/span>regarding the<\/span>\u00a0<\/span>explainability<\/span>\u00a0of complex machine<\/span>–<\/span>learning systems).\u00a0<\/span>\u00a0<\/span><\/p>\n


\n(2) Focus on societal impact: <\/span><\/b>A risk-based regulatory approach is the right way to go. Legal requirements should be based on the societal impact of an AI system\u00a0<\/span>within<\/span>\u00a0its\u00a0<\/span>specific<\/span>\u00a0application field. For example, ADM systems used\u00a0<\/span>on<\/span>\u00a0automated production\u00a0<\/span>l<\/span>i<\/span>nes<\/span>\u00a0or\u00a0<\/span>within\u00a0<\/span>similar environments\u00a0<\/span>may<\/span>\u00a0not require the same scrutiny as ADMs used in the public sector or by credit<\/span>–<\/span>inquiry agencies. Article 9\u00a0<\/span>of the\u00a0<\/span>GDPR<\/span>,<\/span>\u00a0<\/span>which governs<\/span>\u00a0the processing of special categories of personal data, including particularly sensitive data, shows that relevant regulatory models are already in place that could serve as an example<\/span>\u00a0in this regard<\/span>.<\/span>\u00a0However,\u00a0<\/span>simply<\/span>\u00a0identif<\/span>ying<\/span>\u00a0specific high-risk sectors poses the\u00a0<\/span>danger<\/span>\u00a0of\u00a0<\/span>failing to recognize<\/span>\u00a0all application cases with a substantial societal impact. Instead,\u00a0<\/span>we recommend the use of a two<\/span><\/b>–<\/span><\/b>dimensional risk<\/span><\/b>\u00a0<\/span><\/b>matrix for the classification of application cases<\/span><\/b>\u00a0into\u00a0<\/span>four<\/span>\u00a0to\u00a0<\/span>five<\/span>\u00a0different classes, as proposed by\u00a0<\/span>Krafft<\/span>\u00a0and Zweig in\u00a0<\/span>chapter three of\u00a0<\/span>our working paper \u201c<\/span>From principles to practice<\/a><\/strong>\u201d<\/span>.\u00a0<\/span>The first dimension of the risk<\/span>\u00a0<\/span>matrix should\u00a0<\/span>express<\/span>\u00a0the intensity of potential harm for individuals and\u00a0<\/span>society as a whole<\/span>\u00a0(i.e<\/span>.,\u00a0<\/span>the<\/span>\u00a0potential negative impact on fundamental rights, equality or social justice<\/span>;<\/span>\u00a0threats\u00a0<\/span>to<\/span>\u00a0democratic institutions; the number of people affected). The second dimension should\u00a0<\/span>reflect<\/span>\u00a0the\u00a0<\/span>degree to which<\/span>\u00a0potentially affected parties\u00a0<\/span>are dependent\u00a0<\/span>on the AI system (i.e<\/span>.,\u00a0<\/span>i<\/span>s<\/span>\u00a0there a human in the loop<\/span>\u00a0who could overrule the AI system\u2019s decision<\/span>?<\/span>\u00a0Is there\u00a0<\/span>a<\/span>\u00a0possibility\u00a0<\/span>of switching out<\/span>\u00a0the AI system for another?). For cases that show a\u00a0<\/span>strong\u00a0<\/span>potential\u00a0<\/span>for<\/span>\u00a0<\/span>total damage<\/span>,<\/span>\u00a0and therefore fall\u00a0<\/span>in<\/span>to<\/span>\u00a0the highest risk<\/span>\u00a0<\/span>class, regulators\u00a0<\/span>should<\/span>\u00a0<\/span>altogether<\/span>\u00a0prohibit the use of\u00a0<\/span>an<\/span>y<\/span>\u00a0ADM component, whereas cases in the lowest risk<\/span>\u00a0<\/span>class\u00a0<\/span>would<\/span>\u00a0not need any additional ADM-specific regulation.<\/span>\u00a0For cases in high risk classes, we would furthermore welcome the obligation to conduct technology impact assessments.\u00a0<\/span>\u00a0<\/span><\/p>\n


\n(3) Consider how ADM is organizationally, socially and politically embedded: <\/span><\/b>ADM systems are\u00a0<\/span>sociotechnological<\/span>\u00a0frameworks. Their impact on society is influenced not only by the algorithm itself, but also by\u00a0<\/span>the<\/span>\u00a0ADM\u2019s underlying goals<\/span>\u00a0and<\/span>\u00a0decision-making models, the\u00a0<\/span>data\u00a0<\/span>used as\u00a0<\/span>input, the ways in which\u00a0<\/span>the\u00a0<\/span>algorithmic output is used<\/span>,<\/span>\u00a0and the entire organizational<\/span>\u00a0and<\/span>\u00a0political environment surrounding its use. The European approach to AI should be developed from a holistic point of view that considers\u00a0<\/span>all of<\/span>\u00a0<\/span>these factors together. Any regulation focusing solely on the technological aspects of an ADM should be accompanied by other measures that\u00a0<\/span>facilitate<\/span>\u00a0oversight over the entire\u00a0<\/span>sociotechnological<\/span>\u00a0system (see\u00a0<\/span>r<\/span>ecommendation<\/span>\u00a08).\u00a0<\/span>\u00a0<\/span><\/p>\n


\nCurrent regulatory deficiencies and necessary revisions\u00a0<\/span><\/b>\u00a0<\/span>\u00a0<\/span><\/p>\n

(4) Review the scope of GDPR to close legal loopholes:<\/span><\/b>\u00a0<\/span>Given that<\/span>\u00a0algorithmic systems rely<\/span>\u00a0<\/span>strongly<\/span>\u00a0on the processing of personal data, we acknowledge that the GDPR contains a comprehensive framework for the regulation of ADM,\u00a0<\/span>for example<\/span>\u00a0by stipulating certain fairness, transparency and accountability requirements, including the need to conduct a data<\/span>–<\/span>protection impact assessment for high-risk applications. Nonetheless, the current regulation\u00a0<\/span>contained\u00a0<\/span>in GDPR article 22\u00a0<\/span>defines<\/span>\u00a0ADM systems\u00a0<\/span>narrowly,\u00a0<\/span>and<\/span>\u00a0lacks necessary safeguards that would enable\u00a0<\/span>individuals affected by<\/span>\u00a0ADM to exercise their rights. A legal study by Professor Mario Martini, commissioned by the Bertelsmann Stiftung and published in January 2020 (<\/span>available in German<\/a><\/strong>),\u00a0<\/span>examined<\/span>\u00a0five prominent examples of the use of ADM (e.g<\/span>.,<\/span>\u00a0allocation of study places at universities and predictive policing). The analysis\u00a0<\/span>showed<\/span>\u00a0that in many cases, the GDPR does set relevant boundaries for the use of ADM systems in the EU. However, substantial legal uncertainty stems from the legal loophole surrounding partial automation<\/span>\u00a0\u2013<\/span>\u00a0<\/span>that is<\/span>,<\/span>\u00a0when automated systems are being used to prepare and support human decision-making. The upcoming GDPR review should address this regulatory failure. The legal paper\u00a0<\/span>\u201c<\/span>What Are the Benefits of the General Data Protection Regulation for Automated Decision-Making Systems<\/a><\/strong>\u201d\u202f<\/span>by the Bertelsmann Stiftung gives practical suggestions for possible complementary approaches within and beyond the GDPR.<\/span>\u00a0<\/span><\/p>\n


\n(5) <\/strong><\/span>Require comprehensive, context-tailored transparency mechanisms\u00a0<\/span><\/b>towards parties<\/span><\/b>\u00a0affected<\/span><\/b>:<\/span><\/b>\u00a0If<\/span>\u00a0an ADM is deployed in an<\/span>\u00a0application\u00a0<\/span>case<\/span>\u00a0with\u00a0<\/span>a\u00a0<\/span>high\u00a0<\/span>level of\u00a0<\/span>societal impact, the provision of information on\u00a0<\/span>that system\u2019s\u00a0<\/span>functioning<\/span>\u00a0and<\/span>\u00a0underlying goals<\/span>, as well as on<\/span>\u00a0the results of previously conducted assessments\u00a0<\/span>of the system\u2019s technology,\u00a0<\/span>must be required<\/span>. Moreover, this information must be furnished<\/span>\u00a0in a context-tailored and proactive manner. Institutions deploying ADM systems must ensure that the technology is\u00a0<\/span>labeled<\/span>\u00a0as such and that affected parties receive\u00a0<\/span>all<\/span>\u00a0information<\/span>\u00a0necessary<\/span>\u00a0to exercise their legal rights. The subjects of ADM mechanisms must therefore also receive a comprehensible explanation of how\u00a0<\/span>each relevant<\/span>\u00a0decision was\u00a0<\/span>reached<\/span>,<\/span>\u00a0and<\/span>\u00a0must be given the\u00a0<\/span>possibility\u00a0<\/span><\/b>of requesting a<\/span><\/b>\u00a0legal examination of the decision made by the system<\/span><\/b>. Humans in the loop must be trained to function as\u00a0<\/span>mediators who<\/span>\u00a0can provide the necessary\u00a0<\/span>explanations to<\/span>\u00a0affected<\/span>\u00a0parties<\/span>.\u00a0<\/span>\u00a0<\/span><\/p>\n


\n<\/span>(6) Clarify liabilities to reduce uncertainties: <\/span><\/b>To make ADM a safe option for European companies and organizations<\/span>,<\/span>\u00a0while\u00a0<\/span>also\u00a0<\/span>ensuring accountability, clear liability rules are needed. In contrast to\u00a0<\/span>the\u00a0<\/span>product<\/span>\u00a0market<\/span>,\u00a0<\/span>there is currently no<\/span>\u00a0legal framework\u00a0<\/span>regulating<\/span>\u00a0safety and liability\u00a0<\/span>issues for<\/span>\u00a0<\/span>services<\/span>.\u00a0<\/span>Today<\/span>, any institution using automated decision-making\u00a0<\/span>technology\u00a0<\/span>is responsible for its use and abuse<\/span>.<\/span>\u00a0To reduce uncertainties and promote the uptake of ADM systems, the future legal framework must clarify which burdens are carried\u00a0<\/span>specifically\u00a0<\/span>by ADM producers and vendors<\/span>,<\/span>\u00a0<\/span>and which burdens are\u00a0<\/span>by contrast\u00a0<\/span>shared by ADM producers\/vendors and organizational users\u00a0<\/span>that implement<\/span>\u00a0the technology\u00a0<\/span>with<\/span>in<\/span>\u00a0a particular setting (additionally, see recommendation 7 on reversing the burden of proof for affected<\/span>\u00a0parties<\/span>).\u00a0<\/span>\u00a0<\/span><\/p>\n


\n<\/span>(7) Consider and revisit the non-discrimination framework<\/span><\/b>: Given the centrality of the principle of non-discrimination to the values of the\u00a0<\/span>EU<\/span>,<\/span>\u00a0and<\/span>\u00a0considering\u00a0<\/span>various technologies\u2019<\/span>\u00a0potential to induce discrimination at a large scale, any regulation of ADM systems should\u00a0<\/span>take<\/span>\u00a0existing non-discrimination law<\/span>\u00a0into account<\/span>. To be able to address new forms of data-driven discrimination, the fragmented framework for equal treatment needs to be revisited at European level. With\u00a0<\/span>regard<\/span>\u00a0to\u00a0<\/span>the\u00a0<\/span>provision\u00a0<\/span>of\u00a0<\/span>information about the functioning and purpose of ADM systems, non-discrimination law can serve as an example of how the burden of proof can be reversed from the claimant to the responding party (in the case of ADM, shifting the burden of proof from the subject of the decision to the vendor<\/span>).<\/span>\u00a0<\/span>\u00a0<\/span><\/p>\n


\nNecessary policy measures beyond the legal framework\u00a0<\/span><\/b>\u00a0<\/span><\/p>\n

(8) Ensure enforcement of<\/span><\/b>\u00a0the<\/span><\/b>\u00a0legal framework by strengthening oversight mechanisms:<\/span><\/b>\u00a0Algorithmic decision-making does not require\u00a0<\/span>the establishment of<\/span>\u00a0new fundamental rights. However, in order to ensure that existing principles, freedoms and rights\u00a0<\/span>and any<\/span>\u00a0new laws remain enforceable when ADM systems are used, we need to strengthen existing oversight bodies and civil society watchdog organizations through financial means and\u00a0<\/span>through expanded\u00a0<\/span>competence<\/span>–<\/span>building<\/span>\u00a0measures<\/span>. The Bertelsmann Stiftung recommends\u00a0<\/span>the establishment of<\/span>\u00a0<\/span>institutionalized fora\u00a0<\/span>that include<\/span>\u00a0all\u00a0<\/span>relevant<\/span>\u00a0<\/span>oversight bodies<\/span>; these fora should be<\/span>\u00a0<\/span>tasked with<\/span>\u00a0address<\/span>ing<\/span>\u00a0ADM-related issues and\u00a0<\/span>identifying any<\/span>\u00a0legal uncertainties\u00a0<\/span>or<\/span>\u00a0need for further action. Furthermore,\u00a0<\/span>support should be given to research<\/span>\u00a0projects\u00a0<\/span>focusing\u00a0<\/span>on the development of auditing processes<\/span>,<\/span>\u00a0<\/span>as well as to processes of<\/span>\u00a0knowledge exchange between scientific and private sector stakeholders<\/span>.\u00a0<\/span>While<\/span>\u00a0we acknowledge the importance of\u00a0<\/span>algorithm<\/span>\u00a0<\/span>auditing<\/span>,\u00a0<\/span>negative impact\u00a0<\/span>from the use of ADM systems can<\/span>\u00a0also arise due to the\u00a0<\/span>way these<\/span>\u00a0systems\u00a0<\/span>are\u00a0<\/span>social<\/span>ly<\/span>\u00a0embedd<\/span>ed,<\/span>\u00a0<\/span>or due to<\/span>\u00a0the interaction between multiple ADMs<\/span>. These are issues<\/span>\u00a0that cannot be captured\u00a0<\/span>in<\/span>\u00a0<\/span>an\u00a0<\/span>input<\/span>-output analysis of individual algorithms. Oversight bodies should thus establish mechanisms that allow\u00a0<\/span>affected<\/span>\u00a0communities to give\u00a0<\/span>an\u00a0<\/span>account of their situation<\/span>,<\/span>\u00a0<\/span>with a goal of<\/span>\u00a0assess<\/span>ing<\/span>\u00a0systems<\/span>\u2019<\/span>\u00a0<\/span>overall impact rather than\u00a0<\/span>focusing solely on the functioning of\u00a0<\/span>the\u00a0<\/span>technology<\/span>.<\/span>
\n<\/span>Concerning the regulation of digital services in the Single Market, our expert paper \u201c<\/span>Governance of Digitalization in Europe<\/a><\/strong>\u201d suggests the implementation of a networked and decentralized governance model\u00a0<\/span>that<\/span>\u00a0would allow a set of independent sector-specific regulatory authorities (at\u00a0<\/span>the\u00a0<\/span>European, regional, national and subnational\u00a0<\/span>level<\/span>s<\/span>)<\/span>\u00a0to be convened<\/span>\u00a0and coordinat<\/span>ed<\/span>.<\/span>\u00a0In contrast, discussions around the Digital Services Act have raised the prospect of creating a centralized governance model around a new central regulatory authority.\u00a0<\/span>Such<\/span>\u00a0governance issues also need to be considered<\/span>\u00a0<\/span>when discussing how<\/span>\u00a0<\/span>to\u00a0<\/span>strengthen and coordinat<\/span>e<\/span>\u00a0ADM oversight bodies<\/span>.<\/span>\u00a0<\/span><\/p>\n


\n(9) <\/strong>Use labeling to create<\/span><\/b>\u00a0<\/span><\/b>incentives for ethical\u00a0<\/span><\/b>technology\u00a0<\/span><\/b>development:<\/span><\/b>\u00a0The Bertelsmann Stiftung recommends the introduction of an AI-Ethics Label.\u00a0<\/span>Labeling<\/span>\u00a0can offer orientation to developers trying to create ethically sound AI systems,\u00a0<\/span>while also<\/span>\u00a0increas<\/span>ing the<\/span>\u00a0transparency and comparability of products for\u00a0<\/span>users, and<\/span>\u00a0<\/span>provid<\/span>ing<\/span>\u00a0a basis for better enforcement of legal standards\u00a0<\/span>by<\/span>\u00a0oversight and watchdog\u00a0<\/span>organizations.<\/span>\u00a0Different application cases\u00a0<\/span>have<\/span>\u00a0different requirements\u00a0<\/span>for issues\u00a0<\/span>such as transparency and robustness<\/span>, and thus<\/span>\u00a0<\/span>demand<\/span>\u00a0different measures.<\/span>\u00a0We thus recommend a nuanced<\/span>\u00a0<\/span>labeling<\/span>\u00a0approach that\u00a0<\/span>can do<\/span>\u00a0justice to the diversity of application cases<\/span>,\u00a0<\/span>modeled in part after<\/span>\u00a0the energy<\/span>–<\/span>efficiency label<\/span>. More information on the design of such a label can be found in our working paper \u201c<\/span>From principles to practice<\/a><\/strong>.\u201d<\/span>\u00a0<\/span>
\n
\n<\/span>
\n(10) Revise public procurement standards<\/span><\/b>: In order to prevent corporate secrecy from getting in the way of public sector accountability on the use of ADM, EU public procurement rules should be complemented by transparency requirements for these systems. Vendors and developers\u00a0<\/span>that<\/span>\u00a0create ADM systems for use in government should have a duty of care to assist users of ADM systems in\u00a0<\/span>ensuring<\/span>\u00a0transparency, accountability and effective auditing<\/span>.<\/span>\u00a0<\/span>They should\u00a0<\/span>thereby agree to waive trade<\/span>–<\/span>secrecy or other legal claims that\u00a0<\/span>might otherwise\u00a0<\/span>inhibit a proper and full audit of their software.\u00a0<\/span>P<\/span>ublic<\/span>\u00a0procurement standards should\u00a0<\/span>also\u00a0<\/span>be examined in order to determine how they can contribute to\u00a0<\/span>increasing the<\/span>\u00a0divers<\/span>ity of the<\/span>\u00a0landscape of ADM providers and systems. Additionally, we recommend the establishment of a public register<\/span>\u00a0<\/span>that<\/span>\u00a0lists all\u00a0<\/span>deployed\u00a0<\/span>public sector ADM systems\u00a0<\/span>that may have a<\/span>\u00a0significant impact on society,\u00a0<\/span>and additionally contains<\/span>\u00a0information about their providers, their underlying goals and the results of technology impact assessments.\u00a0<\/span>\u00a0<\/span><\/p>\n


\n<\/span>(11) Boost competence<\/span><\/b>–<\/span><\/b>building:\u00a0<\/span><\/b>If ADM is to be used to promote the common good, greater technical competence is needed among the general population,\u00a0<\/span>among company<\/span>\u00a0<\/span>executives<\/span>,<\/span>\u00a0and\u00a0<\/span>particularly<\/span>\u00a0among policymakers and those working in civil service positions. Without a basic understanding of the functioning of AI systems and their limitations,\u00a0<\/span>there is a serious<\/span>\u00a0risk that the\u00a0<\/span>public sector<\/span>\u2019s<\/span>\u00a0<\/span>adoption of AI might\u00a0<\/span>actually cause<\/span>\u00a0more harm than good. The introduction of AI in public sector organizations should thus be accompanied by competence<\/span>–<\/span>building measures for\u00a0<\/span>any<\/span>\u00a0people who will\u00a0<\/span>be making decisions regarding the<\/span>\u00a0<\/span>system\u2019s implementation or\u00a0<\/span>interact<\/span>ing<\/span>\u00a0with the system. We furthermore need systematic initiatives to strengthen broad<\/span>-based<\/span>\u00a0algorithmic literacy<\/span>;<\/span>\u00a0<\/span>that is, i<\/span>rrespective<\/span>\u00a0of educational attainment level and occupation, every citizen should be aware of the relevance of algorithmic systems\u00a0<\/span>in<\/span>\u00a0their\u00a0<\/span>personal or professional<\/span>\u00a0<\/span>lives, be in a position to\u00a0<\/span>deal with such systems<\/span>\u00a0as carefully as necessary<\/span>,<\/span>\u00a0and be informed about mechanisms through which automated decisions can be challenged<\/span>.\u00a0<\/span>The<\/span>\u00a0freely accessible Finnish online course\u00a0<\/span>\u201c<\/span>Elements of\u00a0<\/span>AI\u201d<\/span>\u00a0(available in English<\/span>)<\/span>\u00a0is an example of an effective measure in this sense that could be scaled EU-wide<\/span>.<\/span>\u00a0However, competence-building among citizens cannot replace effective oversight\u00a0<\/span>mechanisms<\/span>,<\/span>\u00a0and<\/span>\u00a0should not be used to shift responsibilities from system developers\u00a0<\/span>to<\/span>\u00a0affected parties.\u00a0<\/span>\u00a0<\/span><\/p>\n


\n<\/span>(12) Promote diversity and innovation for social good: <\/span><\/b>The EU and its member states should promote measures\u00a0<\/span>ensuring<\/span>\u00a0<\/span>that\u00a0<\/span>society as a whole \u2013 and\u00a0<\/span>all\u00a0<\/span>of<\/span>\u00a0its individual\u00a0<\/span>parts\u00a0<\/span>\u2013<\/span>\u00a0<\/span>benefits<\/span>\u00a0from the use of AI.\u00a0<\/span>With data and\u00a0<\/span>top programming talent<\/span>\u00a0<\/span>resting<\/span>\u00a0<\/span>largely<\/span>\u00a0in the hands of a few big tech\u00a0<\/span>companies<\/span>, we are witnessing a tendency\u00a0<\/span>toward<\/span>\u00a0monopolization in the field of AI. However, only a diversity of organizations and individuals developing technology can adequately represent social plurality, avoid discrimination and promote innovation.\u00a0<\/span>The EU should therefore support initiatives that promote diversity in the tech sector and in related academic fields, promote knowledge exchange between these initiatives<\/span>,<\/span>\u00a0and ensure that all affected\u00a0<\/span>parties\u00a0<\/span>have<\/span>\u00a0a seat at the table when decisions about the use of AI technology are being made. The sharing of data for social good should be encouraged<\/span>,<\/span>\u00a0and the infrastructure\u00a0<\/span>needed for this task\u00a0<\/span>should be built. Civil society and\u00a0<\/span>nonprofit<\/span>\u00a0stakeholders should be supported financially and through competence<\/span>–<\/span>building in order to foster AI innovation for social good.\u00a0<\/span>\u00a0<\/span><\/p>\n


\nWe would like to thank Falk Steiner and Joris van Hoboken, whose valuable comments and insights<\/span>\u00a0<\/span>have substantially enriched this paper.\u00a0<\/span>\u00a0<\/span><\/p>\n


\nHere you can download a PDF of our input paper \u201c<\/span>The Urgent Need for Robust Trust<\/a><\/strong>\u201d.\u00a0<\/span>\u00a0<\/span><\/p>\n

\n

This text is licensed under a\u202fC<\/strong><\/a>reative Commons Attribution 4.0 International License<\/strong><\/a><\/p>\n","excerpt":"

On February 19, 2020, the European Commission published its \u201cWhite Paper on Artificial Intelligence \u2013 A European approach to excellence […]<\/p>\n","thumbnail":"https:\/\/www.reframetech.de\/wp-content\/uploads\/sites\/23\/2020\/06\/2390666040_5a9d52b896_o.jpg","thumbnailsquare":"https:\/\/www.reframetech.de\/wp-content\/uploads\/sites\/23\/2020\/06\/2390666040_5a9d52b896_o.jpg","authors":[{"id":634,"name":"Carla Hustedt","link":"https:\/\/www.reframetech.de\/blogger\/carla-hustedt\/"},{"id":2646,"name":"Viktoria\u00a0Grzymek\u00a0","link":"https:\/\/www.reframetech.de\/blogger\/viktoria-grzymek-2\/"}],"categories":[{"id":698,"name":"Political decision-makers","link":"https:\/\/www.reframetech.de\/en\/category\/political-decision-makers\/"}],"tags":[{"id":263,"name":"Algorithm","link":"https:\/\/www.reframetech.de\/en\/tag\/algorithm\/"},{"id":315,"name":"Artificial Intelligence","link":"https:\/\/www.reframetech.de\/en\/tag\/artificial-intelligence\/"},{"id":301,"name":"Ethics of Algorithms","link":"https:\/\/www.reframetech.de\/en\/tag\/ethics-of-algorithms\/"},{"id":504,"name":"EU","link":"https:\/\/www.reframetech.de\/en\/tag\/eu-en\/"},{"id":639,"name":"Publications","link":"https:\/\/www.reframetech.de\/en\/tag\/publications\/"},{"id":349,"name":"Regulation","link":"https:\/\/www.reframetech.de\/en\/tag\/regulation\/"}]}