{"id":2030,"title":"Quality Criteria for Algorithms – Lessons from Existing Compendia","link":"https:\/\/www.reframetech.de\/en\/2018\/07\/17\/quality-criteria-for-algorithms-lessons-from-existing-compendia\/","date":"07\/17\/2018","date_unix":1531813370,"date_modified_unix":1649928256,"date_iso":"2018-07-17T07:42:50+00:00","content":"

What operations should algorithms be allowed to perform? What standards of quality should they be held to? For what purposes may they be used? Although they are deeply relevant for society, we have yet to reach a social consensus on these issues. A number of international organizations seek to answer these questions by formulating quality criteria that ensure the use and development of algorithmic processes are held to high ethical standards. In our working paper \u201cQuality Criteria for Algorithmic Processes\u201d<\/a><\/strong>, we take a closer look at three existing proposals and analyze their strengths and weaknesses.<\/em><\/p>\n

The effects of algorithms on our individual and social realities can no longer be denied: they not only collect, sort, weigh and examine our data \u2013 they are increasingly relied upon in decision-making processes that can have far-reaching implications for those affected by them. Processes with such powerful impact should be subject to soft regulation, and the concept of quality standards for socially relevant algorithms offers a promising approach.<\/p>\n

Technical innovations do not emerge from a vacuum. The creation of algorithmic systems involves the participation of a large variety of stakeholders: clients, programmers and implementers \u2013 to name but a few. It is this heterogeneity of the actors involved, and the ways in which their diverse professions overlap that make it difficult for a uniform professional ethics to develop.<\/p>\n

Several international organizations have thus taken to tackling this issue by focusing on quality criteria for that process which represents the common denominator of all participants \u2013 the creation of algorithmic processes. As encouraging as this strategy may be, we are now faced with several different compendia that exist in isolation from each other. New documents are being drawn up without making use of possible synergy effects and tapping into the knowledge offered by existing catalogues.<\/p>\n

Three quality criteria compendia under the microscope<\/strong><\/p>\n

In our working paper, we examine three current quality criteria compendia for algorithmic processes in order to gain a better understanding of their strengths and weaknesses. The proposals are the \u201cPrinciples for Acountable Algorithms and a Social Impact Statement for Algorithms\u201d<\/strong><\/a> by the FAT\/ML collective, the \u201cAsilomar AI Principles\u201d<\/a><\/strong> by the Future of Life Institute, as well as the \u201cPrinciples for Algorithmic Transparency and Accountability\u201d<\/a><\/strong> of the ACM US Public Policy Council.<\/p>\n

Ranging from the usefulness of individual criteria to (hidden) premises and formal as well as linguistic aspects, we take a closer look at the spectrum of each catalogue\u2019s characteristics and compare them with each other. Finally, we offer a summary of transferable strengths and weaknesses to be avoided in future endeavors. With our work, we hope to facilitate the discourse and provide a reference for future work in the field.<\/p>\n

Many sensible criteria \u2013 too little attention to ethical questions and implementation<\/strong><\/p>\n

The key findings of our study are represented in the infographic below (see figure 1). Several vital aspects have already been incorporated into existing compendia. This includes, for example, sensitizing actors in the process to relevant pre-or post-programming steps, as well as acknowledging a large group of addressees that reflects the heterogeneity of responsible actors. However, there is also room for improvement. Criteria concerned directly with ethical questions are rare, as is a recourse to the role of politics in this context. Also, the compendia do not call for bans on certain uses of algorithms that are inappropriate or indicate how the respective quality criteria catalogs might be implemented in practice. This is where new projects (including our own #algorules<\/a>) must begin \u2013 by drawing on the lessons of previous efforts and developing them further.<\/p>\n

\"\"<\/p>\n

Quality-Criteria Catalogs for Algorithms – A Comparison\u00a0 \u00a0<\/em><\/p>\n

\n

This text is licensed under a\u202fC<\/strong><\/a>reative Commons Attribution 4.0 International License<\/strong><\/a><\/p>\n","excerpt":"

What operations should algorithms be allowed to perform? What standards of quality should they be held to? For what purposes […]<\/p>\n","thumbnail":"https:\/\/www.reframetech.de\/wp-content\/uploads\/sites\/23\/2018\/07\/annie-spratt-608001-unsplash_185x247mm-780x373.jpg","thumbnailsquare":"https:\/\/www.reframetech.de\/wp-content\/uploads\/sites\/23\/2018\/07\/annie-spratt-608001-unsplash_185x247mm-370x370.jpg","authors":[{"id":1506,"name":"No\u00eblle Rohde","link":"https:\/\/www.reframetech.de\/en\/blogger\/noelle-rohde\/"}],"categories":[{"id":698,"name":"Political decision-makers","link":"https:\/\/www.reframetech.de\/en\/category\/political-decision-makers\/"}],"tags":[{"id":640,"name":"Algo.Rules","link":"https:\/\/www.reframetech.de\/en\/tag\/algo-rules-en\/"},{"id":263,"name":"Algorithm","link":"https:\/\/www.reframetech.de\/en\/tag\/algorithm\/"},{"id":302,"name":"Analysis","link":"https:\/\/www.reframetech.de\/en\/tag\/analysis\/"},{"id":274,"name":"Ethics","link":"https:\/\/www.reframetech.de\/en\/tag\/ethics\/"},{"id":301,"name":"Ethics of Algorithms","link":"https:\/\/www.reframetech.de\/en\/tag\/ethics-of-algorithms\/"},{"id":639,"name":"Publications","link":"https:\/\/www.reframetech.de\/en\/tag\/publications\/"}]}