{"id":150,"title":"Nine corrective responses to Algorithmic Decision-Making","link":"https:\/\/www.reframetech.de\/en\/2017\/07\/31\/nine-corrective-responses-to-algorithmic-decision-making\/","date":"07\/31\/2017","date_unix":1501501897,"date_modified_unix":1641380870,"date_iso":"2017-07-31T11:51:37+00:00","content":"<section class=\"section section--body\">\n<div class=\"section-content\">\n<div class=\"section-inner sectionLayout--insetColumn\">\n<section class=\"section section--body section--first\">\n<div class=\"section-content\">\n<div class=\"section-inner sectionLayout--insetColumn\">\n<p id=\"c653\" class=\"graf graf--p graf-after--figure\">Processes of algorithmic decision-making (ADM) now evaluate people in many areas of life. To take advantage of the opportunities ADM offers in the area of participation, one overall goal must be set when ADM processes are planned, designed and implemented: ensuring that participation actually increases. If this is not the case, the use of these tools could in fact lead to greater social inequality.<\/p>\n<p id=\"45d5\" class=\"graf graf--p graf-after--p\">In sum, the opportunities and risks in the examples presented in our <em class=\"markup--em markup--p-em\">working paper \u201c<a href=\"http:\/\/www.bertelsmann-stiftung.de\/de\/publikationen\/publikation\/did\/when-machines-judge-people-1\/\" target=\"_blank\" aria-label=\"Opens in a new tab\" >When Machines Judge People &#8211;\u00a0International Case Studies of Algorithmic Decision-making<\/a><\/em><em class=\"markup--em markup--p-em\">\u201d\u00a0<\/em>point to a number of general factors related to ADM processes that can critically affect participation. These factors involve different aspects of the overall socio-informatic process and can be found on different levels. Here are three examples:<\/p>\n<ul class=\"postList\">\n<li id=\"e61e\" class=\"graf graf--li graf-after--p\"><strong class=\"markup--strong markup--li-strong\">Shaping ADM processes on the micro and macro level<\/strong>: Choosing data and setting criteria at the startof a development process can themselves reflect normative principles which sometimes touch on fundamental social issues.<\/li>\n<li id=\"6bb6\" class=\"graf graf--li graf-after--li\"><strong class=\"markup--strong markup--li-strong\">Structure of suppliers and operators on the macro level<\/strong>: Having a range of ADM processes andoperators can increase participation (e.g. through credit assessments of people who have not been partof the system in the past), can make it easier to avoid the ADM process and can expand possibilities forfalsification. Conversely, monopolistic structures increase the risk that individuals will \u201cfall out of the system\u201d and get left behind.<\/li>\n<li id=\"2c43\" class=\"graf graf--li graf-after--li\"><strong class=\"markup--strong markup--li-strong\">Use of ADM forecasts on the micro, meso and macro level<\/strong>: The interplay of technology, society andindividuals has a major impact on how and when algorithms are used and the influence they thus have.Key questions that must therefore be asked are: How do people (ADM developers and users, and thegeneral public) deal with automated predictions? Do the processes include the possibility of challengingADM results?<\/li>\n<\/ul>\n<p id=\"ecfd\" class=\"graf graf--p graf-after--li graf--trailing\">Criteria are needed for determining the benefits of ADM processes on all levels and in all steps &#8211; from the definition of the goals and the efforts to measure the issues at hand, to data collection, the selecting of algorithms and the embedding of processes in the relevant social context.<\/p>\n<p class=\"graf graf--p graf-after--li graf--trailing\">The nine corrective responses discussed below\u00a0can provide initial impetus for addressing these issues:<\/p>\n<\/div>\n<\/div>\n<\/section>\n<h3 id=\"7708\" class=\"graf graf--h3 graf--leading\">1. Ensure falsifiability<\/h3>\n<p id=\"fbdb\" class=\"graf graf--p graf-after--h3\">ADM processes can learn asymmetrically from mistakes. \u201cAsymmetric\u201d means that the system, by virtue of the design of the overall process, can only recognize in retrospect certain types of its own predictions which proved incorrect. When algorithms learn asymmetrically, the danger always ex-ists that self-reinforcing feedback loops will occur.<\/p>\n<p id=\"592f\" class=\"graf graf--p graf-after--p graf--trailing\"><em class=\"markup--em markup--p-em\">Example: Recidivism predictions used in the legal system<\/em><\/p>\n<\/div>\n<\/div>\n<\/section>\n<section class=\"section section--body\">\n<div class=\"section-divider\">\n<hr class=\"section-divider\" \/>\n<\/div>\n<div class=\"section-content\">\n<div class=\"section-inner sectionLayout--insetColumn\">\n<h3 id=\"00af\" class=\"graf graf--h3 graf--leading\">2. Ensure proper\u00a0use<\/h3>\n<p id=\"6fc5\" class=\"graf graf--p graf-after--h3\">Institutional logic can lead to ADM processes being used for completely different purposes than originally envisioned by their developers. Such inappropriate uses must be avoided.<\/p>\n<p id=\"7219\" class=\"graf graf--p graf-after--p graf--trailing\"><em class=\"markup--em markup--p-em\">Example: Predicting individual criminal behavior<\/em><\/p>\n<\/div>\n<\/div>\n<\/section>\n<section class=\"section section--body\">\n<div class=\"section-divider\">\n<hr class=\"section-divider\" \/>\n<\/div>\n<div class=\"section-content\">\n<div class=\"section-inner sectionLayout--insetColumn\">\n<h3 id=\"7e24\" class=\"graf graf--h3 graf--leading\">3. Identify appropriate logic model for social\u00a0impact<\/h3>\n<p id=\"3f53\" class=\"graf graf--p graf-after--h3\">Algorithm-driven efficiency gains in individual process steps can obscure the question of whether the means used to solve a social problem are generally appropriate.<\/p>\n<p id=\"5b73\" class=\"graf graf--p graf-after--p graf--trailing\"><em class=\"markup--em markup--p-em\">Example: Predicting lead poisoning<\/em><\/p>\n<\/div>\n<\/div>\n<\/section>\n<section class=\"section section--body\">\n<div class=\"section-divider\">\n<hr class=\"section-divider\" \/>\n<\/div>\n<div class=\"section-content\">\n<div class=\"section-inner sectionLayout--insetColumn\">\n<h3 id=\"6cc4\" class=\"graf graf--h3 graf--leading\">4. Make concepts properly measurable<\/h3>\n<p id=\"e07b\" class=\"graf graf--p graf-after--h3\">Social phenomena or issues such as poverty and social ine-quality are often hard to operationalize. Robust benchmarks developed through public discussion are therefore helpful.<\/p>\n<p id=\"8d5a\" class=\"graf graf--p graf-after--p graf--trailing\"><em class=\"markup--em markup--p-em\">Example: Predicting patterns of poverty<\/em><\/p>\n<\/div>\n<\/div>\n<\/section>\n<section class=\"section section--body\">\n<div class=\"section-divider\">\n<hr class=\"section-divider\" \/>\n<\/div>\n<div class=\"section-content\">\n<div class=\"section-inner sectionLayout--insetColumn\">\n<h3 id=\"c363\" class=\"graf graf--h3 graf--leading\">5. Ensure comprehensive evaluation<\/h3>\n<p id=\"7776\" class=\"graf graf--p graf-after--h3\">The normative power of what is technically feasible all too easily eclipses the discussion of what makes sense from a social point of view. For example, the scalability of machine-based decisions can quickly lead to situations in which the appropriateness and consequences for society of using ADM processes have neither been debated nor verified.<\/p>\n<p id=\"389f\" class=\"graf graf--p graf-after--p graf--trailing\"><em class=\"markup--em markup--p-em\">Example: Automatic face-recognition systems<\/em><\/p>\n<\/div>\n<\/div>\n<\/section>\n<section class=\"section section--body\">\n<div class=\"section-divider\">\n<hr class=\"section-divider\" \/>\n<\/div>\n<div class=\"section-content\">\n<div class=\"section-inner sectionLayout--insetColumn\">\n<h3 id=\"66b3\" class=\"graf graf--h3 graf--leading\">6. Ensure diversity of ADM processes<\/h3>\n<p id=\"1b24\" class=\"graf graf--p graf-after--h3\">Once developed, the decision-making logic behind an ADM process can be applied in a great number of instances with-out any substantial increase in cost. One result is that a limited number of ADM processes can predominate in certain areas of application. The more extensive the reach, the more difficult it is for individuals to escape the process or its conse-quences.<\/p>\n<p id=\"90e6\" class=\"graf graf--p graf-after--p graf--trailing\"><em class=\"markup--em markup--p-em\">Example: Preselection of candidates using online personality tests<\/em><\/p>\n<\/div>\n<\/div>\n<\/section>\n<section class=\"section section--body\">\n<div class=\"section-divider\">\n<hr class=\"section-divider\" \/>\n<\/div>\n<div class=\"section-content\">\n<div class=\"section-inner sectionLayout--insetColumn\">\n<h3 id=\"3f3b\" class=\"graf graf--h3 graf--leading\">7. Facilitate verifiability<\/h3>\n<p id=\"fa06\" class=\"graf graf--p graf-after--h3\">Frequently, no effort is made to determine if an ADM process is sufficiently fair. Doing so is even impossible if the logic and nature of an algorithm is kept secret. Without verification by independent third parties, no informed debate on the opportu-nities and risks of a specific ADM process can take place.<\/p>\n<p id=\"56bc\" class=\"graf graf--p graf-after--p\"><em class=\"markup--em markup--p-em\">Example: University admissions in France<\/em><\/p>\n<h3 id=\"3732\" class=\"graf graf--h3 graf-after--p\">8. Consider social interdependencies<\/h3>\n<p id=\"37d9\" class=\"graf graf--p graf-after--h3\">Even when use is very limited, the interdependences between ADM processes and their environment are highly complex. Only an analysis of the entire socio-informatic process can reveal the relationship between opportunities and risks.<\/p>\n<p id=\"a961\" class=\"graf graf--p graf-after--p graf--trailing\"><em class=\"markup--em markup--p-em\">Example: Location-specific pre-dictions of criminal behavior<\/em><\/p>\n<\/div>\n<\/div>\n<\/section>\n<section class=\"section section--body\">\n<div class=\"section-divider\">\n<hr class=\"section-divider\" \/>\n<\/div>\n<div class=\"section-content\">\n<div class=\"section-inner sectionLayout--insetColumn\">\n<h3 id=\"b8a4\" class=\"graf graf--h3 graf--leading\">9. Prevent\u00a0misuse<\/h3>\n<p id=\"dcdb\" class=\"graf graf--p graf-after--h3\">Easily accessible predictions such as scoring results can be used for inappropriate purposes. Such misuse must be pre-vented at all costs.<\/p>\n<p id=\"9d4b\" class=\"graf graf--p graf-after--p graf--trailing\"><em class=\"markup--em markup--p-em\">Example: Credit scoring in the US<\/em><\/p>\n<\/div>\n<\/div>\n<\/section>\n<section class=\"section section--body section--last\">\n<div class=\"section-divider\">\n<hr \/>\n<p><em class=\"markup--em markup--p-em\">This is an excerpt from the working paper \u201c<a href=\"http:\/\/www.bertelsmann-stiftung.de\/de\/publikationen\/publikation\/did\/when-machines-judge-people-1\/\" target=\"_blank\" aria-label=\"Opens in a new tab\" >When Machines Judge People &#8211;\u00a0International Case Studies of Algorithmic Decision-making<\/a><\/em><em class=\"markup--em markup--p-em\">\u201d, written by Konrad Lischka and Anita Klingel, published by the Bertelsmann Stiftung under CC BY-SA 3.0 DE.<\/em><\/p>\n<\/div>\n<div class=\"section-content\">\n<div class=\"section-inner sectionLayout--insetColumn\">\n<p id=\"6f1f\" class=\"graf graf--p graf-after--p graf--trailing\"><em class=\"markup--em markup--p-em\">This publication documents the preliminary results of our investigation of the topic. We are publishing it as a working paper to contribute to this rapidly developing field in a way that others can build on.<\/em><\/p>\n<\/div>\n<\/div>\n<\/section>\n","excerpt":"<p>Processes of algorithmic decision-making (ADM) now evaluate people in many areas of life. To take advantage of the opportunities ADM [&hellip;]<\/p>\n","thumbnail":"https:\/\/www.reframetech.de\/wp-content\/uploads\/sites\/23\/2017\/07\/9460968490_31a4b10465_k.jpg","thumbnailsquare":"https:\/\/www.reframetech.de\/wp-content\/uploads\/sites\/23\/2017\/07\/9460968490_31a4b10465_k.jpg","authors":[{"id":10,"name":"Konrad Lischka","link":"https:\/\/www.reframetech.de\/blogger\/konrad-lischka-2\/"},{"id":114,"name":"Anita Klingel","link":"https:\/\/www.reframetech.de\/blogger\/anita-klingel-2\/"}],"categories":[{"id":698,"name":"Political decision-makers","link":"https:\/\/www.reframetech.de\/en\/category\/political-decision-makers\/"}],"tags":[{"id":639,"name":"Publications","link":"https:\/\/www.reframetech.de\/en\/tag\/publications\/"}]}