The Ethics of AI: Who will be Responsible for the Decisions of AI Applications?


Ayodele Odubela
Data Scientist
SambaSafety

One of the issues often debated in AI as it regards to ethics is who is responsible for the social and financial implications of machine learning decisions. I’ve observed a vast gap between how regulated industries and industries without regulations develop machine learning models.  

Credit card companies, risk monitoring, and other consumer reporting agencies care highly about creating models they can stand behind and legally defend, whereas many consumer products don’t disclose the nature of their social media research or models. 

The same way the insurance and automotive industries consider accidents in a matter of who is at fault, who can we hold responsible when an algorithm predicts poorly? 

Some suggest individual engineers should be responsible for the impact their models have on others’ lives. If this were the case we would see a larger emphasis on ethics and data privacy being taught in computer science and data science programs. Are they at fault because it’s ultimately their responsibility to choose representative training data and use real data to monitor performance? Data Scientists also can consult with outside companies to audit their data and models.  

There are also proponents of policies that would mandate companies that create algorithms that have a drastic impact on human life (in many industries like healthcare and self-driving vehicles it can mean life or death) make their models and training data open to the public. One of the major reasons this is a hard issue to solve is that companies see the algorithms as their “secret sauce”. This business approach has created a high demand and regard for data professionals, however, this comes at the detriment of model transparency. 

Many have heard of the Facebook Research project that manipulated users’ timelines to measure positive or negative effects. In academic research, there are criteria that proper research must meet to be ethical, especially when the subjects err human. The most egregious offense is the violation of user consent. While buried in the Facebook Terms of Service, few users knew Facebook was using them as emotional test subjects. Internet companies are rarely subject to the stringent criteria of academic research or the explainability of consumer reporting agencies due to few policies that preside over their models. 

One of the major issues in determining fault when an algorithm fails is that we don’t have comprehensive policies that provide consumer protections and litigation opportunities when we’re impacted by a machine learning model. Take the credit card industry for example, in 1970 the Fair Credit Reporting Act was passed to protect consumers from advantageous credit companies. Over the years this has expanded to include many companies who are not credit agencies, but their services can determine whether someone will get a job or not. These companies work to create more interpretable models and prefer using flavors of linear regression over black box modes to provide consumers insight into why a decision was made. Data Scientists in these organizations often prioritize creating models they can defend over black-box models even with state-of-the-art performance. 

This brings us back to who is at fault. If an algorithm incorrectly predicts a convicted person will reoffend and in the actual case they didn’t reoffend, who owes the convicted person an explanation? Is it the company for having the most resources to rectify the false positive? Do we blame the engineer for not building a more strict model or for including proxies for race and gender? Do we blame the end-user like a county judge for using a tool like COMPASS to predict recidivism without feedback on how well the model works? 

We are even having conversations about holding the AI themselves accountable, but that’s an intensive task that involves deciding what kinds of consequences or penalties an AI would face after making an incorrect prediction. One solution to simplify the AI responsibility quandary is human in the loop systems. In these scenarios we see AI making recommendations to the human interacting with it. For a remote medical visit, a doctor can receive an image of a growth and have a computer vision program attempt to identify it. The computer can output its 

Ultimately, it’s up to practitioners, including Data Scientists and Machine Learning Engineers, to follow best practices and mitigate how bias can impact their AI models. To have public access to data models created and used by large firms, we need to enact policies that would incentivize open data. We desperately need a push towards learning AI Ethics in all levels of tech instruction including academia, online MooCs, and bootcamps. 

Additional Reading:

Barr, A., 2015. Google Mistakenly Tags Black People as ‘Gorillas,’ Showing Limits of Algorithms [WWW Document]. Wall Str. J. URL https://blogs.wsj.com/digits/2015/07/01/google-mistakenly-tags-black-people-as-gorillas-showing-limits-of-algorithms/ (accessed 1.23.19).

Larson, J., Mattu, S., Kirchner, L., Angwin, J., 2016. How We Analyzed the COMPAS Recidivism Algorithm [WWW Document]. ProPublica. URL https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm (accessed 1.23.19).

O’Neil, C., 2016. Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy, 1 edition. ed. Crown, New York.


Comments

Leave a Reply

Your email address will not be published. Required fields are marked *

ปั๊มไลค์

Like!! I blog quite often and I genuinely thank you for your information. The article has truly peaked my interest.

Related Posts

Recent Posts

Three Amazing Ways AI is Revolutionizing Healthcare - It may not seem like it was too long ago when the idea of artificial intelligence playing a major role…
How 5G is Going to Impact AI in Automation Within Telecom - During this webinar, an industry expert discussed how an automation project comes to life from the initial business problem through…
How Automation Projects Come to Life in Telecom - During this webinar, an industry expert discussed how an automation project comes to life from the initial business problem through…
The Future of AI in Marketing - During this webinar, industry experts discussed where AI in marketing was heading in the future. We’ve included a short transcription…
How AI Has Changed Marketing - During this webinar, industry experts discussed how AI has changed the marketing industry. We’ve included a short transcription of the…
Key Takeaways From Ai4 2020 - Artificial Intelligence Creates the Demand of Innovation, Autonomy, and Personalization Amidst a Crisis There is a seemingly quiet, yet enormous…
Computer Vision Versus Other ML Projects - During this webinar, industry experts discussed computer vision projects versus other machine learning projects within an enterprise setting. We’ve included…
Computer Vision in the Enterprise - During this webinar, industry experts discussed if computer vision computer is commonplace within enterprises that have machine learning models in…
How AI is Enabling Banks to Provide a Better User Experience - During this webinar, industry experts discussed how AI is enabling banks to provide a better user experience for having both…
Where AI is Interacting with Customers in Banking - During this webinar, industry experts discussed the main areas where AI is interacting with customers in banking. We’ve included a…

Popular Posts

Does Healthcare AI Meet Basic Ethics Principles? - Ingrid Vasiliu-Feltes Chief Quality and Innovation Officer MEDNAX, Health Solutions Partner Over the past decade we have noticed an exponential…
Machine Learning and Artificial Intelligence in Banking - Artit "Art" Wangperawong Distinguished Engineer US Bank Introduction Every company’s AI journey is different. We’re all trying to figure out…
Machine Learning for Pricing and Inventory Optimization @ Macy’s - Jolene Mork Senior Data Scientist Macy's Iain Stitt Data Scientist Macy's Bhagyesh Phanse VP, Data Science Macy's Overview In this…
Artificial Intelligence & Cybersecurity: Math Not Magic - Wayne Chung CTO FBI Introduction The field of cybersecurity has slowly progressed from an art to a science. It has…
AI/ML in Investment and Risk Management: Recent Applications, Use Cases, and Implementation Challenges - Arvind Rajan Managing Director - Head of Global & Macro PGIM Fixed Income Introduction Investing is a completely different ballgame…
Top AI Conferences - Interested in learning the latest in AI this year? We’ve compiled a list of the top artificial intelligence conferences in…
Machine Learning in Production: From Research to the Customer - Ameen Kazerouni Lead Data Scientist Zappos Overview In this presentation Ameen Kazerouni, the Lead Data Scientist at Zappos, walks through…
How COVID-19 is Impacting the State of AI in Banking - On this panel, industry experts (listed above) discussed The State of AI in Banking and how COVID-19 is affecting it.…
“Ask Me Anything” with Zappos’s Head of AI/ML Research & Platforms, Ameen Kazerouni - Ameen Kazerouni Head of AI/ML Research & Platforms Zappos Family of Companies Ai4 recently hosted an "Ask Me Anything" session…
The Autonomous Pharmacy: Applying AI and ML to Medication Management Across the Care Continuum - Ken Perez VP of Healthcare Policy Omnicell, Inc. Ken applies artificial intelligence (AI) and machine learning (ML) solutions to medication…