Influence Attacks on Machine Learning


Mark Sherman
Technical Director, Cyber Security Foundations, CERT Division
Carnegie Mellon University Software Engineering Institute

Overview

Mark Sherman explains how deep learning is playing an increasing role in developing new applications and how adversaries can attack machine learning systems in a variety of ways.

Transcription

Hello, this is Mark Sherman from the CERT division with your SEI CyberMinute. Techniques such as multi-level neural nets, or more commonly deep learning, are playing an increasing role in developing new applications. Users need to trust that these systems are operating correctly. Intuitively, one can imagine systems being fooled by visually similar images of wildly different objects.

Watch the full video here.

The muffins and Chihuahua meme produced by Karen Zack, also known as Teeny Biscuit, is compelling to the human eye. But findings reported by researchers at Chubu University in Japan demonstrate machine learning systems can be fooled by as few as one changed pixel in an image. Adversaries can attack learning systems in a variety of ways.

Using the taxonomy suggested by Reno, et al, influence represents one axis of attacker capability. The influence can manifest itself in several different ways. First, the attack could attempt to influence the training data. Incorrect labeling or clustering can result in mislabeled results when deployed.

Second, the attack could attempt to influence the evaluation or test data. The result could be overconfidence of a poorly working system or distrust of a working system. Third, the attack could attempt single poisoning, such as the selective change of input data to the working system. Changing a single pixel can effectively camouflage an image to the machine learning system.

Fourth, the attack could attempt boiling frog poisoning. Active learning systems adjust their behavior based on the inputs they receive. They naturally drift to improve accuracy. In this attack, an adversary provides data that are at the edge of the discrimination algorithm and slowly move the edge where the drift results in incorrect conclusions.

Many organizations such as the SEI, are engaged in research to mitigate these attacks in support of the defense science board’s recommendation for independent verification and validation for machine learning. Thanks for watching this SEI CyberMinute. For more information, please visit our website or send me an email at [email protected]


Comments

Leave a Reply

Your email address will not be published. Required fields are marked *

Related Posts

Recent Posts

Three Amazing Ways AI is Revolutionizing Healthcare - It may not seem like it was too long ago when the idea of artificial intelligence playing a major role…
How 5G is Going to Impact AI in Automation Within Telecom - During this webinar, an industry expert discussed how an automation project comes to life from the initial business problem through…
How Automation Projects Come to Life in Telecom - During this webinar, an industry expert discussed how an automation project comes to life from the initial business problem through…
The Future of AI in Marketing - During this webinar, industry experts discussed where AI in marketing was heading in the future. We’ve included a short transcription…
How AI Has Changed Marketing - During this webinar, industry experts discussed how AI has changed the marketing industry. We’ve included a short transcription of the…
Key Takeaways From Ai4 2020 - Artificial Intelligence Creates the Demand of Innovation, Autonomy, and Personalization Amidst a Crisis There is a seemingly quiet, yet enormous…
Computer Vision Versus Other ML Projects - During this webinar, industry experts discussed computer vision projects versus other machine learning projects within an enterprise setting. We’ve included…
Computer Vision in the Enterprise - During this webinar, industry experts discussed if computer vision computer is commonplace within enterprises that have machine learning models in…
How AI is Enabling Banks to Provide a Better User Experience - During this webinar, industry experts discussed how AI is enabling banks to provide a better user experience for having both…
Where AI is Interacting with Customers in Banking - During this webinar, industry experts discussed the main areas where AI is interacting with customers in banking. We’ve included a…

Popular Posts

Does Healthcare AI Meet Basic Ethics Principles? - Ingrid Vasiliu-Feltes Chief Quality and Innovation Officer MEDNAX, Health Solutions Partner Over the past decade we have noticed an exponential…
Machine Learning and Artificial Intelligence in Banking - Artit "Art" Wangperawong Distinguished Engineer US Bank Introduction Every company’s AI journey is different. We’re all trying to figure out…
Machine Learning for Pricing and Inventory Optimization @ Macy’s - Jolene Mork Senior Data Scientist Macy's Iain Stitt Data Scientist Macy's Bhagyesh Phanse VP, Data Science Macy's Overview In this…
Artificial Intelligence & Cybersecurity: Math Not Magic - Wayne Chung CTO FBI Introduction The field of cybersecurity has slowly progressed from an art to a science. It has…
AI/ML in Investment and Risk Management: Recent Applications, Use Cases, and Implementation Challenges - Arvind Rajan Managing Director - Head of Global & Macro PGIM Fixed Income Introduction Investing is a completely different ballgame…
Top AI Conferences - Interested in learning the latest in AI this year? We’ve compiled a list of the top artificial intelligence conferences in…
Machine Learning in Production: From Research to the Customer - Ameen Kazerouni Lead Data Scientist Zappos Overview In this presentation Ameen Kazerouni, the Lead Data Scientist at Zappos, walks through…
How COVID-19 is Impacting the State of AI in Banking - On this panel, industry experts (listed above) discussed The State of AI in Banking and how COVID-19 is affecting it.…
“Ask Me Anything” with Zappos’s Head of AI/ML Research & Platforms, Ameen Kazerouni - Ameen Kazerouni Head of AI/ML Research & Platforms Zappos Family of Companies Ai4 recently hosted an "Ask Me Anything" session…
The Autonomous Pharmacy: Applying AI and ML to Medication Management Across the Care Continuum - Ken Perez VP of Healthcare Policy Omnicell, Inc. Ken applies artificial intelligence (AI) and machine learning (ML) solutions to medication…