Influence Attacks on Machine Learning


Mark Sherman
Technical Director, Cyber Security Foundations, CERT Division
Carnegie Mellon University Software Engineering Institute

Overview

Mark Sherman explains how deep learning is playing an increasing role in developing new applications and how adversaries can attack machine learning systems in a variety of ways.

Transcription

Hello, this is Mark Sherman from the CERT division with your SEI CyberMinute. Techniques such as multi-level neural nets, or more commonly deep learning, are playing an increasing role in developing new applications. Users need to trust that these systems are operating correctly. Intuitively, one can imagine systems being fooled by visually similar images of wildly different objects.

Watch the full video here.

The muffins and Chihuahua meme produced by Karen Zack, also known as Teeny Biscuit, is compelling to the human eye. But findings reported by researchers at Chubu University in Japan demonstrate machine learning systems can be fooled by as few as one changed pixel in an image. Adversaries can attack learning systems in a variety of ways.

Using the taxonomy suggested by Reno, et al, influence represents one axis of attacker capability. The influence can manifest itself in several different ways. First, the attack could attempt to influence the training data. Incorrect labeling or clustering can result in mislabeled results when deployed.

Second, the attack could attempt to influence the evaluation or test data. The result could be overconfidence of a poorly working system or distrust of a working system. Third, the attack could attempt single poisoning, such as the selective change of input data to the working system. Changing a single pixel can effectively camouflage an image to the machine learning system.

Fourth, the attack could attempt boiling frog poisoning. Active learning systems adjust their behavior based on the inputs they receive. They naturally drift to improve accuracy. In this attack, an adversary provides data that are at the edge of the discrimination algorithm and slowly move the edge where the drift results in incorrect conclusions.

Many organizations such as the SEI, are engaged in research to mitigate these attacks in support of the defense science board’s recommendation for independent verification and validation for machine learning. Thanks for watching this SEI CyberMinute. For more information, please visit our website or send me an email at [email protected]


Comments

Leave a Reply

Your email address will not be published. Required fields are marked *

Related Posts

Recent Posts

COVID-19 Curfew and Social Distancing Enforcement using AI enabled Drones - https://youtu.be/A41BASN9TA8 Amarjot Singh, PhDFounder & CEOSkyLark Labs LLC The SARS-CoV-2(also known as COVID-19 or Novel Corona Virus) infectious outbreak has rapidly…
“Ask Me Anything” with Roboticist & Program Lead of Racing Vehicles at General Motors - Harish SkumarRoboticist & Program Lead of Racing Vehicles General Motors Ai4 recently hosted an “Ask Me Anything” session with Harish…
Bottlenecks in Supply Chains & How AI Can Help - During this panel, industry experts (showed above) discussed the impact of COVID-19 on AI on Supply Chains. We’ve included a…
How COVID-19 is Impacting the State of AI in Supply Chains - During this panel, industry experts (showed above) discussed the impact of COVID-19 on AI on Supply Chains. We’ve included a…
How COVID-19 is Impacting the State of AI in Investment Management - On this panel, industry experts (listed above) discussed the impact of COVID-19 on AI on Investment Management. We've included a…
The State of AI in Investment Management - On this panel, industry experts (listed above) discussed the affects of AI on Investment Management. We've included a short transcription…
The State of AI in Banking - On this panel, industry experts (listed above) discussed what they are most excited about in AI in Banking. We've included…
How COVID-19 is Impacting the State of AI in Banking - On this panel, industry experts (listed above) discussed The State of AI in Banking and how COVID-19 is affecting it.…
The Ethics of AI: Who will be Responsible for the Decisions of AI Applications? - Ayodele Odubela Data Scientist SambaSafety One of the issues often debated in AI as it regards to ethics is who…
“Ask Me Anything” with Krzysztof Geras, PhD - Krzysztof Geras Assistant Professor NYU Department of Radiology Ai4's recently hosted an "Ask Me Anything" session with one of our…

Popular Posts

Does Healthcare AI Meet Basic Ethics Principles? - Ingrid Vasiliu-Feltes Chief Quality and Innovation Officer MEDNAX, Health Solutions Partner Over the past decade we have noticed an exponential…
AI/ML in Investment and Risk Management: Recent Applications, Use Cases, and Implementation Challenges - Arvind Rajan Managing Director - Head of Global & Macro PGIM Fixed Income Introduction Investing is a completely different ballgame…
Top AI Conferences - Interested in learning the latest in AI this year? We’ve compiled a list of the top artificial intelligence conferences in…
Artificial Intelligence & Cybersecurity: Math Not Magic - Wayne Chung CTO FBI Introduction The field of cybersecurity has slowly progressed from an art to a science. It has…
Machine Learning for Pricing and Inventory Optimization @ Macy’s - Jolene Mork Senior Data Scientist Macy's Iain Stitt Data Scientist Macy's Bhagyesh Phanse VP, Data Science Macy's Overview In this…
“Ask Me Anything” with Zappos’s Head of AI/ML Research & Platforms, Ameen Kazerouni - Ameen Kazerouni Head of AI/ML Research & Platforms Zappos Family of Companies Ai4 recently hosted an "Ask Me Anything" session…
Advancements at Siemens Healthineers in AI for Medical Imaging - Bimba Rao Head of Global Artificial Intelligence Engineering Siemens Healthineers Ultrasound Siemens Healthineers background  Siemens Healthineers builds healthcare products and…
Leveraging AI in Cybersecurity Risk Modeling & Mitigation - Christopher Novak Director, Threat Advisory Research Verizon Wireless Introduction Originally, there was a poor understanding of why cyber breaches were…
An Ensemble Approach to Predict Default Risk in Stress Testing - Yun Zheng VP of Innovation & Global Risk Analytics HSBC Overview This presentation discussed the importance of performing stress tests…
Machine Learning and Artificial Intelligence in Banking - Artit "Art" Wangperawong Distinguished Engineer US Bank Introduction Every company’s AI journey is different. We’re all trying to figure out…