{"id":2044,"date":"2019-11-01T05:49:37","date_gmt":"2019-11-01T05:49:37","guid":{"rendered":"http:\/\/kusuaks7\/?p=1649"},"modified":"2024-03-05T17:13:55","modified_gmt":"2024-03-05T17:13:55","slug":"robust-ai-protecting-neural-networks-against-adversarial-attacks","status":"publish","type":"post","link":"https:\/\/www.experfy.com\/blog\/ai-ml\/robust-ai-protecting-neural-networks-against-adversarial-attacks\/","title":{"rendered":"Robust AI: Protecting neural networks against adversarial attacks"},"content":{"rendered":"\t\t<div data-elementor-type=\"wp-post\" data-elementor-id=\"2044\" class=\"elementor elementor-2044\" data-elementor-post-type=\"post\">\n\t\t\t\t\t\t<section class=\"has_eae_slider elementor-section elementor-top-section elementor-element elementor-element-7c6daa0b elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"7c6daa0b\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"has_eae_slider elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-4d3846a\" data-id=\"4d3846a\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-ced64de elementor-widget elementor-widget-text-editor\" data-id=\"ced64de\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\tIn its latest annual report, filed with the Securities and Exchange Commission, tech giant Alphabet\u00a0<a href=\"https:\/\/www.wired.com\/story\/google-microsoft-warn-ai-may-do-dumb-things\/\" target=\"_blank\" rel=\"noopener noreferrer\">warned investors<\/a>\u00a0against the many challenges of artificial intelligence, following the lead of Microsoft, which issued similar warnings last August.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-904d42a elementor-widget elementor-widget-text-editor\" data-id=\"904d42a\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\tRecent advances in deep learning and neural networks have created much hope about the possibilities that AI presents to various domains that were previously thought to be off the limits for computer software. But there\u2019s also concern about new threats AI will pose to different fields, especially where bad decisions can have very destructive results.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-c2b35ae elementor-widget elementor-widget-text-editor\" data-id=\"c2b35ae\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\tWe\u2019ve already seen some of these threats manifest themselves in various ways, including\u00a0<a href=\"https:\/\/bdtechtalks.com\/2018\/03\/26\/racist-sexist-ai-deep-learning-algorithms\/\" rel=\"noopener\">biased algorithms<\/a>,\u00a0<a href=\"https:\/\/bdtechtalks.com\/2018\/04\/16\/artificial-intelligence-deepfakes-blockchain\/\" rel=\"noopener\">AI-based forgery<\/a>\u00a0and the spread of fake news during important events such as elections.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-1e35766 elementor-widget elementor-widget-text-editor\" data-id=\"1e35766\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\nThe past few years have seen the development of a growing discussion around building trust in artificial intelligence and creating safeguards that prevent abuse and malicious behavior of AI models. The various efforts are focused in three fields of fairness,\u00a0<a href=\"https:\/\/bdtechtalks.com\/2018\/09\/25\/explainable-interpretable-ai\/\" rel=\"noopener\">explainability<\/a>\u00a0and robustness.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-029e91b elementor-widget elementor-widget-text-editor\" data-id=\"029e91b\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\nIn an interview with TechTalks,\u00a0<a href=\"https:\/\/researcher.watson.ibm.com\/researcher\/view.php?person=ibm-Pin-Yu.Chen\" target=\"_blank\" rel=\"noopener noreferrer\">Pin-Yu Chen<\/a>, researcher at the MIT-IBM Watson AI Lab, explained why it\u2019s important to create robust AI models and how to evaluate the resilience of artificial intelligence algorithms against abuse and erratic behavior. Chen is member of a team researchers who recently released two papers on AI robustness and presented them at the Association for Advancement of Artificial Intelligence (AAAI) conference.               \t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-724ed51 elementor-widget elementor-widget-heading\" data-id=\"724ed51\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\"><h2>Why AI robustness matters<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-236c084 elementor-widget elementor-widget-text-editor\" data-id=\"236c084\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\tNeural networks, the main components of\u00a0<a href=\"https:\/\/bdtechtalks.com\/2019\/02\/15\/what-is-deep-learning-neural-networks\/\" rel=\"noopener\">deep learning algorithms<\/a>, the most popular blend of AI, have proven to be very accurate at performing complicated tasks such classifying images, recognizing speech and voice, and translating text. But as Chen points out, accuracy can\u2019t be the sole metric to grade an AI model.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-76212c4 elementor-widget elementor-widget-text-editor\" data-id=\"76212c4\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\tA lot of domains require AI models to be trustworthy, Chen explains, which means we must be able to understand how an AI model develops its behavior and how it makes decisions. We also must have tools to evaluate how reliable the AI is in various situations.\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-1d10169 elementor-widget elementor-widget-text-editor\" data-id=\"1d10169\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>\u201cIf we deploy an AI model for some safety-critical task, say autonomous vehicles, we want to make sure the if the self-driving car sees a stop sign it will stop and it will realize what a stop sign means,\u201d Chen says.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-94e57c5 elementor-widget elementor-widget-text-editor\" data-id=\"94e57c5\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\tThat example hides a lot of important details about advances and developments in artificial intelligence. First, AI has helped software find its way into many applications of the physical world.\u00a0<a href=\"https:\/\/bdtechtalks.com\/2019\/01\/14\/what-is-computer-vision\/\" rel=\"noopener\">Computer vision algorithms<\/a>\u00a0are one of the main technologies that are enabling cars to navigate streets\u00a0<a href=\"https:\/\/bdtechtalks.com\/2018\/09\/17\/self-driving-cars-ai-computer-vision\/\" rel=\"noopener\">without human drivers<\/a>. But this also means that mistakes by AI algorithms can have dire and possibly fatal consequences. In 2017, researchers found that by making small tweaks to stop signs, they could\u00a0<a href=\"https:\/\/arstechnica.com\/cars\/2017\/09\/hacking-street-signs-with-stickers-could-confuse-self-driving-cars\/\" target=\"_blank\" rel=\"noopener noreferrer\">fool self-driving cars<\/a>\u00a0to bypass them and cause dangerous situations.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-7fbe301 elementor-widget elementor-widget-image\" data-id=\"7fbe301\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" src=\"https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2018\/12\/ai-adversarial-attack-stop-sign.png?fit=696%2C390&#038;ssl=1\" alt=\"\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-5c5ec95 elementor-widget elementor-widget-text-editor\" data-id=\"5c5ec95\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p style=\"text-align: center;\"><span style=\"font-size: 11px;\">AI researchers discovered that by adding small black and white stickers to stop signs, they could make them invisible to computer vision algorithms (Source: arxiv.org)<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-455bddb elementor-widget elementor-widget-heading\" data-id=\"455bddb\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\"><h2>Adversarial examples<\/h2>\n<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-e7631ff elementor-widget elementor-widget-text-editor\" data-id=\"e7631ff\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\tPreviously, developers created image classification AI algorithms and tested them against one of several popular computer vision datasets to evaluate how fair they scored on their image samples. The higher the score, the more reliable the model was considered to be. But accuracy alone can create a false sense of trust.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-9bf8907 elementor-widget elementor-widget-text-editor\" data-id=\"9bf8907\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\u201cSome neural networks already perform with higher precision that humans. This may make us feel that these AI models are ready for deploying to different tasks,\u201d Chen says, reminding that even the most accurate models can be vulnerable to \u201c<a href=\"https:\/\/bdtechtalks.com\/2018\/12\/27\/deep-learning-adversarial-attacks-ai-malware\/\" rel=\"noopener\">adversarial perturbations<\/a>.\u201d\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-4dc4de6 elementor-widget elementor-widget-text-editor\" data-id=\"4dc4de6\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\tAdversarial perturbations are also known as adversarial examples or adversarial attacks, depending on the context in which they\u2019re created, and they involve making small changes to input data to manipulate the results of an AI model.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-7844955 elementor-widget elementor-widget-text-editor\" data-id=\"7844955\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\tFor instance, when you provide an image classifier algorithm with a photo, it will output a list of possible classes\u2014say cat, dog and wolf\u2014and associate each class with a confidence score ranging between 0 and 1. The class with the highest score is considered the AI model\u2019s prediction for that input. Adversarial perturbation adds small details to the input image in a way that causes the algorithm to change its confidence scores in favor of another class. The ingenuity of adversarial attacks is that the changes made to the input images are not distinguishable to humans.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-a96220b elementor-widget elementor-widget-image\" data-id=\"a96220b\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" src=\"https:\/\/i1.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2019\/02\/ai-adversarial-example-panda-gibbon.png?fit=696%2C271&#038;ssl=1\" alt=\"\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-b29a59f elementor-widget elementor-widget-text-editor\" data-id=\"b29a59f\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p style=\"text-align: center;\"><span style=\"font-size: 11px;\">Source: Arxiv.org<\/span><\/p>\nFor instance, the image above has been blended with a layer of noise to create an adversarial example. Any human will say with a very high level of confidence that both are pictures of a panda. Most people won\u2019t even be able to tell the difference between the two images. But an image classification AI algorithm will classify the right image as a \u201cgibbon\u201d with a 99.3 percent level of confidence.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-926dbf6 elementor-widget elementor-widget-text-editor\" data-id=\"926dbf6\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\tIn the image below, adversarial perturbations have caused a neural network to mistake a turtle for a rifle.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-722398f elementor-widget elementor-widget-image\" data-id=\"722398f\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" src=\"https:\/\/i0.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2018\/12\/ai-adversarial-attack-turtle.png?fit=696%2C394&#038;ssl=1\" alt=\"\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-9df6b18 elementor-widget elementor-widget-text-editor\" data-id=\"9df6b18\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p style=\"text-align: center;\"><span style=\"font-size: 11px;\">Researchers at labsix showed how a modified toy turtle could fool deep learning algorithms into classifying it as a rifle (source: labsix.org)<\/span><\/p>\nWhile neural networks are becoming increasingly efficient at yielding results that match or exceed the accuracy of the human vision system, they can fail in unexpected ways, highlighting the\u00a0<a href=\"https:\/\/bdtechtalks.com\/2018\/08\/21\/artificial-intelligence-vs-human-mind-brain\/\" rel=\"noopener\">stark differences between AI and human intelligence<\/a>.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-ec28bae elementor-widget elementor-widget-text-editor\" data-id=\"ec28bae\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\tIn the past couple of years, there have been growing interest in developing methods to discover and patch adversarial examples in neural networks. Research has proven that even the most accurate AI models can be vulnerable to adversarial attacks, casting doubt over their reliability in critical use cases.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-ddcf611 elementor-widget elementor-widget-text-editor\" data-id=\"ddcf611\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\n\u201cAdversarial examples could be very problematic, because if we\u2019re going to deploy AI in safety-critical or security-sensitive applications, then these models cannot be trusted because they can be easily fooled,\u201d Chen says.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-dbb125c elementor-widget elementor-widget-text-editor\" data-id=\"dbb125c\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\tThe work that Chen and his colleagues at the MIT-IBM Watson AI Lab are doing focuses mainly on evaluating the robustness of AI models against adversarial examples.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-e6988dc elementor-widget elementor-widget-text-editor\" data-id=\"e6988dc\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\u201cRobustness is about worst-case scenario performance,\u201d Chen says. \u201cIt\u2019s about how confident you are that your AI will classify a stop sign as a stop sign under different circumstances and how easy it is for an adversary to manipulate the prediction result of a stop sign into something else.\u201d\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-0904846 elementor-widget elementor-widget-heading\" data-id=\"0904846\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\"><h2>Adversarial attacks against black-box AI models<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-4e5b0f4 elementor-widget elementor-widget-text-editor\" data-id=\"4e5b0f4\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\tPart of evaluating the robustness of all software and computer systems is to test them under duress and attacks. An example is penetration testing, where cybersecurity experts perform different attacks on a system to discover flaws and vulnerabilities.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-ffb62f7 elementor-widget elementor-widget-text-editor\" data-id=\"ffb62f7\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\tLikewise, developers must probe their AI models for vulnerabilities to adversarial perturbations by testing them against various adversarial examples. The first MIT-IBM paper introduces a method to optimize adversarial attacks against black-box AI models.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-3ae4351 elementor-widget elementor-widget-text-editor\" data-id=\"3ae4351\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\u201cIn creating adversarial examples, people usually assume attackers have full knowledge of the model, including training data, network architecture and weights. So nothing is hidden from the attacker,\u201d Chen says.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-eef9a37 elementor-widget elementor-widget-text-editor\" data-id=\"eef9a37\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\tBut this is not a practical assumption, Chen argues, because in many cases, those details are hidden, and the attacker only has access to a black-box AI model. For instance, you can\u2019t use this method to generate adversarial examples on an online AI-based image classification service.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-1022de6 elementor-widget elementor-widget-text-editor\" data-id=\"1022de6\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\tBut this is not a practical assumption, Chen argues, because in many cases, those details are hidden, and the attacker only has access to a black-box AI model. For instance, you can\u2019t use this method to generate adversarial examples on an online AI-based image classification service.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-78d98c7 elementor-widget elementor-widget-text-editor\" data-id=\"78d98c7\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\tBut this is not a practical assumption, Chen argues, because in many cases, those details are hidden, and the attacker only has access to a black-box AI model. For instance, you can\u2019t use this method to generate adversarial examples on an online AI-based image classification service.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-6e3126e elementor-widget elementor-widget-text-editor\" data-id=\"6e3126e\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\u201cAll you can do is upload an image, and the image classifier will tell you for example that the image is 99 percent a cat,\u201d Chen says, adding that previously, developers believed black-box AI models would be resilient against adversarial attacks.\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-635cd0e elementor-widget elementor-widget-text-editor\" data-id=\"635cd0e\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\tHowever, \u201csecurity through obscurity\u201d is a failed approach, as has been proven\u00a0<a href=\"https:\/\/bdtechtalks.com\/2019\/02\/01\/apple-facetime-security-vulnerability\/\" rel=\"noopener\">time and again<\/a>. But several research papers have shown that hiding the details of AI models won\u2019t make them robust against adversarial examples.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-e7d1b5d elementor-widget elementor-widget-text-editor\" data-id=\"e7d1b5d\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\tIn 2017,\u00a0<a href=\"https:\/\/arxiv.org\/abs\/1708.03999\" target=\"_blank\" rel=\"noopener noreferrer\">a paper by the MIT-IBM Watson AI Lab\u00a0<\/a>first showed that with enough examples and testing, an attacker would be able to find adversarial vulnerabilities in AI models without having access to their architecture and inner details. The work proved that output confidence scores alone provide enough information to develop adversarial examples.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-63b4f5c elementor-widget elementor-widget-text-editor\" data-id=\"63b4f5c\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\tBut the method required a lot of queries to create an adversarial example on a single input. For instance, it took millions of tries to convert the image of a bagel into an adversarial example that an AI model would classify as a \u201cgrand piano.\u201d The limitation made the process both slow and costly. Online image recognition platforms usually charge around $1.00 per every thousand queries, raising the price of every adversarial example to thousands of dollars.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-14c0872 elementor-widget elementor-widget-text-editor\" data-id=\"14c0872\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\tIn their new method, called\u00a0<a href=\"https:\/\/arxiv.org\/abs\/1805.11770\" rel=\"noopener\">AutoZOOM<\/a>, the researchers at MIT-IBM Watson AI Lab have managed to dramatically reduce the number of queries required to develop an adversarial example.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-67661a2 elementor-widget elementor-widget-text-editor\" data-id=\"67661a2\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\tIn the case of the bagel image, AutoZOOM was able to generate the adversarial image with approximately 200,000 queries as opposed to 1.16 million queries required by the previous method. The researchers tested AutoZOOM on the CIFAR and MNIST image recognition datasets, and in most cases they were able to reduce the number of queries by over 90 percent.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-f8cd431 elementor-widget elementor-widget-image\" data-id=\"f8cd431\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" src=\"https:\/\/i2.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2019\/02\/ai-adversarial-example-bagel-grand-piano.png?fit=696%2C319&#038;ssl=1\" alt=\"\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-1addce9 elementor-widget elementor-widget-text-editor\" data-id=\"1addce9\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p style=\"text-align: center;\"><span style=\"font-size: 11px;\">Using the AutoZOOM method, researchers at MIT-IBM AI Lab were able to dramatically reduce the number of queries required to generate adversarial examples. (Source: Arxiv.org)<\/span><\/p>\nThe details of AutoZOOM are a bit complicated, but basically the method uses gradient estimations between changes to inputs and outputs to optimize the process of creating adversarial noise. The method also introduces a resizing technique that obviates the need for perturbing every single pixel individually and can make manipulations in batches of pixels.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-13fdde2 elementor-widget elementor-widget-text-editor\" data-id=\"13fdde2\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\u201cWe estimate gradients in an efficient way, and then we reduce the input dimensions such that the attacker doesn\u2019t need to spend so many queries to figure out what is the best direction,\u201d Chen says.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-b42e9f7 elementor-widget elementor-widget-text-editor\" data-id=\"b42e9f7\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\tSince AutoZOOM treats the AI model as a black box, is model-agnostic, which means it works with neural networks can also be employed to test other types of AI architectures such as support vector machines or regression models.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-36efa72 elementor-widget elementor-widget-text-editor\" data-id=\"36efa72\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\tMethods such as AutoZOOM will make it possible to evaluate the robustness of AI models before deploying them. But like most tools that are used to test the security software, malicious actors can also use AutoZOOM to stage adversarial attacks more efficiently.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-b3507e6 elementor-widget elementor-widget-text-editor\" data-id=\"b3507e6\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\u201cActors who really want to generate adversarial examples for malicious behavior might find this technique useful,\u201d Chen says.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-8717ca0 elementor-widget elementor-widget-heading\" data-id=\"8717ca0\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\"><h2>Verifying the robustness of neural networks against adversarial examples<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-5a8698d elementor-widget elementor-widget-image\" data-id=\"5a8698d\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" src=\"https:\/\/i2.wp.com\/bdtechtalks.com\/wp-content\/uploads\/2018\/12\/artificial-intelligence-deep-learning-neural-networks-ai.jpg?fit=696%2C387&#038;ssl=1\" alt=\"\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-b192b78 elementor-widget elementor-widget-text-editor\" data-id=\"b192b78\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\tThe second part of the work done by Chen and his colleagues revolves around creating benchmarks that can measure the resilience of neural networks against adversarial examples.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-ff7ea8a elementor-widget elementor-widget-text-editor\" data-id=\"ff7ea8a\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\u201cHere we want to tell the developer and the user how resistant their neural network and AI model is to adversarial attacks,\u201d Chen says.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-f37eeb7 elementor-widget elementor-widget-text-editor\" data-id=\"f37eeb7\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\tCalled\u00a0<a href=\"https:\/\/arxiv.org\/abs\/1811.12395\" target=\"_blank\" rel=\"noopener noreferrer\">CNN-Cert<\/a>, the method probes convolutional neural networks (CNN) to find their threshold of resistance against perturbations. CNNs are currently the most complicated and advanced type of neural networks and are used in various field such as self-driving cars, medical imaging, facial recognition and speech recognition.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-83aec11 elementor-widget elementor-widget-text-editor\" data-id=\"83aec11\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\u201cWhat\u2019s special about this paper is that the certification method has been optimized for convolutional neural networks. Previous works were focused on simpler neural network models such as multilayer perceptrons,\u201d Chen says.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-9883e53 elementor-widget elementor-widget-text-editor\" data-id=\"9883e53\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\tContrary to the AutoZOOM method, CNN-Cert requires full visibility into the structure of a neural network. The method uses mathematical techniques to define thresholds on the input-output relationships of each layer and each neuron. This enables it to determine how changes to the input in different ranges will affect the outputs of each unit and layer.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-47fb534 elementor-widget elementor-widget-text-editor\" data-id=\"47fb534\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\tCNN-Cert first executes the process on single neurons and layers and then propagates it across the network. The final result is a threshold value that determines the amount of perturbations the network can resist before its output values become erroneous.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-e117227 elementor-widget elementor-widget-text-editor\" data-id=\"e117227\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\tThis is important because adversarial attacks basically play on these boundaries by changing input values in ways to manipulate the prediction output values of neural networks, Chen explains.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-e725048 elementor-widget elementor-widget-text-editor\" data-id=\"e725048\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\u201cIf we can put intervals on these input vectors and allow these intervals to propagate through the layers we define, we can figure out how perturbations in the input will look like in the output, and when we establish a range, we can also give guarantees on the performance on the model,\u201d Chen says.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-e604877 elementor-widget elementor-widget-text-editor\" data-id=\"e604877\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\tThe certification is input specific, which means CNN-Cert must be applied individually to different images.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-0d26ec3 elementor-widget elementor-widget-text-editor\" data-id=\"0d26ec3\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\u201cSome images that are easier to manipulate, while others are harder. So we can\u2019t make a binary decision on whether a model is robust or not. We try to provide a certificate for each input data and how confident your model is in terms of the prediction results for that data,\u201d Chen says.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-1c08d1e elementor-widget elementor-widget-text-editor\" data-id=\"1c08d1e\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\tThe goal of CNN-Cert is to provide a certification, a label of robustness that will tell you the level of trust you can put into your AI model on different types of input. CNN-Cert is independent on the adversarial attack algorithm, so it can be applied to existing attacks as well as unseen and stronger attacks in the future.\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-8bf9013 elementor-widget elementor-widget-text-editor\" data-id=\"8bf9013\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\tChen hopes that in the future, methods such as CNN-Cert can help establish standards that AI models must meet before being deployed. This is especially important in fields such as self-driving cars and healthcare, where an unreliable AI model can have dire consequences on the lives of people.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-9da1c01 elementor-widget elementor-widget-text-editor\" data-id=\"9da1c01\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\u201cWe are deploying AI models in critical situations, we have high expectations from these AI models,\u201d Chen says. \u201cSo we want them not just to be accurate, but also robust. Robustness is very important not just because there\u2019s room for adversaries to manipulate AI models, but also because when AI models are deployed in the field, they\u2019re not functioning in their ideal training environment. They will encounter things they haven\u2019t seen before. We must make sure they can generalize to new things they haven\u2019t seen before in their training data. They have to be robust to perturbations from the environment as well as adversaries.\u201d\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-f1b2d7c elementor-widget elementor-widget-video\" data-id=\"f1b2d7c\" data-element_type=\"widget\" data-e-type=\"widget\" data-settings=\"{&quot;youtube_url&quot;:&quot;https:\\\/\\\/www.youtube.com\\\/embed\\\/Sz9I_zE4jto?version=3&amp;rel=1&amp;fs=1&amp;autohide=2&amp;showsearch=0&amp;showinfo=1&amp;iv_load_policy=1&amp;wmode=transparent&quot;,&quot;video_type&quot;:&quot;youtube&quot;,&quot;controls&quot;:&quot;yes&quot;}\" data-widget_type=\"video.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"elementor-wrapper elementor-open-inline\">\n\t\t\t<div class=\"elementor-video\"><\/div>\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"<p>We&rsquo;ve already seen some of the threats manifest themselves in various ways, including&nbsp;biased algorithms,&nbsp;AI-based forgery and the spread of fake news during important events such as elections. The past few years have seen the development of a growing discussion around building trust in artificial intelligence and creating safeguards that prevent abuse and malicious behavior of AI models. The various efforts are focused in three fields of fairness,&nbsp;explainability&nbsp;and robustness. &nbsp;It&rsquo;s important to create robust AI models and to evaluate the resilience of artificial intelligence algorithms against abuse and erratic behavior.&nbsp;<\/p>\n","protected":false},"author":109,"featured_media":2611,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"content-type":"","footnotes":""},"categories":[183],"tags":[97],"ppma_author":[1946],"class_list":["post-2044","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-ml","tag-artificial-intelligence"],"authors":[{"term_id":1946,"user_id":109,"is_guest":0,"slug":"ben-dickson","display_name":"Ben Dickson","avatar_url":"https:\/\/www.experfy.com\/blog\/wp-content\/uploads\/2020\/04\/medium_8aaf6bea-c4c1-455f-8156-8007d70910f8-150x150.jpg","user_url":"https:\/\/bdtechtalks.com\/","last_name":"Dickson","first_name":"Ben","job_title":"","description":"Ben Dickson is an experienced software engineer and tech blogger. He contributes regularly to major tech websites such as the Next Web, the Daily Dot, PCMag.com, Cointelegraph, VentureBeat, International Business Times UK, and The Huffington Post."}],"_links":{"self":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/2044","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/users\/109"}],"replies":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/comments?post=2044"}],"version-history":[{"count":4,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/2044\/revisions"}],"predecessor-version":[{"id":36249,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/2044\/revisions\/36249"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media\/2611"}],"wp:attachment":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media?parent=2044"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/categories?post=2044"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/tags?post=2044"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=2044"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}