Artificial Intelligence (AI) and machine learning experts are warning of the risk of data poisoning attacks that could work against large-scale datasets commonly used to train learning models. deep in many AI services.
Data poisoning occurs when an attacker forges the training data used to generate deep learning model. This action is meant to influence decisions AI makes in a way that is hard to track.
Also: These experts are racing to protect AI from hackers. Time is running out.
By secretly changing the source information used for training machine learning algorithmsdata poisoning attacks have the potential to be extremely powerful as AI will learn from incorrect data and can make ‘wrong’ decisions that cause have significant consequences.
There is currently no evidence of real-world attacks involving poisoning of web-scale datasets. But now, a team of AI and machine learning researchers from Google, ETH Zurich, NVIDIA and Robust Intelligence say they have demonstrated the ability to attack with poison that malicious examples “guaranteed” will appear in the web-scale dataset used to train the largest machine learning models.
Although large deep learning models are resilient to random noise, even a very small amount of adversarial noise in the training sets (i.e. an attack) poisoning) is enough to introduce intentional errors in the model’s behavior.
Using techniques they devised to exploit how the dataset works, the researchers say, they were able to poison 0.01% of the prominent deep learning dataset with little effort, the researchers say. and low cost. Although 0.01% is unlike many datasets, the researchers warn that it is “enough to poison a model”.
This attack is known as ‘split view poisoning’. If an attacker can gain control of a web resource indexed by a particular dataset, they can poison the collected data, rendering it inaccurate, potentially negatively affecting the data collection. pole to the whole algorithm.
One way that attackers can achieve this goal is to simply purchase expired domains. Domains expire frequently and can then be purchased by others — this is the perfect opportunity for a data poisoner.
The adversary does not need to know the exact time at which a customer will download a resource in the future: by owning the domain, the adversary ensures that any future downloads will be, the researchers said. tainted data collection”.
Also: ChatGPT and more: What AI Chatbots Mean for the Future of Cybersecurity
Researchers point out that buying a domain name and exploiting it for malicious purposes not a new idea — cybercriminals use it to help spread malware. But attackers with different intentions have the ability to poison an extensive dataset.
Furthermore, the researchers detailed a second type of attack, which they call front-runner poisoning.
In this case, the attacker doesn’t have full control over the particular dataset — but they can accurately predict when web resources will be accessed to include a dataset snapshot. With this knowledge, an attacker can poison the dataset just before the information is collected.
Even if the information reverts to its original, unmanipulated form after just a few minutes, the dataset will still be inaccurate in the snapshot taken when the malicious attack was in progress.
One resource that relies heavily on sourcing machine learning training data is Wikipedia. But the nature of Wikipedia means that anyone can edit it — and according to the researchers, an attacker “can poison a training set derived from Wikipedia by making edits malice”.
The Wikipedia dataset doesn’t rely on the active page, but a snapshot taken at a specific time — meaning that attackers with the right timing could maliciously edit the page and force If the data collection model is incorrect, this data will be permanently stored in the dataset.
“An attacker who can predict when a Wikipedia page will be deleted to include in the next snapshot can do the poisoning right before the deletion. Even if the revision is quickly reverted on the active page dynamic, snapshot will contain malicious content — forever,” the researchers wrote.
The way Wikipedia uses a well-documented protocol to create snapshots means that the snapshot time of each article can be predicted with great accuracy. The researchers suggest that this protocol can be exploited to poison Wikipedia pages with a success rate of 6.5%.
That percentage doesn’t sound like a lot, but the large number of Wikipedia pages and the way they’re used to train machine learning datasets means it’s possible to feed inaccurate information to the models.
Also: The best password manager
The researchers note that they do not edit any Wikipedia pages directly, and that they have informed Wikipedia of the attacks and potential measures to combat them as part of a responsible disclosure process. duty. ZDNET has reached out to Wikipedia for comment.
The researchers also note that the purpose of publishing the paper is to encourage others in the security field to conduct their own research on how to protect AI and machine learning systems from attacks. dangerous work.
“Our work is just the starting point for the community to develop a better understanding of the risks involved in creating models from web-scale data,” the paper said.
