Today's AI/ML headlines are brought to you by ThreatPerspective

Digital Event Horizon

A Canary in the Coal Mine: The Looming Threat of AI Data Leaks


A shocking new study has revealed the alarming lack of privacy and security measures in place to protect sensitive data in AI systems. By planting a massive 2 million digital "canaries" into the AI supply chain, researchers discovered that at least three AI providers were compromised by attackers who used their access to siphon prompts, and multiple resellers were found to be logging and passing along sensitive data. The study highlights the need for more robust security measures in the AI ecosystem.

  • Researchers planted 2 million digital "canaries" into the AI supply chain to test its security.
  • AI providers were found to have inadequate privacy and security measures in place.
  • At least three AI providers were compromised, with attackers siphoning sensitive information from prompts.
  • Multiple cases of AI providers violating end-user privacy settings, including Zero Data Retention, were discovered.
  • Some providers were scraping secret URLs from prompts, violating privacy terms.
  • Researchers suggest hosting models on-premises servers as the most effective means of preventing leakage.


  • In a shocking exposé, a team of researchers has revealed the shocking truth about the security of AI systems. By planting a massive 2 million digital "canaries" into the AI supply chain, the researchers discovered a shocking lack of privacy and security measures in place to protect sensitive data. The experiment, designed to test the limits of AI data leakage, has left many in the industry reeling and searching for answers.

    The researchers, led by HD Moore, used a collection of AWS access keys, empty accounts, passwords, and unique URLs to test the security of various AI providers. The goal was to see if sensitive information could leak through the vast network of AI services and how it would be exploited if it did. The results were nothing short of alarming.

    At least three AI providers were compromised by attackers who used their access to siphon prompts, which contained sensitive information. Multiple resellers in the inference marketplace were found to be logging and passing along prompts, allowing attackers to steal sensitive data. In addition, multiple cases of AI providers not following end-user privacy settings, including Zero Data Retention, were discovered.

    The experiment also revealed that some providers were scraping secret URLs put into prompts, in violation of privacy terms. This was found to be the case for a Chinese AI lab, which fetched secret URLs from more than half of the prompts sent to its models. The lab's systems would typically show up two or three days after the request, highlighting the need for more robust security measures.

    The researcher's findings have sparked concerns about the lack of privacy and security in the AI ecosystem. "We sent nearly two million AI prompts containing unique, trackable secrets to AI providers, then watched where those secrets surfaced," Moore wrote. "What we found revealed security breaches, prompt theft, and widespread failures to protect customer data."

    Moore suggests that the most effective means for preventing leakage is to host models using on-premises servers. However, this option is expensive, and for those who can't afford it, the next best option is to receive service directly from a first-tier provider that provides "one contract and a privacy policy to read."

    The researchers' experiment has left many in the industry searching for answers and wondering how to prevent these types of breaches in the future. As AI continues to grow in power and importance, it's clear that security and privacy will be at the forefront of our minds.



    Related Information:
  • https://www.digitaleventhorizon.com/articles/A-Canary-in-the-Coal-Mine-The-Looming-Threat-of-AI-Data-Leaks-deh.shtml

  • https://arstechnica.com/security/2026/11/a-leaky-ai-supply-chain-is-exposing-secret-user-data-at-an-alarming-rate/


  • Published: Fri Nov 20 15:42:49 2026 by llama3.2 3B Q4_K_M











    © Digital Event Horizon . All rights reserved.

    Privacy | Terms of Use | Contact Us