Reading view
Google releases new privacy controls for activity history, personalization
Open-source security is posing challenges governments can’t easily solve
An epidemic of cyberattacks on open-source software has mounted in recent months, making clear how uniquely difficult it is to protect the publicly available code, from both a policy and a technical perspective, that serves as the foundation for so much of the digital world.
While open-source software security got a boost in attention under President Joe Biden — whose administration grappled with the fallout from the potentially catastrophic Log4j flaw that emerged in 2021 — a number of open-source experts say that government protection efforts have suffered setbacks under President Donald Trump. Many also say companies that heavily rely on open-source software, which is basically all of them, haven’t shouldered enough of the responsibility for safeguarding it.
“What we’re seeing is years of lack of investment sustainment in open-source software that is finally starting to catch up to us, where it seems like every week there’s a new supply chain compromise,” said Jack Cable, who held a role at the Cybersecurity and Infrastructure Security Agency where he worked on open-source security before departing under Trump.
The advancements of frontier artificial intelligence models stand to exacerbate the risk further, while simultaneously illustrating what makes defending open source difficult: Project Glasswing said shortly after its announcement that it had uncovered 6,202 high- or critical-severity vulnerabilities in a scan of more than 1,000 open-source projects, but that it had disclosed only 502 of them to open-source project maintainers and only 75 had been patched as of May 22 (albeit some due to typical patching lagtimes).
At the same time, there are questions about how much the government can help, even as overseas governments seek to focus on open-source security.
The evolution of open-source risk
There are a series of factors contributing to the current threat to open-source software, experts say.
One is simply that attackers go to the area where they can get the highest return on their work. Compromising open-source software gives them the chance to get into the supply chain and exploit additional targets.
“Twenty years ago, open source was still fairly niche,” said Æva Black, who also worked on open-source security at CISA but left when Trump came back into power. “The potential blast radius if you managed to compromise open source was relatively small, because back then the world didn’t run on open source. Now almost everything runs on open source,” she said, from modern cars to satellites.
Another part is the nature of open-source software itself.
“It’s a symptom [of having] lots of open source [that] is a little bit under-maintained or not cared for enough, so that we spend too little effort and money and infrastructure on them,” said Daniel Stenberg, who is the creator and maintainer of cURL, a popular open-source project. “Lots of open source is being maintained by small teams, lots of volunteers, and I think that that’s a tough situation.”
That doesn’t mean the maintainers are to blame, Stenberg said. The companies that rely on open-source need to be diligent about using it, Black said.
“What we’re seeing in that realm right now is not new; it is more advanced and far more widespread,” she said. “The problem remains that companies who use open source — because open source is by far the most efficient way to collaborate on non-product value features — most companies are not implementing a responsible and safe utilization pathway.”
Open-source projects lack a systematic way to handle coordinated vulnerability disclosures, unlike companies or industry groups with formal processes, said Dan Lorenc, CEO and co-founder of Chainguard. Project maintainers sometimes aren’t reachable, and those who are available are flooded with reports, many of them unverified findings from AI tools that waste their time without adding value..
Of course, some of those vulnerability reports turn out to be legitimate. “Mythos and AI models have contributed to an uptick in the number of vulnerabilities and things that we’re able to find” in open-source software, said Alex Zenla, chief technology officer for the cybersecurity company Edera.
All of that leaves more room for companies, non-profits and world governments to improve open-source security.
A moment of momentum
While open-source software security isn’t a new issue, the 2021 discovery of the Log4j flaw sounded alarms within the cybersecurity community. Jen Easterly, then the director of CISA, called it “one of the most serious I’ve seen in my entire career, if not the most serious,” with the potential to affect hundreds of millions of devices given the ubiquitous nature of the popular open-source logging library.
A year later, the Cyber Safety Review Board released its report on the incident, concluding that swift action from industry and government averted a disaster. But the incident “called attention to security risks unique to the thinly-resourced, volunteer-based open source community,” it wrote. “This community is not adequately resourced to ensure that code is developed pursuant to industry-recognized secure coding practices and audited by experts.”
The U.S. government actions after included some steps focused specifically on open-source software such as creation of the Open-Source Software Security Initiative and hires of well-regarded open-source security experts at CISA such as Black, but also some steps that could be applied more generally and still help with open-source security, such as greater promotion of secure-by-design, memory-safe languages and software bills of materials (SBOMs).
Some of the Biden administration work on open-source security started before Log4j, such as provisions from an executive order he issued in 2021 that directed CISA along with the Office of Management and Budget and General Services Administration to issue guidance to agencies.
The administration’s 2023 cybersecurity strategy also stepped into the long, thorny discussions over software liability, with a mention of open-source security: “Responsibility must be placed on the stakeholders most capable of taking action to prevent bad outcomes, not on the end-users that often bear the consequences of insecure software nor on the open-source developer of a component that is integrated into a commercial product.“ The Biden administration always indicated that addressing software liability would take a prolonged battle ahead.
Under Trump, many of the Biden administration’s efforts have languished. CISA’s splashy hires on open-source are gone, including Black, Tim Pepper and Anjana Rajan. Also departed are leading figures on secure-by-design and SBOMs, with CISA personnel cutbacks slicing deep.
No one has seen any sign that the national cyber director-led Open-Source Software Security Initiative is active, with few participants remaining in government today. The Trump administration cyber strategy doesn’t mention open-source.
“The loss of open-source experts at CISA “is unfortunate, and it will be hard for the government to try to rebuild capacity, but I do think now more than ever CISA has a core role to play to secure open source software,” Cable said.
The pressure is mounting
It’s not that the issue is getting zero attention from those in a position to make a difference. Nick Andersen, the acting director of CISA, said last month that open-source security was an area of particular concern for him.
Andersen responded to concerns about CISA staffing levels on open-source security and spoke more broadly on the topic in a statement to CyberScoop.
“As artificial intelligence and other technologies have the power to transform how vulnerabilities are discovered and exploited, CISA recognizes that the open source software (OSS) that underpins much of the nation’s critical infrastructure will need to be hardened,” he said. “CISA actively collaborates with our partners on shared priorities, including OSS security, to ensure time and resources are spent where they matter the most. We have an immensely talented team, but are also accelerating our hiring in critical areas, to strengthen the nation’s defenses against cyber threats.”
The Office of the National Cyber Director did not respond to requests for comment.
There’s been some activity on Capitol Hill, too. The Securing Open Source Software Act, which Cable worked on during a stint as a Senate staffer, would direct CISA and other agencies to take actions to mitigate open-source software security risks, but the legislation has stalled since its introduction in 2022. A portion of the bill, however, was included in the Department of Homeland Security funding law Trump signed in April, directing CISA to brief Congress on the value of establishing something like an open source program office, which some companies use to manage open source within a given firm.
Senate Intelligence Committee Chairman Tom Cotton, R-Ark., has pushed the executive branch to improve its awareness of foreign adversaries playing roles in open-source software used by national security-focused agencies.
The annual defense policy bill in the House calls on the Defense Department’s chief information officer to report to Congress on a plan to secure open-source software supply chains, saying lawmakers are “concerned that the Department lacks sufficient visibility into the origins, maintenance, and security of OSS applications and software dependencies.”
That defense authorization bill language is “really beneficial, and I think it signals acknowledgement of this changing of culture” around open-source security risks, said Hayden Smith, founder of HuntedLabs, whose company won a contract with the Space Development Agency on supply chain security — agency work that the defense bill singled out.
“The report language is the first time the Hill is trying to get a true handle on foreign influence in open source code where they have oversight,” he said, saying it was a “piece of the puzzle” along with Cotton’s letter and a memo from Secretary of Defense Pete Hegseth last year about foreign influence in the Pentagon supply chain. “It’s good and would trickle down into everyone who provides software to the department.”
Zenla, though, believes trying to isolate China from open-source systems isn’t in and of itself a good idea.
“I don’t think that that makes a lot of sense, because they’re actually pretty good things that people contribute to open source,” she said. “Not everyone is malicious, and what are we going to do, spy on every single open source maintainer?” It’s more about doing things like making sure that highly-classified systems are set up in a separate way, she said.
Europe is also taking action to secure open-source software that the United States doesn’t seem ready or willing to do right now. Germany, for instance, devotes grants to the security of open-source projects, although Stenberg pointed out that sometimes money doesn’t equate to maintainers being able to fix flaws more quickly, depending on the project’s size.
The Cyber Resilience Act (CRA) adopted by the Council of the European Union in 2024 could offer another road on open-source security. The CRA requires those who use open-source software products as part of any commercial activity to take certain security measures.
Black said that when she was at CISA, there were discussions between the agency and European counterparts about finding compatible ideas on open-source security, but that momentum died with the Trump administration.
But “Europe kept rolling, and now has in place a new legal framework that is set to really reshape open-source security for potentially the whole world, but certainly for anyone who wants to work with Europe on open source,” she said.
Lorenc recently wrote that “open source isn’t governable.” He said an organization like a neutral nonprofit, possibly using some government funding, should take responsibility for things like coordinating vulnerability disclosure into one pipeline. He also said there needs to be one authority in charge of “forking” — that is, taking a project and assigning stewardship elsewhere — when a maintainer isn’t responsive to vulnerabilities.
There are differing opinions on how much past government warnings, advisories and guidance have helped. Smith gave some credit to government agencies that “have all responded to open source attacks using the means they have.”
Stenberg said that “I don’t think they make any big dent at all in the big scheme of things.” They might get some attention initially, “then two years later we all forgot about them, and they actually didn’t change much.”
Ideally, everyone could get on the same page, Zenla said. “The best way to do this is if people actually collaborated on a global scale on some sort of regulation around this, but that seems nearly impossible at the current moment,” she said. (The United Nations’ Open Source Week runs all this week.)
But if there’s an upside to the spate of attacks on open-source software, it’s the energy it gives to how better to secure it, Lorenc said, invoking the political saying to never let a good crisis go to waste.
“Everyone knows the industry has to change,” he said. “This is a really good crisis, and the right things are happening in the right places, and organizations are rethinking their culture around software development, and they know what they have to do. It’s just something that’s never been top of the priority list for the last 10 years. Now it is, and they’re doing it, and it’s, ‘Can we do it fast enough?’”
The post Open-source security is posing challenges governments can’t easily solve appeared first on CyberScoop.
Cloudflare teams up with big browsers to help websites tell welcome from unwelcome visitors
How software development’s speed obsession enabled TeamPCP’s chaos crusade
TeamPCP is on a rampage through open-source software.
In less than four months, the threat actor has compromised and injected malicious code into more than 1,000 software packages. The extraordinary spree has transformed how software developers and maintainers distribute and manage their code, as their dependencies and repositories have become one of the most effective and prevalent attack vectors this year.
While there has been a host of technical exploits, TeamPCP’s greatest attack has been the uprooting of trust — repeatedly proving that most organizations fail to verify the code they ingest into their systems is legitimate, abusing a nearly blind faith that much of the software development industry relies on to power today’s modern economy.
Starting with Trivy in February, TeamPCP’s attacks have shaken that trust many times over.
The scale of TeamPCP’s attacks lies partly in the automated systems companies use to deploy code, like CI/CD pipelines. It is also capitalizing on new security gaps created by developers’ increasing reliance on AI. Yet, with relatively low effort and unoriginal tactics, TeamPCP is wrecking open-source frameworks and underlying systems at levels the technology community has rarely reckoned with.
“Developers didn’t do a great job of analyzing the security of their open-source dependencies before but, now with AI, there’s in some cases virtually no human in the loop or any kind of sanity check on what these tools are doing,” Feross Aboukhadijeh, founder and CEO at Socket, told CyberScoop.
“You have agents installing packages that haven’t been vetted,” he said. “When an attacker gets in, the impact is even broader because there’s less checks and balances to stop it from affecting everybody.”
TeamPCP hasn’t identified a new problem or proved anything novel. The crux of these attacks hinge on a central theme — defensive vulnerabilities the entire software industry has known about for years. Researchers and developers know the open source trust model is broken and susceptible to sabotage. Yet, the software industry has not fixed this problem.
“The speed and scale of these attacks is what makes it most notable, not necessarily the methodology behind it, because at the core it is really about exploiting third-party trusts that we have,” said Kimberly Goody, senior manager at Google Threat Intelligence Group.
Software packages are typically subjected to intensive security monitoring to test for vulnerabilities and poisoned updates before they are released to live environments.
Yet, the real vulnerability highlighted by TeamPCP lies further up the chain of command with the organizations or individuals that publish these packages to the wider market, according to Nathaniel Quist, manager of cloud threat intelligence at Palo Alto Networks.
“It is their responsibility to secure their credentials and not provide a jump off point to trigger a supply-chain event,” he said. “Everything that interacts with or crosses through that zone must be highly monitored and controlled to ensure a compromise can be contained quickly and easily.”
TeamPCP’s motivation
TeamPCP, like any prolific cybercriminal, has captured significant attention from threat hunters since it emerged in late 2025. Google attributes the activity to one core operator.
The company said it traced TeamPCP’s residential and mobile IP address connections to South Africa, indicating the primary operator was located there during at least some of its attacks.
“We don’t believe that there’s an established core group, at least not yet, and that a lot of this has been conducted by an individual,” Goody said. Google declined to name the core operator or confirm it knows the person’s true identity.
Palo Alto Networks said the core manager of TeamPCP uses the “ResoluteXBF” handle on multiple platforms. The cybersecurity firm is also tracking two additional core members: “diencracked” and “Shinigami.”
If TeamPCP is primarily run by one person, law enforcement has a rare opportunity to make a lasting impact with a single arrest.
TeamPCP has collaborated with other cybercriminals, but most of those partnerships were short-lived and ended in a public feud or otherwise failed to get off the ground in any meaningful way, Goody said.
Researchers have linked TeamPCP to extortion crews, dark web forums and affiliates including Lapsus$, ShinyHunters, Vect, DragonForce, BreachForums and “HasanBroker.” TeamPCP listed about 4,000 private code repositories on a dark web forum with an asking price of $95,000.
The actions to date, including unpredictable behavior, indicate motivations beyond financial gain and a “clear desire for notoriety,” Goody said. “They seem to like to make chaos.”
Quist draws the same conclusion from his months-long investigation, noting that it encourages other cybercriminals to get in on the action, at one point offering financial rewards for the largest software supply-chain attack.
TeamPCP isn’t in the game for extortion payments, he said. “These actors are more interested in the underground street cred they are gaining” and “causing as much damage and mayhem as possible.”
Victims abound, but exposure limited
TeamPCP has been remarkably noisy, opportunistically injecting malware into open-source software for the purpose of stealing credentials for Kubernetes environments, Amazon Web Services, Microsoft Azure, Google Cloud and many other connected services.
The group’s claimed victim list is staggering: Checkmarx, Bitwarden, LiteLLM, Telnyx, Mercor AI, PyTorch Lightning, AntV, SAP, GitHub, TanStack, UiPath, MistralAI, Microsoft DurableTask, Red Hat and Nx Console.
The full collection of packages compromised or poisoned by TeamPCP to date accounts for roughly 500 million weekly downloads combined, according to Quist.
While the breadth of potential downstream compromise flowing from those downloads is substantial, many endpoints infected with those malware-riddled packages aren’t exposed to the internet and less susceptible to attack, he added.
“I don’t think there’s going to be a very extremely large number of victims,” Quist said. “There’s going to be a lot of people who potentially could be compromised and have potentially vulnerable packages in their environment, but that doesn’t necessarily mean they’re in an exploitable position.”
While these incidents have grabbed headlines, TeamPCP hasn’t accumulated payouts nearly as large as other cybercriminals. The broader reputational impact it has wrought, however, is massive.
TeamPCP has publicly claimed more than 10,000 victims and about $90,000 in extortions, according to Quist.
“They might not be making a lot of money, but they are causing a lot of impact,” Goody said. “Their campaigns have been very disruptive.”
How TeamPCP’s operating model targets development
TeamPCP’s victim list has grown as its hijacked open-source repositories on npm, PyPI, GitHub and other outsourced developer tools that are incorporated into upstream code running in production environments.
Developer laptops and other endpoints that are assigned to install, build and publish software widely contain keys and access to source code that create incredibly valuable supply-chain targets for attackers, Amitai Cohen, head of the attack vector intel team at Wiz, explained during a June presentation on TeamPCP at SleuthCon in Arlington, Va.
The group targets CI runners, which are automated systems that build, test, and publish code. TeamPCP injects malware into the code repositories these runners maintain. When other developers pull that code into their own systems, they unknowingly download the malware alongside it.
Some of these artifacts, including Python libraries, npm registries and GitHub Actions, are downloaded almost immediately by thousands or millions of developers who’ve set their runners up to consistently pull the latest version, according to Cohen. “We as a security industry have taught them that that is the right thing to do. You want to use the latest version because you want to be protected against vulnerabilities, and obviously you want to benefit from all the latest features.”
That instinct is exactly what TeamPCP exploits. By compromising one company’s CI/CD workflow, the group gains access to every downstream user who automatically pulls that infected code. “This is what allows [TeamPCP] to leverage initial access to some patient zero, some company that had a vulnerability in their CI/CD workflow, in order to gain access to their downstream users,” Cohen said. “That’s just how the software supply chain works. Everything has dependencies upon dependencies upon dependencies.”
Some of the packages compromised by TeamPCP were live for almost 13 hours, but security practitioners have responded by identifying code-injection attacks much quicker now, pulling some compromised repositories within 15 minutes, said Ben Read, director of strategic intelligence at Wiz.
The threat group’s operations remain high-tempo. TeamPCP infects new software packages almost daily, validates compromises and captures sensitive data within 24 hours, according to Wiz researchers.
The threat group has consistently evolved its tactics, developing payloads in JavaScript and Python while spreading from local files to Kubernetes application programming interfaces and bundled software development kits. Most recently, it’s been stealing credentials via custom protocols.
The group’s ambitions have expanded beyond its own attacks. TeamPCP is also responsible for a self-replicating piece of malware known as Mini Shai-Hulud, which infected hundreds of software packages across open-source registries in back-to-back attack sprees last month. A TeamPCP affiliate published the full source code for the malware on GitHub last month and encouraged other cybercriminals to use it for their own campaigns.
“TeamPCP is going for volume. They are not being discriminating, they’re not necessarily trying to be stealthy or trying to maximize ROI. They’re going for an all-of-the-above strategy,” Read said during the Sleuthcon presentation.
Defensive gaps create openings for attack
TeamPCP’s attack spree has also underscored how difficult it is for organizations to revoke compromised secrets. Multiple victims have experienced recurring infections, sometimes falling prey to TeamPCP three times within a month, because they didn’t rotate secrets properly, Cohen said.
At its core, these attacks highlight a direct trade-off organizations accept when they update software quickly to fix vulnerabilities, but learn that doing so too quickly could expose them to illegitimate registries containing malware.
TeamPCP has targeted what Aboukhadijeh describes as a “public good,” open-source registries that were never perfect but widely trusted and rarely turned into a point of entry for supply-chain attacks.
Rapid open source software installation is one of the most dangerous things an organization can do right now, he said, adding that there’s a roughly 1 in 10 chance that any package installed by an organization could trigger an active attack.
TeamPCP has compromised security scanners, password managers, automation tools, data visualization software, and CI/CD infrastructure across various environments.
And it’s lifted a trove of credentials and other sensitive data from victims.
Researchers like Cohen at Wiz, who have been tracking this attack spree since the beginning, are nearing a breaking point.
“This is also too hard on us. We’re very tired. I’m sure a lot of people working on this problem space are very tired, and it’s just kind of become untenable,” Cohen said.
“You can’t keep existing in a world where you wake up every morning and some super prevalent package is compromised and everybody’s just going to be using it like nothing,” he added. “We need to start taking this a bit more seriously.”
The post How software development’s speed obsession enabled TeamPCP’s chaos crusade appeared first on CyberScoop.
Anthropic rolls out Claude Fable 5, but it's available for a limited time
Windows 11 KB5094126 & KB5093998 cumulative updates released
Everybody Is Vibe Coding But Nobody Told the Security Team
AI-driven development is not something organizations can or should block. But it must be governed.
The post Everybody Is Vibe Coding But Nobody Told the Security Team appeared first on SecurityWeek.
Hands on with Intelligent Terminal, an AI-powered Windows Terminal
Brave Software releases Origin for a paid, bloat-free browsing experience
Hola Browser for Windows compromised to deliver cryptominer
Anthropic confirms Claude Mythos-class models will roll out to the public
Anthropic’s restricted Claude Mythos model may be coming to Claude Code
GitHub says internal repositories were impacted in poisoned VS Code extension attack
GitHub said late Tuesday that internal repositories were exfiltrated after an employee device was compromised through a poisoned Visual Studio Code extension, an incident that underscores the growing risks facing software development platforms and the ecosystems built around third-party developer tools.
The Microsoft-owned company said in posts on X that it detected and contained the compromise, removed the malicious extension version, isolated the affected endpoint and began an incident response investigation. The company’s current assessment is that the activity involved GitHub-internal repositories only.
GitHub also said a claim from TeamPCP, a hacking group behind attacks targeting software development packages, that 3,800 repositories were impacted was “directionally consistent” with its investigation so far. It said critical secrets were rotated Tuesday, with the highest-impact credentials prioritized first. The company said it continued to analyze logs, validate secret rotation and monitor for follow-on activity.
The company has not publicly named the extension involved or attributed the activity to a particular group. TeamPCP reportedly advertised the material for sale on a cybercrime forum and threatened to release it if no buyer emerged.
Information surfaced Wednesday that the incident may be related to a separate issue with Nx Console, a Visual Studio Code tool that helps engineering teams organize large codebases, coordinate build pipelines and run tests efficiently. According to a security advisory posted on GitHub, one of the Nx Console maintainers was compromised in a prior security incident that leaked their GitHub credentials. An attack then used those credentials to push a malicious version of the extension to the VS Code Marketplace. Those credentials have since been temporarily revoked.
With millions of installs, Nx Console is a fixture of professional JavaScript development. It is exactly the kind of tool that sits deep inside a developer’s working environment, which would have direct access to source code, credentials and build systems.
NX CEO Jeff Cross posted on X Wednesday that his company has been working with Microsoft to determine the full scope of the incident.
“Initially, Microsoft indicated to us that there were 28 installs of the malicious version 18.95.0. Based on our own analytics for the compromised version, we currently believe the number of users who received the malicious package may be significantly higher; potentially over 6k installs,” the post reads.
“This is my top priority right now,” Cross continued. “Our team has been, and continues to be focused on understanding exactly what happened, helping affected users, hardening our systems and release processes, and being as transparent as possible throughout the investigation.”
The episode also follows a series of supply chain attacks involving npm, PyPI, Docker and other developer ecosystems. In those incidents, attackers have often targeted maintainers, packages or credentials rather than attacking end users directly. The multiple attacks show how fragile development environments have become as threat actors increasingly target them. A single compromised developer account, package, extension or build process can create access to many downstream systems.
GitHub has said it has no evidence that customer data stored outside the affected repositories was affected.
Visual Studio Code extensions are widely used by developers to add functions to Microsoft’s code editor, including support for programming languages, testing tools, cloud services and artificial intelligence assistants. Because these extensions often operate inside development environments, a malicious or compromised extension can be positioned close to source code, credentials and build systems.
“The thing people underestimate about VS Code extensions is that they have full access to everything on the developer’s machine,” Charlie Eriksen, a security researcher at Aikido Security, told CyberScoop. “EDR doesn’t cover this layer at all. What’s missing for most organisations is any kind of visibility into what’s actually running on developer machines and the ability to control it.”
Trojanized extensions have appeared in the VS Code Marketplace before. Security researchers have identified malicious extensions posing as legitimate development tools, including packages used to steal credentials, mine cryptocurrency or exfiltrate data. Some have accumulated large installation counts before removal, reflecting the difficulty of policing open plugin ecosystems at scale.
For GitHub, the breach comes amid broader scrutiny of the security of developer infrastructure. The platform sits at the center of software production for companies, governments, open-source maintainers and independent developers. Its internal systems and code are of obvious interest to attackers because GitHub’s services support code hosting, package distribution, automation and identity workflows across much of the software industry.
GitHub said it would publish a fuller report when the investigation is complete.
Update: May 20, 12:55 p.m.: This story has been updated with information about a related security incident with Nx Console.
The post GitHub says internal repositories were impacted in poisoned VS Code extension attack appeared first on CyberScoop.
Microsoft plans to improve Windows 11 driver quality in 2026
AI might cut false positives, but it won’t stop the slop
As defenders get their hands on newer AI models with more powerful cybersecurity capabilities like Anthropic’s Mythos and OpenAI’s Daybreak, organizations are being told to prepare for a flood of new vulnerability reports.
But for bug bounty programs across the nation, that day may already be here, as yesterday’s frontier models and today’s open-source AI tools have dramatically increased the volume of bug reports flowing into companies around their own products or on larger bounty platforms online.
GitHub, one of the world’s largest online code repositories, said it is tightening its definition of a “complete” bug report after a significant increase in AI-assisted submissions over the past year.
Although the influx has had some benefits, many reports are submitted without proof of concept, are reliant on unrealistic attack scenarios or cover issues already listed as ineligible. As a result, the company is having difficulty separating signal from noise.
“This isn’t unique to GitHub,” wrote Jarom Brown, senior product security engineer at GitHub. “Programs across the industry are grappling with the same challenge, and some have shut down entirely.”
Brown said GitHub does not want to ban the use of AI generated reports entirely, calling it a “force multiplier” for security in the right context. But in a world where it’s never been easier to use AI to generate theoretical bugs, the company wants researchers to go the extra mile to confirm that their discoveries can actually be exploited in real-world conditions.
What we need is the same standard we’ve always expected: validation,” Brown wrote. “An AI-assisted finding that’s been verified, reproduced, and submitted with a working proof of concept is a great submission. An unvalidated output submitted as-is without reproduction or demonstrated impact is not.”
Grant Bourzikas, chief security officer at Cloudflare, said triaging bugs and proving they can be exploited has always been one of the hardest parts of vulnerability research, and AI vulnerability scanners and code have “made it worse.”
For instance, code written in C and C++ programming languages are vulnerable to a range of exploits – like buffer overflows and out-of-bounds reading and writing – that don’t exist in memory safe languages like Rust. AI tools scanning software written in memory unsafe programming languages are far more likely to generate false positives.
But one of the biggest flaws continues to be that AI tools are also designed to give the user what they’re asking for, even when it’s not there. This leads to the generation of bug reports filled with speculation and qualifiers around exploitability that require human follow up.
“That’s a reasonable bias for an exploratory tool,” Bourzikas wrote. “It’s a ruinous one for a triage queue, where every speculative finding spends human attention and tokens to dismiss, and that cost compounds across thousands of findings.”
Cloudflare recently shared results from testing Mythos on 50 of its own code repositories, looking for exploits. Bourzikas called Mythos “a different kind of tool doing a different kind of work” from other frontier models, and that it made significant progress in reducing false positives.
For example, he pointed to two Mythos capabilities that stood out compared to other models: chaining exploits together and generating its own proof-of-concept code to confirm exploitability.
Older models could spot many of the same bugs, but they often couldn’t figure out how to exploit them effectively, or show that the issue could be exploited in real world conditions.
Others have argued that the gap in bug hunting capabilities between newer frontier AI models and older ones, or open source models available today is not as large as advertised.
Swedish software developer Daniel Stenberg, lead developer for curl, an open source file transfer tool used around the world, recently wrote about his experience with Mythos Preview. Like others, he has also seen a higher volume of AI-fueled bug reports over the past year, but said the flood of low-quality reports has tapered off significantly since March as models have improved.
Curl is mature and polished by the standards of most software: Stenberg estimates each line of code has been rewritten or altered at least four times, and he said he has used both human and AI tools in the past to implement hundreds of bug fixes over Curl’s existence.
That makes it a unique testing ground for the enhanced capabilities of Mythos, which was reportedly so powerful at finding vulnerabilities that Anthropic opted not to release it to the general public.
After gaining access to Mythos, Stenberg received the results of a scan of 178,000 lines of curl code. Ultimately, the scan flagged five “confirmed” vulnerabilities. Further exploration by human researchers found that 4 of the bugs were false positives or had no security impact. The one remaining bug Mythos found? A low-severity flaw that will be fixed in a regular June update.
Even as he praised the impact of AI on cybersecurity generally, Stenberg concluded that for all the hype, Mythos is only “a bit better” than previously released models.
“My personal conclusion can however not end up with anything else than that the big hype around this model so far was primarily marketing,” he wrote. “I see no evidence that this setup finds issues to any particular higher or more advanced degree than the other tools have done before Mythos.”
The post AI might cut false positives, but it won’t stop the slop appeared first on CyberScoop.