ClassicConnect

[WinReset]

I'd like to talk about how artificial intelligence is being used as a security auditor for software, such as the recent Claude Mythos Preview announcement where Anthropic stated that Mythos found a 17-year-old vulnerability in FreeBSD that allowed anyone to gain root on a system running NFS. Now, they deem Mythos to be too dangerous to release to the public, and this exact same thing happened to GPT-2 in 2019 where OpenAI deemed it dangerous because it could write a believable fake news article. But this, this is on a completely different scale than GPT-2 was. From what I've heard, Anthropic is holding back Mythos from the public because Mythos has found tons of zero-days across a wide range of software. If this is indeed true, then how will the evolution of AI models such as Mythos affect the cybersecurity sector in a couple years? I'd like to know what you guys think about this.
_________________

[LyraNovaHeart]

Mythos is mostly marketing.

Sure, it can find vulnerbilities and zero days... but that was already possible with both Sonnet 4.5/4.6, and Opus 4.5/4.6/4.7. Even then, claims like "Mythos found 271 vulnerbilities in Firefox" aren't showing you the full picture.

Most people will think that it can just find those on it's own, really, it needs assistance from humans still via Agentic setups. It'd be near impossible by itself especially given how large the codebase of Firefox 150 was.

AI is not and will never be able to replace cybersecurity on it's own, as security is only as strong as it's weakest link. If anything, I think it creates more vulnerbilities than it does solve them.
_________________
I'm one day closer to being who I wanna be~

[epicness]

This is just my two cents as someone who's done a solid amount of cybersecurity work, both real-world as well as in CTFs and other competitions.

NOTE: I will say that personally I am GenAI-averse, but I'm not the "all AI users are the devil" polarized type of GenAI-averse. I'm also keeping the scope of this response to specifically GenAI as it pertains to cybersecurity - ethical, environmental, etc. worries are a different discussion entirely.

There's a lot of different angles to approach this kind of thing. First, Mythos and GenAI security research as a whole. I can definitely say AI has been involved in a lot of security research as of late. I think this especially continues to hold true as researchers continue to figure out how they can implement it effectively (MCP tooling, agents, output validation, etc.) There's so many conflicting details - claims that Mythos only found 1 low-severity bug in curl, while Mozilla made a raving post claiming that Mythos found nearly 300 vulns in their software.

As for the "danger" aspect, they say that they're working on safeguards. GenAI has been notorious for jailbreaks, or safeguards that just straight up don't work. There's an entire article about "deaths linked to chatbots" and threat actors are using AI for malicious tooling every time you turn around, for God's sake. Something tells me, unless they neuter the model, that isn't going to change. If I recall correctly, Anthropic has a program to remove some cyber-reltaed safeguards on their models now, but if we have DPRK IT workers getting jobs at Fortune 500s and major tech companies left and right, who says they can't get into a safeguards removal program?

The FreeBSD NFS bug is effectively a nothingburger in the real world (requires NFS + Kerberos) and, in fact, an identical bug was found and fixed in MIT Kerberos almost 20 years before it was identified in FreeBSD (see: CVE-2007-3999). But it is still interesting to see that it spotted the buggy code. The notable Copy Fail Linux privesc was AI-assisted - a human researcher noted a potential cause of vulnerabilities in the Linux crypto subsystem and then an AI system scanned the code. I still think that AI is not at the point of fully autonomously finding novel vulnerabilities in major software - yes, even with Mythos. There's always gonna need to be a human in the loop, because even with Mythos it's demonstrated that there's still noise and exaggerations all over the place.

I think it's also worth noting that "vibe coding", no matter how good an AI is, is still much more likely to introduce vulnerabilities (especially if you don't know what you're doing.) There's been so many stories of vibe coding introducing vulnerabilities, or just plain doing something stupid. And I think the inherent "high permissions" that people give AI is not going to make this any better. (It's worth noting that this is also a problem in LLM-based live chats - and prompt injection-based attacks are really only something that can be protected against and filtered, not fully patched like other types of injection.)

And besides the technical part, there are other important parts to this too. Bug bounty and vulnerability disclosure programs are getting swamped by AI slop submitted by people looking for a quick buck and it's getting harder and harder for security triage teams to handle. Real AI-assisted vulnerabilities are popping up too, as I stated, but there's a lot more of the slop, and it often looks just real enough to warrant a closer look. I'm pretty sure AI has gained traction on the triage team side as well, and as someone who's nearly had a critical, 5-figure bug nullified by a bad triage, I'm not entirely sure that's a good thing without a trained human in the loop lol.

I think it's also important to note a few things on the career and learning side. The cyber industry has the same problem that most of tech does right now - AI hysteria, mass layoffs due to AI (or AI budgets - who knows?), and a lot of bogus info/training/posts out there. I think one particularly notable thing to talk about is the CTF scene. For those who don't know, capture-the-flag (CTF) is a type of cybersecurity competition that basically involves performing specific related tasks (digital forensics, reverse engineering, web hacking, etc.) to get "flags" (text strings that indicate a challenge solve), submit them for points, and climb to the top of the scoreboard. It has been very important for networking, recruitment, learning, and even just a source of fun for many, and it's something I had been involved in since late 2024 up until about March of 2026. In recent times, agentic AI solvers have taken over CTF and are solving challenges at inhuman speeds - which, arguably, defeats much of the purpose of CTF as a test of skill and a means of learning. I've seen many people bow out as a result of this, including some of the best competitors in the world, and I ended up bowing out eventually as well after an experience at an on-site competition showed me how truly bad CTF had become. I personally do not think AI belongs in a competition with the direct goal of proving yourself and learning along the way, but clearly some people do. I've seen a lot of people saying "CTF is dead" and honestly, I agree - and with its "death" comes a shuttering of a learning opportunity that many aspiring cybersecurity professionals before, including me, gladly participated in.

To sum it up: I think GenAI is definitely changing the world of cybersecurity. Partly because of the hysteria, partly because of talented security researchers using it as an assistant to their research, but also because it's led to a lot of recklessness across the board that I think could make the world overall less secure. That being said, I do think some positive things have come out of GenAI's involvement in cybersecurity - but I'm not entirely sure if they outweigh the negatives.
_________________
Reverse engineering nerd, occasionally other things
"epic/he/her/him/epic/epic/epic/epic/ness uses light mode" - tox 2022