OpenAI Info Leak, BitCoin ATM Hack, GitHub RSA SSH Key, Measuring AI Security – ASW #234

Nope, I don't own a Ferrari, but I grew up watching Thomas Magnum drive a Ferrari 308 GTS on Magnum, P.I. That's also pretty much the only reason I grabbed this article, other than to use it as a compare and contrast with OpenAI's notification this week.

Ferrari's response is light on details, but the article notes "...no payment details and/or bank account numbers or other sensitive payment information, nor details of Ferrari cars owned or ordered had been stolen." So, at least there's an implication that Ferrari had sufficient logs and monitoring to make such statements.

The initial communication about this information leak was rough, with a "we feel awful about this" variation on taking security seriously and creeping towards a blame-the-intern approach by shifting the blame to an open source library.

This blog post makes up for that -- although I'd still quibble with the framing of "outage" as opposed to a clearer indication of account information leak. They go into details of the scope of who may have been affected, what what disclosed, what the underlying problem was, and how it was addressed. That kind of transparency is what we like to see in these kinds of write-ups.

The underlying problem was a race condition in redis-py, whose fix seems pretty easy to understand in hindsight. They used Python's "asyncio.shield()" method to protect the request/response queue from being corrupted by a cancelled request. Also nice to see tests added with this commit.

It also looks like OpenAI dealt with another cacheing issue that could lead to an account takeover. Bug bounty researcher @naglinagli describes the flaw in that thread, which includes some links to "Web Cache Deception Attacks" (also this pdf).

I almost skipped over this because -- yawn -- another crypto hack. But in reading the security incident, I was struck by the three bullet points about what happened. An attacker abused a video upload feature to upload a custom app to the server's deployment folder, from which the server was configured to automatically launch anything there. Ouch. It sounds like a lot of design improvements could have been made here to prevent that.

The theme emerging this week is how to communicate security incidents, including communications that don't directly stem from incidents. I'll begrudgingly accept "we take security seriously" if an incident report goes into details that support that assertion. And I'll happily read "in an abundance of caution" write-ups that practice transparency about potential security issues.

In this case, their "RSA SSH private key was briefly exposed in a public GitHub repository" (ouch). This was human error, not a compromise, and they did the right things by immediately rotating keys and letting users know what was going on.

Although, in a few cases, SSH clients weren't very clear about what was going on. We'll ask John about that...

We've been talking about logs recently, so this article grabbed my attention. It also fits our theme of incident write-ups. In this case, Datadog found a CloudTrail bypass, reported it to AWS, and AWS fixed it.

The broader picture is that that AWS customers wouldn't necessarily ever be aware of this. There's no equivalent CVE for tracking and measuring these kinds of incidents. On the other hand, AWS was able to resolve this without customer action needed and their audit of the impacted services indicated to malicious activity. So, it's nice that customers don't need to do anything here and none were apparently impacted by it.

There's more to AI security than prompt injections, and there's more than just adversarial attacks against uses like face recognition. This blog post and the paper it links proposes terminology and techniques for discussing risks and not only distinguishing between security and safety, but defining what safety should mean within the context of AI. It also explains why threat modeling most familiar to appsec may only be narrowly applicable and insufficient for reasoning about these systems.

We just covered this in episode 233, where it was disclosed as affecting Pixel's default image editor. Now it looks like Windows image editors join Android in keeping cropped out image areas within the image file's data.

Seriously. How is it to write code to crop an image?

Unfortunately, I'm partially posting this in jest. When I first saw the article, I was hoping to see mention of some developers union that was pushing companies to care more about code quality or something, but no...it's just a starry-eyed manifesto that probably won't get much traction.

Whoops. Grumpy John's coffee still hasn't kicked in on a Monday morning...

While I usually don't post links straight to vendors, Socket has released a "safe" version of npm which basically filters "npm install" requests through their filters to minimize the chance of installing "malware, typosquats, install scripts, protestware, telemetry" etc.

Why could Microsoft/Github/npm not come out with this???