Hackers are pwning packages at an exhausting clip.
But the hacks are hackneyed. What’s new is the doom cycle: Code that steals keys to publish code to steal more keys.
A zombie army of infected code. And AI is making it worse.
GitHub Actions are a trap
Trivy is an open-source security scanner. But if you used Trivy in late March, you had a bad time.
On March 19th, hackers pushed a version of Trivy that tried to smuggle secrets from anywhere it ran. Trivy cited a “misconfiguration” in their continuous integration (CI) system, GitHub Actions.
But the exploit was less a misconfiguration and more a GitHub Actions trap.
Here’s a simplified version of how Trivy got pwnd1:
# INSECURE. DO NOT USE.
on:
pull_request_target
jobs:
check:
steps:
- uses: action/checkout@deadbeefdeadbeefdeadbeefdeadbeefdeadbeef
with:
ref: refs/pull/${{ github.event.pull_request.number }}/merge
- uses: ./.github/actions/setup-go
- uses: some/go-static-analysis@c0ffeec0ffeec0ffeec0ffeec0ffeec0ffeec0ffAt first glance, this code looks fine:
- No secrets referenced.
- Third-party actions pinned to an immutable hash.
- Check out a pull request. Perform some static analysis.
But this code is a verbatim antipattern from a 2021 GitHub blog post titled “preventing pwn requests”:
if the
pull_request_targetworkflow only […] runs untrusted code but doesn’t reference any secrets, is it still vulnerable?Yes it is
The problem is pull_request_target:
pull_request_target– plunks a nice, juicyGITHUB_TOKENinto the environment.actions/checkout– takes an optional parameterpersist-credentials, which removes secrets if set tofalse. But the default for the parameter istrue.
Setting the persist-credentials parameter to
false has been an open issue in GitHub Actions since
2021.
Your $HOME is a crime scene
Once hackers had Trivy’s keys, they published a new version of Trivy to steal more keys.
LiteLLM used Trivy in their CI. The same CI they used to publish code to PyPI, the Python software registry. When LiteLLM’s CI ran the compromised Trivy, hackers nabbed their publishing key.
And on March 24th, when Callum McMahon fired up his IDE, his MacBook froze. And that’s how he discovered the LiteLLM hijack.
McMahon’s MacBook was flailing at bad code that hackers snuck into LiteLLM. And the bad code trying to steal credentials:
~/.netrc~/.aws/credentials~/.config/gcloud~/.config/gh~/.azure~/.docker/config.json~/.npmrc~/.git-credentials~/.kube/
Files that are typically strewn around $HOME
directories, full of tokens and keys, often unencrypted.
AI and the supply chain doom spiral
We’ve dealt with problems like unencrypted credentials, unpinned dependencies, and CI footguns forever.
But AI has accelerated everything, including repeating security mistakes.
On the day of the Trivy compromise, I asked Claude, “how do I scan docker registry images for security vulnerabilities?”
The reply, in part:
CI/CD Integration Example (GitHub Actions with Trivy)
- name: Scan image for vulnerabilities
uses: aquasecurity/trivy-action@master
Broken in two ways:
- Unpinned references –
masteris a reference that changes all the time. If hackers zombify the repo, I’d be the first victim. - Active vulnerability – No mention whatsoever of the CVE posted that day. I never asked, so Claude never checked.
Meanwhile, Vercel’s CEO has attributed his company’s recent data breach to a hacker that was “accelerated by AI.” And Anthropic’s latest hype tour includes briefing the US Federal Reserve Chair about vulnerabilities unearthed by their frontier model.
Bad guys with LLMs get superpowers. Good guys with LLMs fall prey to mid-2010’s CI problems.
And the same tool that can root out 27-year-old
security problems in OpenBSD, will still tell you to pin your GitHub
actions to @master.
My GitHub Actions example is a simpler verison of the action removed in aquasecurity/trivy #10259.↩︎