litellm Supply Chain Attack (March 2026): How TeamPCP Backdoored a PyPI Package Used by Millions

On March 24, 2026, a package downloaded 3.4 million times per day became a delivery mechanism for credential-stealing malware. Here’s exactly what happened - verified, timestamped, and actionable.

⚠️ Incident Status: Resolved. Versions 1.82.7 and 1.82.8 were yanked from PyPI on March 24, 2026 at approximately 16:00 UTC -roughly 3 hours after publication. The CVE assigned is CVE-2026-33634 (CVSS4B score: 9.4). If you installed litellm on March 24, 2026 between 10:39 UTC and 16:00 UTC, treat your environment as compromised and follow the remediation steps in this post.

Background: What is litellm?

litellm is an open-source Python library and proxy layer that provides a unified interface for calling virtually every major LLM provider - OpenAI, Anthropic, Cohere, Mistral, Gemini, and more - with a single consistent API. It is the connective tissue of the modern AI development ecosystem, downloaded approximately 3.4 million times per day on PyPI.

This was not a typosquatted package. This was not a look-alike with a similar name. This was a direct compromise of the real, official litellm package on PyPI, published using the legitimate maintainer’s stolen credentials. That distinction is what makes this incident so significant - and so hard to detect.

The Full Attack Timeline - Verified & Sourced

This attack did not happen in isolation. It was the fourth stage of an ongoing, multi-week supply chain campaign by a threat group known as TeamPCP. The following timeline is sourced from official advisories and verified analyses by Datadog Security Labs, Snyk, Endor Labs, Kaspersky, and the litellm team.

Stage 1 - Late February 2026: Trivy Initial Compromise

TeamPCP used previously stolen credentials to compromise Trivy, the widely used open-source container and filesystem security scanner maintained by Aqua Security. The attack injected malicious code into the trivy-action and setup-trivy GitHub Actions, as well as the v0.69.4 release tag. The compromised Trivy action was designed to exfiltrate secrets from any CI/CD pipeline that ran it - specifically targeting PYPI_PUBLISH, NPM_TOKEN, GITHUB_TOKEN, and similar high-value secrets from GitHub Actions runner environments.

Stage 2 - March 19, 2026: Aqua Security GitHub Compromise

Using credentials harvested from compromised Trivy CI/CD runners, TeamPCP accessed Aqua Security’s internal GitHub organization. They renamed 44 repositories - all with the prefix tpcp-docs- and the description “TeamPCP Owns Aqua Security” - a public declaration of access that served as both provocation and proof of capability. the compromised github account its likely of the owner which is this

Stage 3 - March 23, 2026: Checkmarx KICS + OpenVSX

The campaign moved to Checkmarx (kics-github-action, ast-github-action) and two OpenVSX extensions: ast-results 2.53.0 and cx-dev-assist 1.7.0. The KICS payload used the same C2 infrastructure: checkmarx[.]zone. The fallback pattern: if direct C2 contact failed, the payload created a docs-tpcp repository using the victim’s GITHUB_TOKEN - leaving a permanent marker of compromise.

Stage 4 - March 24, 2026: litellm on PyPI

10:39 UTC  litellm v1.82.7 published to PyPI — malicious source injection
10:52 UTC  litellm v1.82.8 published to PyPI — .pth persistence injection
~11:00 UTC FutureSearch engineer Callum McMahon detects anomaly while testing
           a Cursor MCP plugin that pulled litellm as a transitive dependency
11:XX UTC  Community GitHub issue #24512 opened
13:03 UTC  Issue #24512 closed "not planned" by compromised maintainer account
           Hundreds of bots flood thread to dilute discussion
13:XX UTC  Community opens parallel issue #24518
13:XX UTC  Hacker News thread reaches 324 points — widespread community awareness
16:00 UTC  PyPI quarantines both packages, removes from registry
20:15 UTC  PyPI quarantine lifted, clean version restored

How did TeamPCP get the PyPI credentials?

litellm used Trivy in its own CI/CD pipeline, pulling it via apt without a pinned version. When the litellm CI/CD ran during the compromised window, it pulled the backdoored Trivy action, which silently exfiltrated the PYPI_PUBLISH token from the GitHub Actions runner. With that token, TeamPCP published directly to PyPI - bypassing the normal Git-tagged release process entirely. No corresponding tag or GitHub release exists for either v1.82.7 or v1.82.8.

The attack chain: compromised Trivy → stolen PYPI_PUBLISH token → malicious packages published directly to PyPI. Every unpinned CI/CD dependency is a potential entry point into your entire release pipeline.

🦠 The Malware: A Three-Stage Attack

Security researchers at Endor Labs, JFrog, Snyk, and Datadog all published independent, verified analyses of the payloads. Here is exactly what the malware did.

Payload Differences: v1.82.7 vs v1.82.8

The two versions used different injection techniques:

v1.82.7 - Source Code Injection: The payload was base64-encoded and embedded directly inside litellm/proxy/proxy_server.py. It executed whenever anything imported litellm.proxy - the standard import path for LiteLLM’s proxy server mode.

v1.82.8 - .pth File Injection (More Dangerous): This version added litellm_init.pth to site-packages/. The .pth mechanism fires on every Python interpreter startup with no import required - including when pip, python -c, or an IDE’s language server starts Python. This means the payload could execute during CI/CD build steps, not just at application runtime.

⚠️ Critical finding from Snyk: The litellm_init.pth file in v1.82.8 is correctly declared in the wheel’s RECORD file with a matching hash. pip install --require-hashes would have passed. Hash verification confirms a file matches what PyPI advertised - it does not tell you whether the advertised content is malicious. The only detection path is auditing whether a package installs .pth files containing patterns like subprocess, base64, or exec.

Stage 1: Credential Harvester

The first stage swept the compromised system for credentials across every standard location:

# Targets confirmed by Endor Labs analysis:
~/.ssh/                           # SSH private keys
~/.aws/credentials                # AWS credentials
~/.config/gcloud/                 # Google Cloud credentials
# Kubernetes secrets across ALL namespaces
~/.kube/config                    # Kubeconfig
.env / .env.local / .env.prod     # Environment files
~/.npmrc                          # NPM tokens
~/.pypirc                         # PyPI tokens
# Cryptocurrency wallet files (common locations)

All harvested data was packaged as an encrypted archive named tpcp.tar.gz and exfiltrated via HTTPS POST to models.litellm[.]cloud - a domain deliberately crafted to impersonate an official litellm service.

Stage 2: Kubernetes Lateral Movement

If a Kubernetes service account token was discovered, the malware escalated dramatically:

# Lateral movement logic (simplified representation based on Endor Labs analysis)
# Reads ALL cluster secrets across ALL namespaces
# Deploys privileged alpine:latest pods to EVERY node in kube-system
# Each pod mounts the HOST FILESYSTEM at /host
# Installs persistent backdoor on the host via the mount

This gave the attacker root-equivalent access to the underlying host of every Kubernetes node in a compromised cluster.

Stage 3: Persistent Backdoor

Each compromised system received a backdoor installed at two levels:

# Persistence path (local machine)
~/.config/sysmon/sysmon.py
~/.config/systemd/user/sysmon.service  # systemd user service

# Persistence path (Kubernetes nodes, via host mount)
/root/.config/sysmon/sysmon.py
/etc/systemd/system/sysmon.service     # system-level service

# Behavior: polls for additional payloads
# C2: https://checkmarx[.]zone/raw

The sysmon name was chosen to blend in with legitimate system monitoring tools.

The CanisterWorm Component

Kaspersky's independent analysis identified an additional component: CanisterWorm - a self-replicating worm that propagated the campaign into the JavaScript npm ecosystem. More than 20,000 repositories are considered potentially vulnerable as a downstream result.

⚠️ Destructive Capability - Verified by Kaspersky: The malware contains an explicit check that wipes the entire Kubernetes cluster and all its nodes if it detects Farsi as the system language or the Tehran timezone. In all other regions, it defaults to credential theft. TeamPCP has explicitly claimed credit for this behavior.

Am I Affected? How to Check

Quick Version Check

# Check installed version
pip show litellm | grep Version

# If Version: 1.82.7 or Version: 1.82.8 — treat as compromised
# Do NOT simply upgrade. The payload may have already executed.

Check for .pth File (v1.82.8)

# Search Python environments and caches
find ~/.cache/uv -name "litellm_init.pth" 2>/dev/null
find /usr -name "litellm_init.pth" 2>/dev/null
find ~/.virtualenvs -name "litellm_init.pth" 2>/dev/null
find . -name "litellm_init.pth" 2>/dev/null

# Check site-packages directly
python3 -c "import site; print(site.getsitepackages())"
# Then: ls <output_path>/*.pth

Check for sysmon Backdoor

# Check for persistence files
ls -la ~/.config/sysmon/ 2>/dev/null
systemctl --user status sysmon.service 2>/dev/null

# Check system-level service
systemctl status sysmon.service 2>/dev/null

# Scan for the sysmon script pattern
find / -name "sysmon.py" 2>/dev/null | grep -v "proc"

Scan GitHub Actions CI/CD Exposure

The litellm official security advisory provides a scanning script. Key logic:

# Scan your GitHub org for workflows that ran during the incident window
# and installed litellm 1.82.7 or 1.82.8

WINDOW_START = "2026-03-24T10:39:00Z"
WINDOW_END   = "2026-03-24T16:00:00Z"
COMPROMISED  = {"1.82.7", "1.82.8"}

# Full script: https://docs.litellm.ai/blog/security-update-march-2026

Remediation — Step by Step

# 1. Remove the compromised package
pip uninstall litellm -y

# 2. Purge all package caches
pip cache purge
rm -rf ~/.cache/uv           # uv package manager cache
rm -rf ~/.cache/pip          # pip cache

# 3. Remove .pth file if found
find / -name "litellm_init.pth" -delete 2>/dev/null

# 4. Remove sysmon backdoor
rm -rf ~/.config/sysmon/
systemctl --user disable sysmon.service 2>/dev/null
systemctl --user stop sysmon.service 2>/dev/null
systemctl disable sysmon.service 2>/dev/null
systemctl stop sysmon.service 2>/dev/null

# 5. Reinstall a clean, explicitly pinned version
pip install "litellm==1.82.6"

# 6. Verify
pip show litellm | grep Version
# Expected: Version: 1.82.6

Rotate These Credentials Immediately

If your environment was exposed, rotate every one of these without exception:

☐ SSH private keys          → generate new keypairs, update authorized_keys everywhere
☐ AWS / GCP / Azure creds   → rotate IAM keys, revoke old ones
☐ Kubernetes SA tokens      → rotate via kubectl
☐ GitHub / GitLab PATs      → revoke and regenerate in Settings > Tokens
☐ NPM / PyPI publish tokens → revoke via registry settings
☐ All .env secrets          → API keys, DB passwords, JWT secrets, everything
☐ Crypto wallet keys        → if applicable, move funds to new wallets

Indicators of Compromise (IOCs)

All IOCs are sourced from public, verified analyses. Cross-reference with your own logs and SIEM.

What This Tells Us About Supply Chain Attacks in 2026

The architecture of this attack reveals the current state of the threat:

Trivy (security scanner) — unpinned, trusted in CI/CD
  ↓ PYPI_PUBLISH token stolen from runners
litellm PyPI (3.4M daily downloads)
  ↓ .pth file executes on every Python startup
Developer workstations + CI/CD environments
  ↓ CanisterWorm propagation
npm ecosystem — 20,000+ potentially vulnerable repos

As Datadog’s analysis states: “This is a case study in how stolen CI/CD credentials from an initial attack had cascading impact across multiple ecosystems in a matter of days.”

The particularly dangerous pattern here: TeamPCP specifically targeted security tools - Trivy (vulnerability scanner), KICS (infrastructure scanner), and litellm (AI model routing). Security tools run with elevated permissions. Compromising one is a force multiplier that gives the attacker a pipeline into every project that trusts it.

Security tools are trusted implicitly and run with elevated permissions. That trust, combined with unpinned dependencies, made Trivy the perfect entry point for a cascading multi-ecosystem compromise.

Three Hardening Steps That Would Have Broken This Attack Chain

1. Pin every dependency — including in CI/CD tool installation steps

# Vulnerable (what litellm's CI/CD did)
- run: apt-get install trivy

# Safe (pins the version)
- uses: aquasecurity/trivy-action@0.28.0   # pinned by commit SHA is even better

2. Audit .pth files in all Python environments

# Run this periodically
find $(python3 -c "import site; print(site.getsitepackages()[0])") -name "*.pth" \
  | xargs grep -l "subprocess\|base64\|exec\|__import__"

3. Use Datadog’s open-source Supply-Chain Firewall

pip install supply-chain-firewall
# Wraps pip install and blocks known-malicious packages automatically
scfw pip install litellm

and before we finish off lets see the timeline of the attack in a brief flow view

Key Takeaways

1. This is a real, verified, timestamped incident - CVE-2026-33634, CVSS 9.4, confirmed by six independent security organizations
2. litellm v1.82.7 and v1.82.8 are malicious - installed between 10:39–16:00 UTC on March 24, 2026 = treat as compromised
3. The root cause was an unpinned Trivy dependency in litellm’s own CI/CD - one pinned version tag would have broken the entire attack chain
4. Hash verification does not protect against legitimate-credential attacks - v1.82.8 passes all integrity checks
5. TeamPCP specifically targets security tooling - highest trust + highest permissions = maximum impact
6. Rotate all credentials immediately if affected - the harvester was indiscriminate
7. This campaign is explicitly ongoing - per Endor Labs: “Each compromised environment yields credentials that unlock the next target”

🔗 Verified Primary Sources

• litellm Official Security Advisory - BerriAI / LiteLLM team
• Snyk Technical Analysis - Callum McMahon
• Datadog Security Labs Full Timeline - IOCs included
• Kaspersky Blog - CanisterWorm
• Wiz Blog - KICS Stage
• FutureSearch - First Disclosure
• The Hacker News Coverage

Already rotated your credentials and patched your environments? Good. Now audit every other unpinned dependency in your CI/CD pipelines — because TeamPCP is still active.

SEO Keywords: litellm supply chain attack 2026, TeamPCP PyPI backdoor March 2026, CVE-2026-33634, litellm 1.82.7 1.82.8 compromised, Trivy CI/CD compromise, PyPI malicious package 2026, litellm credential stealer, supply chain security Python, PYPI_PUBLISH token stolen, CanisterWorm npm worm