Methodology

Tagging rules

Rules for data/tags.yaml. The eight layers are defined on the Model page; this file records how incidents are assigned to them. npm run data enforces the schema.

Schema

In-model row:

<incident-slug>:
  primary: 4              # exactly one layer number, 1..8
  secondary: [1]          # zero or more, never the primary
  actor: attacker         # optional: vendor_eval | operator | attacker | unknown
  related: [<slug>]       # optional, any row: the same event described from another side
  fixable: true           # true, false, or null
  vendor_response: patched
  in_the_wild: true

Outside-the-model row (no primary or secondary):

<incident-slug>:
  out_of_model: true
  class: capability       # capability | offensive_use | resource_theft | context | model_safety | adjacent
  fixable: true
  vendor_response: patched
  in_the_wild: false

Each row gets a one-line # comment above it with the reason, citing the Root Cause or Impact text.

In or out of the model

An incident is in-model only when agent tooling was the attack surface: the agent, its tools, runtime, config, supply chain, or credentials. Otherwise it is out_of_model: true with one class:

• capability: an AI found or built an exploit against non-agent software.
• offensive_use: AI used as the instrument against humans or non-agent systems, including impersonating an AI product as a lure.
• resource_theft: the key was stolen or leaked outside agent tooling (infostealer, phishing, code leak, credential resale) and then spent on inference or agent compute (LLMjacking).
• context: regulatory, legal, and policy events, trend reports, and surveys that are not themselves an incident. Kept because the report cites them, so the catalog stays fully tagged.
• model_safety: jailbreaks and harmful-content generation where the requester is the user, nothing acts on a system, and no third party is harmed. A guardrail is a content judgment, not a provenance control; defeating one with no action or target is out of the model.
• adjacent: ecosystem supply chain compromises with no agent or AI involvement as surface, instrument, capability, or payload target.

Worms and compromised packages (the Shai-Hulud lineage and similar): in-model, primary by the supply chain rule and secondary 1, when the payload harvested agent or AI tooling credentials and configs, or the compromise spread through agent tooling. adjacent when the AI angle is only that an AI-adjacent vendor was hit.

Agent runtime includes inference servers, LLM gateways, memory stores, vector databases, model registries, and ML platforms. Incidents against them are in-model.

Layer 1 primary covers any case where the run held a credential it should not have: over-broad, longer-lived than the run, or mishandled by the platform (stored unencrypted, exposed by an endpoint, leaked by the tool, or not bound to its issuer). resource_theft stays for keys stolen outside agent tooling and then spent.

An agent that harms a third party with no attacker involved is in-model; the failed link is whichever control should have stopped the action.

Model Capability Extraction: when the attacker defeated a platform control (for example, tricking the model into decrypting its own protected output), it is in-model at the layer of the defeated control. When extraction used ordinary API access at scale with no control defeated (distillation campaigns), it is out_of_model, class context.

Trusted AI Platform Abuse: in-model layer 4 when agent tooling executed or loaded the hosted artifact (a malicious model file loaded by a tool, a package pulled by an agent). out_of_model, class offensive_use, when the hosted artifact was a lure aimed at humans.

Fabricated Security Intelligence: in-model layer 7 when a review or approval process consumed the fabricated evidence (a maintainer acted on it, a CVE or advisory was issued, a fix was merged). out_of_model, class offensive_use, when it was only submitted or published and nobody acted.

Outside-the-model rows are counted and charted on /stats ("Outside the model"), never dropped.

Choosing the primary layer

• The primary is the link whose failure made the incident possible, not where the damage showed up.
• When unsure between two layers, pick the earlier number and say so in the reason. A taxonomy mapping (for example, Agent Sandbox Escape is layer 6) beats this tiebreak.
• An exposed server or endpoint with no authentication is primary 6. When it executes code, primary 6 with secondary 5; code execution that needs an authenticated user, or that the model emits, is primary 5.
• A network path out of a sandbox (DNS, proxy, open egress) is an isolation failure, primary 6. Layer 8 is primary only when the harm is data sent to a destination the agent chose (exfiltration, publication).
• An SSRF primitive is primary 8, with 6 secondary when it is reachable without authentication.
• A config or instruction file that runs or loads on open or clone is primary 5.
• An injected page or document that steers the agent is primary 2; add 8 if data left through an agent-chosen destination, and 1 if credentials were taken.
• Supply chain:
  • primary 4 when the artifact's origin was the first failure (typosquat, hallucinated name, maintainer account socially engineered);
  • primary 1 when a leaked or stolen credential is the documented root cause, or when an account was compromised by an unknown method;
  • primary 7 when the publish pipeline accepted what it should not have (forged provenance, an unreviewed push feeding a release).
• Oversight integrity is layer 7: monitoring disabled, tests altered to hide it, approval prompts that misdescribe an action.

Field meanings

• actor (optional, in-model rows only): who drove the failing action.
  • vendor_eval: the vendor's own evaluation or research agent caused the harm (third-party evaluators running agents count here too).
  • operator: a deployed agent acting for its operator, with no attacker involved.
  • attacker: an attacker drove the agent or exploited the platform. Vulnerability disclosures use this, since exploitation needs an attacker.
  • unknown: not determinable from the entry.
• related (optional, any row): slugs of other incidents that describe the same event from another side. Each row keeps its own tags; the validator checks the slugs exist.

• fixable: whether a code fix closes the class. true (a vendor patch closed it), false (architectural), null (unknown).
• vendor_response: what the vendor did, independent of fixable. An architectural incident can still be patched when the vendor changed its process or product. patched | disputed | not_a_vulnerability | no_response | mixed | unknown. Use mixed for multi-vendor advisories where vendors responded differently, and give the per-vendor split in the reason. A researcher saying there is nothing to patch is not a vendor response.
• in_the_wild: true if real victims or active exploitation of this incident's failure were reported. Internal red-teaming is false, even when a byproduct CVE is later exploited elsewhere.