Every phishing wave tells you what people are afraid to lose. For fifteen years that was online banking, then parcel deliveries, then Office 365 mailboxes, then multi-factor prompts themselves. The SANS Internet Storm Center has now flagged phishing that impersonates AI services such as ChatGPT, which suggests the crews doing volume phishing have decided enough people would panic at losing their AI subscription to make it worth the template work.

That's the part worth sitting with. Phishing lure selection is a decent proxy for what a population actually depends on. Nobody builds a convincing fake billing page for a service their targets don't care about. The lure has migrated because the dependency has.

The uncomfortable inventory question

Here is a test worth running on your own estate this week, and the answer is rarely comfortable. Pull your expense claims for the last quarter and grep for AI vendors. Then pull your enterprise agreements and see how many of those same vendors appear. In most organisations the two lists barely overlap.

What that gap represents operationally:

  • Accounts created with work email addresses but no federation, so no conditional access, no session control, no central MFA policy and no way to revoke on leaver day.
  • Payment sitting on personal or departmental cards, which means the billing notifications go to an individual inbox rather than a finance mailbox that has seen ten thousand invoices and knows what a fake one looks like.
  • Chat histories containing whatever people pasted in — contract clauses, code, incident notes, customer names — behind a password the organisation has never seen and cannot rotate.

A credential-phish against that account is not a nuisance. It's an unlogged data breach with no forensic trail on your side.

Why the billing lure works better than it should

The classic corporate phish has to fight a decade of user education. People have been trained to be suspicious about mailbox quota warnings and DocuSign requests. An email saying your AI subscription payment failed and access is suspended has almost no immune response behind it, because we never ran awareness training for a service category that didn't exist three years ago.

It also carries genuine urgency, which is the thing awareness training never really beats. If someone has built their working week around a tool — drafting, summarising, code review — and gets told access stops in twenty-four hours, they will click. Not because they're careless. Because the consequence feels immediate and the cost of checking feels high.

The email Mertens describes is well made, timed for the end of the month when billing processes restart, and aims to collect payment details. His framing in the diary is the right one: phishing works on fear of losing "something". The category of things people fear losing has quietly expanded, and our controls haven't followed.

The second-order risk: API keys and tokens

Consumer subscription theft is the visible layer. The layer underneath is worse. Anyone in your organisation doing real work with these platforms holds API keys, and those keys are billable, transferable and rarely scoped properly. Keys turn up in Postman collections, in .env files committed to internal repos, in Teams messages and in runbooks marked internal-only.

A phished platform login usually means access to the key management console. From there the attacker doesn't need your network at all — they mint a key, run inference on your account, and you find out via a billing alert three weeks later if you're lucky and via a spend cap you never set if you're not. There's no lateral movement, no EDR telemetry, no domain controller to defend. It happens entirely outside the boundary you spent your budget on.

What actually reduces this, in order of effort

I'm deliberately not offering a maturity model here. Three things, cheapest first.

Find them. Your identity provider's application discovery — Entra ID's cloud app discovery, or whatever your CASB does — will surface OAuth grants and sign-ins to AI platforms if anyone authenticated with a work identity. Cross-reference with expenses for the ones that used a personal signup. This is an afternoon of work and it will change the conversation with your leadership team, because the number is always higher than they expect.

Federate what you can, block what you can't. Where a vendor supports SAML or OIDC and your licence tier allows it, bring the account into your tenant so conditional access and revocation actually apply. Where the vendor doesn't, or where the business won't pay for the enterprise tier, make a decision and record it. Shadow usage that leadership has explicitly accepted is a risk position. Shadow usage nobody has looked at is negligence with better PR.

Rotate keys and set spend caps. Every platform worth using has hard spend limits and key expiry. Almost nobody turns them on. It costs nothing and it converts an open-ended financial exposure into a bounded one.

The awareness training gap is the real finding

Most of our phishing simulation programmes are running templates that reflect 2019's threat surface. Fake HR portals, fake invoice approvals, fake password expiry. If you can't remember the last time your simulation platform sent an AI-billing lure, that's a coverage hole, and it's the kind that shows up in your click-through numbers as a nasty surprise rather than a gradual trend.

The fix is boring: add the templates, brief the service desk on what a genuine subscription notification from your sanctioned vendors looks like, and give people a low-friction way to check. The last part matters most. If verifying an email takes longer than clicking it, they'll click it. That has always been true, and no amount of training slides has changed it.

The pattern, not the incident

None of this is a novel attack. It's template reskinning against a new dependency, which is exactly what commodity phishing crews do and do well. The lesson isn't about AI at all — it's that our phishing defences are indexed against the applications we officially manage, while attackers index against what people actually use. Every time those two lists diverge, someone builds a lure for the gap.

Close the inventory gap and the lure loses most of its power. Leave it open and you'll keep finding out about your AI estate from a billing alert.

Source: Phishing Campaigns Targeting AI Solutions Providers (SANS Internet Storm Center)