Same workload. Both risks.

Every platform team I've worked with runs two parallel programs against the same cluster. One is the upgrade project: a version deadline, a list of removed APIs, a migration checklist. The other is vulnerability management: a scanner, a ticket queue, a monthly severity report.

They almost never talk to each other. The upgrade list is keyed by API group and version. The vulnerability list is keyed by image digest. Neither is keyed by the thing you actually operate — the workload — so nobody notices when the same Deployment shows up on both.

That overlap is the most useful list in your cluster, and it takes about twenty minutes of shell to produce. It's worth producing because the two problems have the same remedy: you were going to touch that workload anyway. Bumping the base image and migrating the manifest in one PR costs one review, one rollout, and one round of soak time. Doing them six weeks apart costs three of each.

Step 1 — find what's actually blocking the upgrade

There are two different questions here, and conflating them is where most upgrade audits go wrong.

Question A: what objects are stored using an API that's going away?

This is the one that maps to workloads. Cluster scanners like kube-no-trouble (kubent) and Pluto read live objects and tell you the kind, namespace, and name of everything using a deprecated or removed API version.

kubent --target-version 1.32 --output json > kubent.json

jq -r '.[] | [.Kind + "/" + .Namespace + "/" + .Name, .ApiVersion, .ReplaceWith] | @tsv' \
  kubent.json | sort > blockers.tsv

Deployment/checkout/api-gateway     apps/v1        apps/v1
Ingress/edge/storefront             networking.k8s.io/v1beta1   networking.k8s.io/v1
PodDisruptionBudget/payments/ledger policy/v1beta1 policy/v1

Note the caveat that trips people up: kubent reports what is stored in the cluster right now. If your manifests still contain the old API version in Git but the object was applied through a chart that already normalizes it, the cluster looks clean and your next helm upgrade still fails. Scan both — the live cluster and your rendered manifests.

Question B: what is still calling a removed API?

This one catches controllers, operators, CI jobs, and that one cron script nobody owns. The API server has exposed a purpose-built metric for it since Kubernetes 1.19:

kubectl get --raw /metrics \
  | grep '^apiserver_requested_deprecated_apis'

apiserver_requested_deprecated_apis{group="flowcontrol.apiserver.k8s.io",
  removed_release="1.32",resource="flowschemas",subresource="",version="v1beta3"} 1

The removed_release label is the important one — it tells you the exact version where this stops working, so you can sort by deadline rather than by anxiety. The metric only reports APIs that are deprecated and have a scheduled removal, so anything in that output is a real, dated problem.

The gauge alone doesn't say who's responsible. If you're scraping the API server with Prometheus, join it against request traffic to get the user agent:

apiserver_requested_deprecated_apis
  * on (group, version, resource, subresource)
  group_right() sum by (group, version, resource, subresource, removed_release, user_agent)
    (rate(apiserver_request_total[7d]))

Use the longest range your retention allows. A weekly reconcile loop is invisible in a 1-hour window, and those are exactly the callers that surprise you at 2 a.m. on upgrade night.

Step 2 — inventory images by workload

Scanner output is keyed by image. To join it against anything operational, you first need a map from workload to image.

kubectl get deploy,sts,ds -A -o jsonpath='{range .items[*]}{.kind}{"/"}{.metadata.namespace}{"/"}{.metadata.name}{"\t"}{range .spec.template.spec.containers[*]}{.image}{" "}{end}{"\n"}{end}' \
  | sort > workload-images.tsv

Two things worth knowing about that command. Requesting multiple resource types is what keeps .kind populated on each item — ask for a single type and kubectl strips it from the list entries, and your keys come out as /checkout/api-gateway. And it reads .spec, not .status, so mid-rollout you'll get the intended image rather than what's serving traffic. That's usually what you want for planning, but say so out loud if you hand the list to someone else.

Then scan. Trivy will do the whole cluster and key its output the same way:

trivy k8s --report all --severity HIGH,CRITICAL --format json -o vulns.json cluster

jq -r '
  .Resources[]
  | select(.Results != null)
  | [ .Kind + "/" + .Namespace + "/" + .Name,
      ([.Results[].Vulnerabilities // [] | length] | add // 0)
    ] | @tsv
' vulns.json | awk -F'\t' '$2 > 0' | sort > vulns.tsv

Filtering to HIGH and CRITICAL isn't laziness, it's the only way the output stays legible. A full-severity scan of a normal cluster returns thousands of rows and gets ignored, which is worse than not scanning.

Step 3 — join them

Both files are now keyed by Kind/namespace/name. That's the whole trick.

join -t $'\t' \
  <(cut -f1 blockers.tsv | sort -u) \
  <(sort -u vulns.tsv)

Deployment/checkout/api-gateway    23
Deployment/payments/ledger-api     11
Ingress/edge/storefront            4

On the clusters I've run this against, the intersection is typically 5–15% of workloads. It is never zero, and it is never random. The same services show up on both lists for a structural reason: a workload that's been running long enough to accumulate a deprecated API version has also been running long enough for its base image to drift, because both are symptoms of the same thing — nobody has had a reason to open that repo in a year.

That's also why this list is tractable. You are not being asked to fix everything. You're being asked to fix the dozen services that have been quietly aging, and you get two problems closed per PR.

Step 4 — order by blast radius, not by count

Don't sort the intersection by CVE count. A workload with 40 findings behind an internal-only Service is less urgent than one with 6 that terminates public traffic. Pull four signals and rank on those:

# replicas — is there any redundancy at all?
kubectl get deploy -A -o custom-columns=\
'W:.kind,NS:.metadata.namespace,NAME:.metadata.name,REPLICAS:.spec.replicas'

# disruption budgets — will a node drain stall on this?
kubectl get pdb -A

# ingress-exposed — which Services are reachable from outside?
kubectl get ingress -A -o jsonpath='{range .items[*]}{.metadata.namespace}{"/"}{range .spec.rules[*].http.paths[*]}{.backend.service.name}{" "}{end}{"\n"}{end}'

What this turns into

The output of a morning's work is a short, ordered list where every row justifies itself: this workload blocks the upgrade, carries N high-severity CVEs, serves public traffic, and runs one replica. That sentence gets budget in a way that "we have 4,000 vulnerabilities" never has.

Practically, the sprint plan writes itself:

• Intersection, internet-exposed — one PR each, base image bump plus API migration together. Do these first; they're the reason to run the exercise.
• Intersection, internal — same PR shape, batched by owning team.
• Blockers only — mechanical manifest migration, safe to batch aggressively.
• CVEs only — back to the normal patch cadence. It was never the emergency.

Where this breaks down

Being honest about the limits, because they matter if you try to run this as a standing process:

• It's a snapshot. Both files are stale the moment a deploy lands. Fine for a one-off audit, not something to hand a compliance auditor.
• Bare Pods, Jobs, and CronJobs are missing from the step 2 query. Add them if you run meaningful workloads that way — plenty of teams do, in the namespaces they think about least.
• It doesn't know about reachability. A CRITICAL in a library the process never loads still counts as a CRITICAL here. Trivy can filter unused packages in some ecosystems, but you'll still be triaging by hand.
• Multi-cluster doesn't compose. Running this across fifteen clusters means fifteen sets of files and no fleet-level ordering, which is where most people quietly stop.

None of that makes the exercise less worthwhile. Run it once before your next upgrade and you'll have a better-ordered backlog than either of your two existing ones.