How to check deprecated APIs before an EKS upgrade

Before every EKS minor upgrade someone has to answer one question: does anything we run still use an API version the target release stops serving? The answer is never in one place.

The examples below use the upgrade most EKS teams are planning now: 1.34 to 1.35. EKS 1.34 leaves standard support on 2 December 2026, after which the control-plane fee goes from $0.10 to $0.60 per cluster-hour (price your fleet here). Swap in your own versions; the method is the same for any hop.

Why the API server can't tell you

Kubernetes stores each object once, at a storage version, and converts it on the way out to whatever version the client asks for. A Deployment created through extensions/v1beta1 years ago reads back as apps/v1 today. So kubectl get shows you objects at the version you asked for, not the version the client that wrote them used.

What actually breaks after the upgrade is that client: the Helm chart, the CI job, the operator or the script that still sends the old apiVersion and now gets a 404. Finding deprecated APIs means finding those clients, not those objects. We go deeper on this in the upgrade check tools comparison.

The three places deprecated APIs hide

The middle row catches people out. If a stored release manifest contains an API the cluster no longer serves, the next helm upgrade after the cluster upgrade fails with unable to build kubernetes objects from current release manifest. Helm's documented fix is the helm-mapkubeapis plugin, which rewrites the stored manifest. Run it before the control-plane upgrade, not during the incident.

There's a fourth place worth naming even though it isn't a built-in API: add-on CRDs. Recent Kubernetes releases have mostly removed alpha and beta APIs that few production clusters use. The removals listed in Pluto's versions.yaml make the point:

For most EKS clusters the 1.35 hop removes nothing built-in that they use. The things more likely to stop you are an add-on whose installed version doesn't support 1.35, a CRD version that add-on is dropping, and changes that aren't APIs at all: from 1.35 the kubelet refuses to start on cgroup v1 nodes by default. A deprecated-API scan that comes back clean is one gate passed, not a green light.

Step 1: read EKS upgrade insights

EKS already watches API-server traffic for calls to deprecated APIs and reports them per target version. It's free and it's the closest thing to ground truth for live writers. In the console it's the Upgrade insights tab on the cluster; from the CLI:

aws eks list-insights --region us-east-1 --cluster-name prod \
  --filter categories=UPGRADE_READINESS,kubernetesVersions=1.35,statuses=ERROR,WARNING

aws eks describe-insight --region us-east-1 --cluster-name prod --id <insight-id>

describe-insight returns the deprecated API, its replacement, the version that stops serving it, and clientStats: the user agent, how many requests it made and when it last called. The user agent is how you find the owner.

Two properties of insights decide how you read them. They refresh every 24 hours, and they look at a rolling 30-day window. So a client you fixed today keeps the insight red for up to 30 days, and a client that runs less often than monthly (a quarterly report, a disaster-recovery runbook) may never appear at all. Insights also see nothing that hasn't called the API yet: not your Git repository, not your stored Helm releases.

Step 2: run Pluto on Git and on Helm releases

Fairwinds' Pluto is the most actively maintained static scanner, and the one to put in CI. It reads files, rendered charts from stdin, and the release manifests Helm stores in the cluster.

pluto detect-files -d ./deploy -t k8s=v1.35.0 -o wide

helm template my-release ./chart -f values.yaml | pluto detect - -t k8s=v1.35.0

pluto detect-helm -o wide -t k8s=v1.35.0

The target needs the leading v (k8s=v1.35.0, not 1.35). Pluto's exit code is built for CI: 2 means something deprecated was found, 3 means something removed, 4 means the replacement isn't available in the target yet. Add -r (--only-show-removed) when you want a pipeline that only fails on what actually breaks. There's a GitHub Action in the repository, and the rule file covers cert-manager and Istio CRDs as well as core Kubernetes.

Step 3: kubent, with a caveat

kube-no-trouble (kubent) is still the most-cited tool in upgrade guides, and it's pleasant to use: one binary that reads the cluster's last-applied annotations, Helm 3 releases and files.

brew install kubent
kubent -t 1.35 -o json -e

The caveat: the latest release, 0.7.3, is from August 2024, and its rule files stop at deprecated-1-32.rego. It has no rules for anything removed in 1.33 or later, so a clean kubent run against a 1.35 target does not mean clean. It also exits 0 on findings unless you pass -e. Use it for older hops, or as a second opinion, not as the gate.

Step 4: KubePug, for current data

KubePug takes the opposite trade-off. Its binary is older (v1.7.1, October 2023), but it doesn't ship its rules: it downloads a database generated from the Kubernetes API definitions, which its maintainers regenerate hourly. Install it as a krew plugin to run it against the cluster, or use the standalone binary on files.

kubectl krew install deprecations
kubectl deprecations --k8s-version=v1.35

kubepug --k8s-version=v1.35.0 --input-file=./deploy/ --error-on-deleted --error-on-deprecated
helm template my-release ./chart | kubepug --k8s-version=v1.35.0 --input-file=-

--input-file on a directory reads the YAML in it but not its subdirectories. For air-gapped CI, download the database and pass it with --database.

Step 5: kube-upgrade-check, for more than APIs

kube-upgrade-check is our open-source CLI (Apache-2.0). It reads a live cluster with the access you already have and looks at each object's managedFields, which record the apiVersion every field manager last wrote with. That is evidence of which client still writes the old version, which plain listing can't give you.

kubectl krew install upgrade-check
kubectl upgrade-check --target 1.35

# in a pipeline: JSON out, exit 4 at high severity or above
kubectl upgrade-check --target 1.35 -o json --fail-on high

Beyond removed APIs it checks kubelet and control-plane configuration, in-tree volume plugins, node runtime and version skew, add-on compatibility for eight add-on catalogs, and upgrade preflight. It also prints what it couldn't check: on EKS the control-plane flags are hidden, so several hundred configuration rules come back as not evaluated rather than as passed.

What it doesn't do yet: it doesn't read stored Helm release manifests and it has no mode for files in Git. Keep Pluto in CI for those.

Cross-check with the API server and audit logs

Since 1.19 the API server counts requests to deprecated APIs in a metric you can read with plain kubectl:

kubectl get --raw /metrics | grep apiserver_requested_deprecated_apis

The labels name the group, version, resource and the release that removes it. The count comes from whichever API-server instance answered and resets when it restarts, so a silent result isn't proof of absence. For durable evidence, enable the audit log type in EKS control-plane logging and query for the deprecation annotation in CloudWatch Logs Insights:

fields @timestamp, userAgent, requestURI
| filter @logStream like /kube-apiserver-audit/
| filter ispresent(`annotations.k8s.io/deprecated`)
| stats count(*) by userAgent, requestURI

Audit logs carry a CloudWatch ingestion cost, so turn them on for the few weeks before an upgrade if you don't keep them on permanently.

How the checks compare

A practical order for 1.34 → 1.35

1. Three to four weeks out: turn on audit logging if it's off, read upgrade insights, and run kubectl upgrade-check --target 1.35 for the APIs, add-ons and node gates in one pass.
2. Gate new changes: add Pluto to CI with -t k8s=v1.35.0 -r so nothing new using a removed API merges while you clean up.
3. Fix the stored releases: run pluto detect-helm and rewrite any hits with helm-mapkubeapis, then redeploy the chart with the new API versions.
4. Move add-ons first: upgrade third-party add-ons (Karpenter, cert-manager, ingress controllers) to versions that support 1.35. EKS-managed add-ons such as kube-proxy follow the control plane.
5. Re-check, then upgrade: confirm insights and audit logs are quiet for the clients you fixed, upgrade the control plane, then the nodes. Our multi-version EKS upgrade guide covers the sequencing when you're more than one version behind.

The gap no free tool closes

Put the table together and the gap is plain: no single tool covers runtime API usage, manifests, add-on compatibility and the support deadline together. Insights see calls but not files. Pluto sees files and Helm releases but not calls. kubent and KubePug see objects. None of them knows that your add-on's installed version is incompatible with the target, or that the cluster bills at six times the rate in six weeks. Most teams run two or three of these and reconcile the results in a spreadsheet.

The closest thing to one command for a live cluster is the free kubectl upgrade-check --target 1.35 above. For a fleet, Runtimez is read-only Kubernetes intelligence: what breaks on your next Kubernetes or EKS upgrade, what is at risk in production right now, and why an incident happened. Connected to a cluster, it puts deprecated APIs still called at runtime, CRD versions, nine add-on catalogs and each cluster's end-of-support date into one ranked list, and every finding quotes the vendor's release note with a read-only kubectl command to check it yourself. For manifests that haven't been applied yet, keep Pluto in CI either way.

Where this breaks down

• Runtime evidence has a window. Insights look back 30 days, the metric resets with each API-server restart, and audit logs only exist from the day you enabled them. A job that runs quarterly can pass every runtime check and still break.
• Static scanners see only what you point them at. A chart in another team's repository, objects created by the Terraform Kubernetes provider, or resources an operator generates at runtime won't be in ./deploy.
• Built-in APIs are the small part. Tools that only know Kubernetes' own APIs say nothing about Karpenter, Argo CD or ingress-controller CRDs, and an add-on that lags the cluster version blocks the upgrade just as hard. Pluto covers cert-manager and Istio; for the rest, check each add-on's compatibility matrix.
• Deprecated isn't removed. A deprecated API keeps working until its removal release. Blocking a 1.35 upgrade on a warning for 1.38 wastes the window; filter to what the target actually removes.
• Tool data ages. kubent's rules stop at 1.32 and will report a clean 1.35 cluster. Check when a scanner's rules were last updated before you trust its silence.

Frequently asked questions

How do I check for deprecated APIs before an EKS upgrade?

Check all three places they hide. Read EKS upgrade insights (aws eks list-insights filtered to UPGRADE_READINESS and your target version) for clients calling deprecated APIs. Run Pluto on your Git manifests, rendered charts and stored Helm releases. Then run a live-cluster check such as kube-upgrade-check or KubePug, and cross-check the API server's apiserver_requested_deprecated_apis metric or audit logs.

Why does an EKS upgrade insight stay red after I fixed the client?

Upgrade insights refresh every 24 hours and look at a rolling 30-day window of API-server traffic. A client you fixed today can keep the insight in ERROR or WARNING for up to 30 days. aws eks describe-insight shows each client's last request time, which tells you whether the calls have actually stopped.

Is kubent still reliable for Kubernetes 1.33 and later?

Not as a gate. The latest kubent release, 0.7.3, is from August 2024 and its rules stop at Kubernetes 1.32, so it reports nothing for APIs removed in 1.33 or later. Pluto and KubePug have current data; KubePug downloads a database its maintainers regenerate hourly.

Can kubectl tell me which API version a resource was created with?

Not from kubectl get. The API server converts stored objects to whatever version you request, so an object written through an old version reads back at the new one. The closest evidence on the object is metadata.managedFields, which records the apiVersion each field manager last wrote with. For clients, use upgrade insights, the deprecated-API metric or audit logs.

What does Kubernetes 1.35 remove?

Per Pluto's versions.yaml, 1.35 removes storage.k8s.io/v1alpha1 VolumeAttributesClass and storagemigration.k8s.io/v1alpha1, both alpha APIs few production clusters use. The bigger 1.35 changes are not APIs: the kubelet refuses to start on cgroup v1 nodes by default, and installed add-ons must support the new version.