Ingress to Gateway API: migrate without a big-bang (with a kind lab)

Migrate from Ingress to Kubernetes Gateway API without a big-bang, with a hands-on kind lab and practical guidance on ownership, ReferenceGrant, observability, and rollback.

From Ingress to Gateway API without a big-bang

Problem statement in 60 seconds

In multi-team clusters, “classic” Ingress often breaks down on three very practical points: extensibility via annotations, portability, and ownership.

Ingress was intentionally kept small: it mainly covers TLS termination and simple host/path routing, and in practice everything beyond that depends on controller-specific extensions (usually annotations). See the Ingress documentation and the Gateway API migration guide. Kubernetes recommends Gateway over Ingress, and the Ingress API is “frozen” (still GA/stable, but with no further feature development). “Frozen” does not mean “deprecated” or “removed”; it means a stable contract.

What you typically see in the wild (and what platform teams are expected to solve):

  • Annotation hell / fragmentation: every controller has its own key-value strings, often with semantics that do not exist in the API itself.
  • Ownership issues: one Ingress object mixes infrastructure choices (entry points, TLS, LB specifics) with app routing. In a shared cluster this often means: either everyone can touch everything, or teams must open tickets for every change.
  • Vendor/controller lock-in: once you depend on controller-specific annotations, switching gets expensive and risky.

When Ingress is still perfectly fine (and it is okay to say that explicitly):

  • You have one team that manages the controller and all Ingress resources (self-service is not the issue).
  • Your routing is simple (host/path + TLS) and you use few or no controller extensions.
  • You intentionally do not want to invest in a new governance model yet, and you do not have strong multi-tenant pressure.

Reality check (February 2026): if you run community ingress-nginx, there is a hard lifecycle driver. Kubernetes SIG Network + SRC announced retirement in this announcement and later clarified details in a follow-up statement/update. That does not mean “Ingress disappears,” but it does mean an internet-facing component without security updates becomes a governance and risk discussion you usually cannot win by postponing. Always check current maintenance/release status before finalizing production decisions.

Gateway API: the minimum you need to understand

The core is not “more features,” but roles + explicit contracts. Kubernetes explicitly describes this design as role-oriented (infra provider / cluster operator / app developer), portable, expressive, and extensible. Also, many capabilities that needed annotations with Ingress are now modeled directly in the API.

In practice, for HTTP you mostly interact with these four objects:

  • GatewayClass: cluster-scoped “which controller manages this?”, comparable to IngressClass/StorageClass; you need at least one.
  • Gateway: the entry point / infrastructure instance (listeners: protocol/port/hostname/TLS), usually owned by platform or cluster operators.
  • HTTPRoute: app-level routing rules (host/path/header/query matches, filters, backends).
  • ReferenceGrant: explicit handshake for cross-namespace references (for example Route → Service in another namespace, or Gateway → Secret in another namespace). Without a grant, the reference is invalid by design.

Status conditions & events change your debugging workflow

Gateway API leans heavily on status conditions so you can quickly see why something does not work: was the object accepted by the controller, were references resolved, was the dataplane programmed?

SIG Network GEP on status/conditions (GEP-1364) explains Gateway API conditions as positively polarized for the happy state: Accepted, ResolvedRefs, and Programmed (and Ready as Extended). Error conditions are negatively polarized and appear when they are True. The exact condition set can vary by controller and feature level, but Accepted, ResolvedRefs, and Programmed appear in practice across most implementations.

Practical difference versus Ingress debugging: less “controller logs + guessing,” more “kubectl get/describe + status.parents + events,” and only then deeper controller investigation.

Hands-on lab: Gateway API in kind

Goal: in under 30 minutes, get a working lab with host-based, path-based routing and weighted traffic split (canary). This is copy/paste and laptop-friendly.

Requirements

  • Docker
  • kind
  • kubectl
  • Helm 3
  • curl

(You do not need a LoadBalancer in kind: we test via kubectl port-forward, just like in the Envoy Gateway quickstart.)

Controller choice: why Envoy Gateway for this lab?

For a lab you want three things: fast setup, clear docs, and support for standard features such as weighted backendRefs.

  • Envoy Gateway: quickstart is a single Helm install plus port-forward flow, with explicit docs for HTTP routing and traffic splitting using weight on backendRefs.
  • Alternatives (also valid, but with different setup ergonomics):
    • NGINX Gateway Fabric has a kind quickstart with NodePort mappings in kind config + Helm install.
    • Project Contour can run Gateway API via static or dynamic provisioning; still fine, just a few more moving parts for a short demo.

Controller-specific vs generic:

  • Generic (spec): GatewayClass/Gateway/HTTPRoute/ReferenceGrant semantics, allowedRoutes, matching rules, weight, and status conditions (Accepted/ResolvedRefs/Programmed).
  • Controller-specific: how the dataplane is rolled out (Deployment/DaemonSet), how you get an address in kind (LB/NodePort/port-forward), which extra Policy CRDs exist, and which conformance features (Core/Extended/Implementation-specific) are actually supported.

Step-by-step

Create a kind cluster

kind create cluster --name gwapi-lab
kubectl cluster-info --context kind-gwapi-lab

Install Envoy Gateway

According to the quickstart, install Gateway API CRDs + Envoy Gateway with Helm and wait for the deployment to become Available.

helm install eg oci://docker.io/envoyproxy/gateway-helm \
  --version v1.7.0 \
  -n envoy-gateway-system --create-namespace

kubectl wait --timeout=5m -n envoy-gateway-system deployment/envoy-gateway --for=condition=Available

Expected output (indicative):

kubectl -n envoy-gateway-system get deploy,po

You want one deployment in Available state and pods in Running.

Namespaces + sample app (2 services)

We create an infra namespace for the Gateway and an app namespace with two Services (echo-v1 and echo-v2). The echo image uses the Kubernetes Gateway API demo image path.

kubectl apply -f - <<'YAML'
apiVersion: v1
kind: Namespace
metadata:
  name: gateway-infra
---
apiVersion: v1
kind: Namespace
metadata:
  name: team-a
  labels:
    gw-access: "true"
---
apiVersion: v1
kind: Service
metadata:
  name: echo-v1
  namespace: team-a
  labels:
    app: echo
    version: v1
spec:
  selector:
    app: echo
    version: v1
  ports:
  - name: http
    port: 3000
    targetPort: 3000
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: echo-v1
  namespace: team-a
spec:
  replicas: 1
  selector:
    matchLabels:
      app: echo
      version: v1
  template:
    metadata:
      labels:
        app: echo
        version: v1
    spec:
      containers:
      - name: echo
        image: registry.k8s.io/gateway-api/echo-basic:v20251204-v1.4.1
        ports:
        - containerPort: 3000
        env:
        - name: VERSION
          value: "v1"
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
---
apiVersion: v1
kind: Service
metadata:
  name: echo-v2
  namespace: team-a
  labels:
    app: echo
    version: v2
spec:
  selector:
    app: echo
    version: v2
  ports:
  - name: http
    port: 3000
    targetPort: 3000
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: echo-v2
  namespace: team-a
spec:
  replicas: 1
  selector:
    matchLabels:
      app: echo
      version: v2
  template:
    metadata:
      labels:
        app: echo
        version: v2
    spec:
      containers:
      - name: echo
        image: registry.k8s.io/gateway-api/echo-basic:v20251204-v1.4.1
        ports:
        - containerPort: 3000
        env:
        - name: VERSION
          value: "v2"
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
YAML

Expected output / checks:

kubectl -n team-a get svc,deploy

You should see echo-v1 and echo-v2 services and deployments.

GatewayClass + Gateway (infra-owned)

We create a GatewayClass pointing to Envoy Gateway’s default controllerName gateway.envoyproxy.io/gatewayclass-controller (see quickstart). In real clusters, always verify this matches your controller version.

Then we create one Gateway with listeners for *.example.test, allowing Routes only from namespaces labeled gw-access=true (immediately a governance improvement over from: All). This allowedRoutes pattern aligns with the Gateway API security model.

kubectl apply -f - <<'YAML'
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: eg
spec:
  controllerName: gateway.envoyproxy.io/gatewayclass-controller
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: shared-gw
  namespace: gateway-infra
spec:
  gatewayClassName: eg
  listeners:
  - name: http
    protocol: HTTP
    port: 80
    hostname: "*.example.test"
    allowedRoutes:
      namespaces:
        from: Selector
        selector:
          matchLabels:
            gw-access: "true"
YAML

Expected output / checks:

kubectl -n gateway-infra get gateway shared-gw
kubectl -n gateway-infra describe gateway shared-gw
kubectl get gatewayclass eg -o jsonpath='{.spec.controllerName}'; echo

What to look for:

  • Accepted=True on the Gateway (the controller accepted it).
  • In kind, you will often see the top-level Gateway condition as Programmed=False with reason AddressNotAssigned. This happens because Envoy Gateway creates a LoadBalancer Service for the dataplane by default, and kind has no external load balancer, so EXTERNAL-IP stays <pending>.
  • That does not mean routing is broken. In that case, check:
    • status.listeners[].conditions contains Programmed=True and Accepted=True, and/or
    • the proxy Service for this Gateway exists (see below) and can be port-forwarded.

(Optional) Proxy Service check:

kubectl -n envoy-gateway-system get svc \
  --selector=gateway.envoyproxy.io/owning-gateway-namespace=gateway-infra,gateway.envoyproxy.io/owning-gateway-name=shared-gw

You should see a Service with TYPE=LoadBalancer and a PORT(S) mapping such as 80:<nodePort>/TCP. In kind, EXTERNAL-IP usually remains <pending>.

Create HTTPRoute (host-based + path-based)

We create one HTTPRoute in team-a that:

  • routes by host on app.example.test
  • splits by path:
    • /v1 → echo-v1
    • /v2 → echo-v2
kubectl apply -f - <<'YAML'
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: app-route
  namespace: team-a
spec:
  parentRefs:
  - name: shared-gw
    namespace: gateway-infra
  hostnames:
  - "app.example.test"
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /v1
    backendRefs:
    - name: echo-v1
      port: 3000
  - matches:
    - path:
        type: PathPrefix
        value: /v2
    backendRefs:
    - name: echo-v2
      port: 3000
YAML

Expected output / checks:

kubectl -n team-a get httproute app-route
kubectl -n team-a get httproute app-route -o yaml

In .status.parents[].conditions, you should at least see Accepted=True and ResolvedRefs=True. This Accepted/ResolvedRefs model (with reasons such as BackendNotFound) is a strong troubleshooting signal. backendRefs point to a Service (not a Deployment/Pod), and port is the Service port (not targetPort).

Traffic splitting / canary (weighted backendRefs)

Weighted routing is part of Gateway API: spec.rules.backendRefs accepts multiple backends with relative weight. Envoy Gateway documents this explicitly in the traffic splitting docs.

We create a second HTTPRoute on another hostname (canary.example.test) with a 90/10 split between v1 and v2:

kubectl apply -f - <<'YAML'
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: canary-route
  namespace: team-a
spec:
  parentRefs:
  - name: shared-gw
    namespace: gateway-infra
  hostnames:
  - "canary.example.test"
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /
    backendRefs:
    - name: echo-v1
      port: 3000
      weight: 90
    - name: echo-v2
      port: 3000
      weight: 10
YAML

Test via port-forward

Without a LoadBalancer in kind, use port-forward to the Envoy service. The quickstart selector works well; add a fallback in case it returns nothing.

export ENVOY_SERVICE=$(kubectl get svc -n envoy-gateway-system \
  --selector=gateway.envoyproxy.io/owning-gateway-namespace=gateway-infra,gateway.envoyproxy.io/owning-gateway-name=shared-gw \
  -o jsonpath='{.items[0].metadata.name}')

kubectl -n envoy-gateway-system port-forward service/${ENVOY_SERVICE} 8888:80

Fallback if ${ENVOY_SERVICE} is empty:

if [ -z "${ENVOY_SERVICE}" ]; then
  ENVOY_SERVICE=$(kubectl -n envoy-gateway-system get svc \
    -o custom-columns=NAME:.metadata.name,TYPE:.spec.type --no-headers \
    | awk '$2=="LoadBalancer" || $2=="NodePort" {print $1; exit}')
fi

if [ -z "${ENVOY_SERVICE}" ]; then
  echo "No suitable Envoy Service found. Check with:"
  echo "kubectl -n envoy-gateway-system get svc"
  exit 1
fi

Test host + path routing:

curl -s -H "Host: app.example.test" http://localhost:8888/v1 | head
curl -s -H "Host: app.example.test" http://localhost:8888/v2 | head

Expect output including host, path, namespace, and pod.

Example (shortened):

{"host":"app.example.test","path":"/v1","namespace":"team-a","pod":"echo-v1-6d7b6c8f9f-abcde"}

Test canary split (run this several times; you should occasionally see a different backend pod):

for i in $(seq 1 30); do
  curl -s -H "Host: canary.example.test" http://localhost:8888/ \
    | grep -o 'echo-v[12]-[^"]*' | head -n1
done

Compatibility note: traffic splitting via weight is spec-defined, but controllers differ in details and conformance. Always verify this in your own dataplane.

Delegation & governance: team autonomy without cluster-admin

Gateway API is explicitly built around personas/roles (infra provider / cluster operator / app developer) so ownership can be separated without turning self-service into all-or-nothing.

Ownership model: who manages what

A pragmatic model that works in many mid/large organizations:

  • Platform team (or cluster operator) manages:
    • GatewayClass(es) (cluster-scoped)
    • Gateways in an infra namespace (listeners, TLS, allowedRoutes, policies)
  • Product/app teams manage:
    • HTTPRoute(s) in their own namespace(s)
    • Services/Deployments (backends)

In Gateway API, TLS termination lives at the Gateway listener level (typically not app-owned), while HTTPRoutes are app-owned.

Namespaces, RBAC, and allowedRoutes: your first guardrail

Route attachment is intentionally a handshake: routes reference a Gateway via parentRefs, but the Gateway listener decides whether routes from other namespaces may attach (allowedRoutes). Kubernetes describes this as a bidirectional trust model.

Safe default for shared Gateways:

  • allowedRoutes.namespaces.from: Selector with a selector on namespace labels
  • Prefer the standard namespace name label (kubernetes.io/metadata.name) or labels only platform can set.
  • Keep gateway-access labels platform-only (RBAC on namespaces update/patch) and apply them through a namespace onboarding flow.

RBAC pattern (sketch):

  • App team gets create/update on HTTPRoute in its own namespace
  • No permissions on Gateways in gateway-infra
  • Platform team manages Gateway/GatewayClass and labels namespaces allowed to attach (or runs a namespace onboarding flow)

ReferenceGrant: cross-namespace without an “open bar”

Cross-namespace references are not allowed by default; the owner of the target object must explicitly grant them. That is exactly what ReferenceGrant is for.

Important nuance: “All cross-namespace references … require a ReferenceGrant” (route attachment to Gateway is a separate mechanism).

Practical example (app team route → shared service):

apiVersion: gateway.networking.k8s.io/v1
kind: ReferenceGrant
metadata:
  name: allow-team-a-to-call-shared
  namespace: shared-services
spec:
  from:
  - group: gateway.networking.k8s.io
    kind: HTTPRoute
    namespace: team-a
  to:
  - group: ""
    kind: Service

This pattern appears directly in Gateway API security guidance.

Golden path: templates + policy checks (without vendor lock-in)

What “golden path” means here: app teams submit HTTPRoute intents within a strict, validated framework.

Concretely:

  • Templates: a standard HTTPRoute skeleton (hostnames, parentRefs, allowed filters) + standard Service port conventions.
  • Policy-as-code checks in commit/PR and admission:
    • “hostnames must belong to team-owned domains”
    • “parentRefs only to approved Gateways”
    • “no cross-namespace backendRefs without ReferenceGrant”
    • “no wildcard hostnames in app namespaces” This can be enforced with Kubernetes ValidatingAdmissionPolicy.

Anti-patterns (where things go wrong)

  1. allowedRoutes.from: All on a shared Gateway in a multi-team cluster (implicit broad attach surface).
  2. No explicit domain delegation per namespace: this invites hostname/domain hijacking scenarios (first-come-first-served conflicts and later hostname expansion).
  3. Allowing cross-namespace refs informally without ReferenceGrant governance: this creates avoidable security risk.

Observability & debugging

You want a standard set of signals, controller-agnostic where possible, plus a few controller-specific places to check.

Signals to use

  1. Status conditions on Gateway/GatewayClass/HTTPRoute
    • GEP-1364 defines the direction: Accepted, ResolvedRefs, Programmed as positive summary conditions, with error conditions when relevant. Exact condition sets can vary by implementation/feature level.
  2. Events (kubectl describe) on Gateway/HTTPRoute: often immediate clues such as NotAllowed, BackendNotFound, NoMatchingListenerHostname, etc.
  3. Controller logs (when status/events are not enough)
    • kind setup example: docker logs cloud-provider-kind
    • Envoy Gateway: logs in envoy-gateway-system (deployment/pods)
  4. Controller-specific metrics (implementation-dependent; check controller docs).
  5. Dataplane debugging gotcha: Envoy Gateway quickstart notes that privileged ports can map to unprivileged ports internally, which can confuse debugging.

Checklist: route does not work

  1. Verify controller is running (kubectl -n envoy-gateway-system get po,deploy).
  2. kubectl get gatewayclass and check status/Accepted.
  3. kubectl -n gateway-infra describe gateway shared-gw
    • Accepted=True? Programmed=True? Address present?
  4. kubectl -n team-a get httproute app-route -o yaml
    • status.parents[].conditions: Accepted, ResolvedRefs?
  5. Hostname match: HTTPRoute hostnames must match Gateway listener hostname.
  6. allowedRoutes match: is your route namespace allowed by selector? allowedRoutes semantics (All|Selector|Same) are in the Gateway API spec reference.
  7. Backend exists + correct port: BackendNotFound / ResolvedRefs=False is a common signal.
  8. Cross-namespace backendRefs? Then verify ReferenceGrant in the target namespace.
  9. Controller-specific constraints: for Envoy Gateway, backendRefs are currently primarily Service-only; invalid backendRefs can result in HTTP 500 for the traffic portion routed there (with traffic splitting).
  10. Then check controller logs + reconcile errors.

Common failure modes

  • Hostname mismatch (Gateway listener vs HTTPRoute hostnames): route exists but is ignored.
  • Namespace not allowed by allowedRoutes selector: Accepted=False / NotAllowedByListeners.
  • Missing ReferenceGrant for cross-namespace backendRefs: ResolvedRefs=False or controller rejection; by design.
  • Conflicts / hostname claim hijacking when delegation is too broad and policy checks are missing.
  • Controller quirks: for example Envoy Gateway privileged-port mapping behavior.

Migration plan without a big-bang

The Gateway API migration guide helps with concept mapping and manual conversion, but does not fully cover live migration and controller-specific behavior. That is exactly where migration strategy and governance matter.

Here is a plan that works in real clusters.

Run Ingress and Gateway in parallel

  • Keep your existing Ingress controller running while you roll out a Gateway API controller in parallel.
  • Use Gateway API for new routes first, or for a limited subset of existing routes.
  • Important: the Ingress API does not disappear from Kubernetes (frozen but still GA), so parallel run is a legitimate transition step.

Canary routes / limited hostnames

Start with low blast radius:

  • Pick a subdomain or dedicated hostname set, for example gw.example.com or *.gw.example.com, and route only that through Gateway.
  • Or use a limited set of apps/teams as early adopters.

Gateway API supports traffic splitting with weights on HTTPRoute backendRefs, which you can use for canary rollouts.

If your controller does not yet support weights:

  • Use host-based canary (canary.app.example.com) or header-based routing (if supported) as an alternative, and keep the split mechanism outside the dataplane (for example weighted DNS records). Controller support varies, so always verify implementation/conformance.

TLS migration: certificates and secrets

  • Move TLS termination explicitly to Gateway listeners and inventory which certs currently terminate on Ingress.
  • Decide where TLS Secrets are managed (for example cert-manager or manual) and migrate them in a controlled way.
  • Use ReferenceGrant if you need cross-namespace Secret references; without a grant, the reference remains invalid.
  • Validate at minimum: SNI/hostname matching, full certificate chain, and renewal behavior.

Make hostname ownership explicit

  • Reserve domain slices per team (for example team-a.example.com).
  • Enforce this with admission/policy: hostnames must stay in team scope, no wildcard hostnames in app namespaces, and only approved parentRefs.
  • Define conflict-resolution rules (who wins on overlap) before teams publish routes independently.

Rollback plan

Rollback should be boring:

  • Keep Ingress resources intact and switch traffic back via:
    • DNS back to the old entry point, or
    • load balancer/service switch (infra dependent), or
    • hostnames pointing back to the Ingress endpoint.
  • Reuse the same acceptance checks (see below) to validate rollback.

When are you “done”

Define explicit acceptance criteria, otherwise migration drifts.

  • Functional:
    • All external routes migrated and validated (host/path/TLS/redirects/etc).
    • No remaining dependency on controller-specific Ingress annotations for core routing.
  • Governance:
    • Gateway (infra) ownership and HTTPRoute (app) ownership are clear, including RBAC and namespace onboarding flow.
    • allowedRoutes and, where needed, ReferenceGrant policies are in place as guardrails.
  • Operability:
    • Runbooks for “route does not work,” standard dashboards/alerts for gateway dataplane, and clear escalation path to platform team.
  • Decommission:
    • If you used community ingress-nginx, explicitly plan around the March 2026 retirement timeline and security posture.

Further Reading

Conclusion

Gateway API is primarily an operational upgrade: less “everything in one Ingress object,” more explicit contracts between infra and apps, with status-driven debugging and stronger multi-team guardrails.

Summary in 5 bullets:

  • Ingress is stable but frozen; Kubernetes recommends Gateway for new capabilities and better organization fit.
  • Gateway API separates ownership: GatewayClass/Gateway infra-owned, HTTPRoute app-owned; this matches real team structures.
  • allowedRoutes + ReferenceGrant provide a real multi-tenant security handshake (not an open bar).
  • Status conditions (Accepted, ResolvedRefs, Programmed) are your primary debug interface.
  • Migration without big-bang: run in parallel, start with limited hostnames/canary weights, and make “done” measurable.

Recurring server or deployment issues?

I help teams make production reliable with CI/CD, Kubernetes, and cloud—so fixes stick and deploys stop being stressful.

Explore DevOps consultancy

  1. The Kubernetes Node Readiness Controller: never schedule pods on half-ready nodes

    The Node Readiness Controller (NRR) stops Kubernetes from scheduling pods on nodes that are not truly ready by declaratively managing taints based on custom node conditions.

    1989 words
  2. KEDA autoscaling on Kubernetes in production: architecture, trade-offs, and best practices

    KEDA brings event-driven autoscaling to Kubernetes by connecting external signals such as queue backlog and consumer lag to HPA. This article covers architecture, behavior, trade-offs, failure modes, and practical production design principles.

    4830 words
  3. Kubernetes multi-tenant governance: managing multi-tenant Kubernetes clusters

    Kubernetes multi-tenancy can save costs, but it requires governance. This article goes deep on soft vs. hard multi-tenancy, risks (noisy neighbors, privilege escalation), and best practices such as RBAC, resource quotas, network isolation, policy-as-code and cost allocation.

    6624 words