How To Optimize Traffic In Multi-Cloud Kubernetes With Multiple Istio Ingress Gateways

|

As enterprises increasingly adopt multi-cloud strategies, managing traffic efficiently across clusters in different cloud providers becomes a complex but critical task. Kubernetes offers a powerful abstraction for orchestrating containerized workloads, but its native networking stack doesn’t handle multi-cloud traffic routing out of the box. That’s where Istio—a powerful service mesh—comes in. By using multiple Istio Ingress Gateways strategically, organizations can route, balance, and secure traffic between Kubernetes clusters across cloud boundaries.

This article will walk you through how to optimize traffic in a multi-cloud Kubernetes environment using multiple Istio Ingress Gateways, complete with architectural insights and code examples.

Understanding the Multi-Cloud and Multi-Gateway Challenge

A multi-cloud Kubernetes setup typically involves:

  • Multiple clusters in different cloud environments (e.g., AWS, GCP, Azure).

  • Latency and performance variations between inter-cloud traffic.

  • Redundancy and disaster recovery needs.

  • Cross-cluster service communication.

Challenges include:

  • Ensuring low-latency and high-availability routing.

  • Balancing global and regional traffic.

  • Handling failover efficiently.

  • Maintaining secure mTLS connections between services.

Using multiple Istio ingress gateways, each exposed in their respective cloud region, we can intelligently route traffic based on geo-location, health checks, latency, and business logic.

Architecture Overview

The high-level architecture includes:

  • Two or more Kubernetes clusters in different cloud regions.

  • Istio deployed in each cluster with its own ingress gateway.

  • Global DNS or load balancer (e.g., Cloudflare, AWS Global Accelerator) to route initial traffic to the right cluster.

  • Istio VirtualServices and DestinationRules to control routing logic per service.

  • Cross-cluster service discovery (via DNS or mesh expansion).

Deploy Istio in Each Cluster

Install Istio using the istioctl command in each cluster. Here’s an example for cluster A:

bash
istioctl install --set profile=default \
--set values.global.meshID=mesh1 \
--set values.global.multiCluster.clusterName=cluster-a \
--set values.global.network=network-a \
--set values.gateways.istio-ingressgateway.type=LoadBalancer

Repeat for cluster B with respective values.

Configure Cross-Cluster Communication

Each cluster must recognize the others’ services. You can do this via:

  • Endpoint discovery using East-West gateways.

  • Shared service registry (via Istio multi-primary configuration).

Example mesh configuration:

yaml
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: reviews-cluster-b
spec:
addresses:
- 240.0.0.2
hosts:
- reviews.bookinfo.svc.cluster.local
location: MESH_INTERNAL
ports:
- number: 9080
name: http
protocol: HTTP
resolution: DNS
endpoints:
- address: istio-ingressgateway.istio-system.svc.cluster-b.local

This config tells cluster A to route traffic for the reviews service to cluster B’s ingress gateway.

Set Up Multiple Istio Ingress Gateways

Each cluster gets a public-facing ingress gateway.

Let’s expose two separate gateways:

yaml
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
metadata:
name: custom-gateway
namespace: istio-system
spec:
components:
ingressGateways:
- name: istio-ingressgateway
enabled: true
label:
istio: ingressgateway
k8s:
service:
type: LoadBalancer

Apply this with istioctl install -f gateway.yaml.

Now each cluster’s ingress gateway is independently exposed to the internet or load balancers.

Deploy Sample App With VirtualServices

Let’s deploy the Bookinfo sample app in both clusters:

bash
kubectl apply -f samples/bookinfo/platform/kube/bookinfo.yaml

Define VirtualService per cluster to control where traffic should go.

Cluster A (Primary Route):

yaml
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: bookinfo-route
spec:
hosts:
- "*"
gateways:
- istio-ingressgateway
http:
- match:
- uri:
prefix: /productpage
route:
- destination:
host: productpage
port:
number: 9080

Cluster B (Backup or region-specific route):

Modify the VirtualService to use different weights for canary deployment or region preference:

yaml
http:
- route:
- destination:
host: productpage
port:
number: 9080
weight: 80
- destination:
host: productpage.cluster-b.global
port:
number: 9080
weight: 20

Use Global Load Balancer or GeoDNS

To route users to the nearest or healthiest cluster, you can use:

  • Cloudflare Load Balancer

  • AWS Global Accelerator

  • GCP Traffic Director

For example, Cloudflare’s Load Balancer allows geo-based routing:

json
{
"proxied": true,
"steering_policy": "geo",
"default_pools": ["cluster-a", "cluster-b"],
"region_pools": {
"NA": ["cluster-a"],
"EU": ["cluster-b"]
}
}

Enable mTLS Across Clusters

Istio can enforce secure communication between clusters using mutual TLS.

Enable strict mTLS:

yaml
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
name: default
namespace: istio-system
spec:
mtls:
mode: STRICT

Make sure both ingress gateways have correct certificates (use SPIFFE or Istio’s built-in CA).

Implement Health Checks and Failover

Use DestinationRule with outlierDetection to automatically shift traffic when one cluster is unhealthy.

yaml
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
name: productpage-dr
spec:
host: productpage
trafficPolicy:
outlierDetection:
consecutive5xxErrors: 2
interval: 5s
baseEjectionTime: 30s

This will eject bad endpoints from the pool.

Observability Across Clusters

Use Istio’s telemetry stack (Prometheus, Kiali, Grafana) per cluster, or federate metrics into a central observability layer.

You can deploy Prometheus Federation or use tools like Thanos or Grafana Cloud.

Example Prometheus scrape config:

yaml
- job_name: 'istio-mesh'
scrape_interval: 15s
honor_labels: true
static_configs:
- targets: ['istio-ingressgateway.istio-system.svc.cluster.local:15090']

Conclusion

Optimizing traffic in a multi-cloud Kubernetes environment using multiple Istio ingress gateways is a powerful strategy to improve performance, resilience, and global user experience. By deploying Istio gateways in each cluster, configuring VirtualServices and DestinationRules for granular routing, and leveraging global load balancers or DNS-based steering, you can achieve intelligent cross-cloud traffic management.

The core benefits of this architecture include:

  • Performance Optimization: Route users to the nearest or fastest cluster.

  • High Availability: Failover seamlessly between clusters when outages occur.

  • Security: Enforce mTLS and consistent authentication across clouds.

  • Scalability: Grow and manage services independently per cloud or region.

  • Observability: Gain deep insights into traffic behavior across the mesh.

However, success depends on careful setup: managing DNS resolution, certificate trust between clusters, and consistent configuration practices using GitOps or CI/CD pipelines. As enterprises grow their cloud presence, multi-gateway Istio setups are not just an option—they become essential for operational excellence.