A configuration change caused Google Cloud’s VMware Engine to lose inter‑zone network resilience, disrupting stretched cluster connectivity for some customers.

A recent configuration change inadvertently stripped Google Cloud’s VMware Engine of its inter‑zone network resilience, leaving stretched clusters unable to maintain connectivity across zones for several customers.

What happened

Google Cloud’s VMware Engine provides a fully managed VMware environment that spans multiple availability zones, enabling high‑availability workloads. In mid‑July, an internal update altered the network routing settings that underpin the service’s cross‑zone redundancy.

The change disabled the fail‑over paths that normally allow traffic to reroute if a zone experiences an outage. As a result, clusters configured for stretched deployments lost the ability to communicate between zones, effectively reducing them to single‑zone operation.

Impact on customers

Customers with mission‑critical workloads reported intermittent connectivity failures, increased latency, and in some cases, temporary loss of access to virtual machines. The issue primarily affected enterprises that rely on VMware’s stretched cluster feature for disaster recovery and load balancing.

Google’s status page confirmed that the problem was limited to a subset of regions and that no data loss occurred, but the reduced resilience forced some users to roll back to single‑zone configurations until a fix was applied.

Google’s response

Google Cloud issued an advisory acknowledging the misconfiguration and began rolling out a corrective update within hours of detection. The company also offered affected customers temporary credits and technical assistance to restore their stretched clusters.

In a follow‑up blog post, Google emphasized that the incident highlighted the need for more rigorous testing of network‑related changes in managed services.

  • Review and validate network configuration changes in staging environments
  • Implement automated health checks for cross‑zone connectivity
  • Maintain documentation of fail‑over paths for rapid troubleshooting
We regret the inconvenience caused and are committed to restoring full resilience to VMware Engine as quickly as possible.

For a detailed account of the incident, see The Register’s coverage of Google Cloud’s VMware service issue.