OpenAI explains that its Habitat infrastructure now runs on Azure Cosmos DB, enabling it to serve more than one billion ChatGPT users while reducing global read latency from roughly 120 ms to 10 ms.
OpenAI’s Habitat platform has migrated its storage layer to Azure Cosmos DB, a move that the company says cuts read latency for its massive ChatGPT user base from around 120 ms to just 10 ms.
Why Azure Cosmos DB?
Azure Cosmos DB offers globally distributed, multi‑model capabilities that let Habitat store and retrieve data close to users wherever they are. The service’s low‑latency reads and automatic replication align with OpenAI’s need to serve over a billion active ChatGPT sessions simultaneously.
Performance gains in practice
OpenAI reports that the new backend reduces the average read latency by an order of magnitude, dropping from roughly 120 ms to 10 ms. This improvement translates into faster response times for end‑users and lower computational overhead for the inference layer.
The latency reduction also helps mitigate regional disparities, ensuring that users in distant locations experience comparable performance to those near OpenAI’s primary data centers.
Implications for scalability
By leveraging Cosmos DB’s elastic scaling, Habitat can dynamically adjust throughput as demand spikes, supporting future growth beyond the current billion‑plus user count without a major architectural overhaul.
- Global data distribution reduces cross‑region round‑trip times
- Automatic scaling matches traffic patterns
- Built‑in multi‑region consistency guarantees data integrity
OpenAI’s shift underscores a broader industry trend toward cloud‑native, globally distributed databases for AI workloads, where latency and scalability are paramount.
For readers interested in OpenAI’s broader infrastructure strategy, see the OpenAI on Momentum profile and explore related stories in the Infrastructure category.