Source time Edition date
IndustryOpenAI
Rapidly scaling online storage to serve over 1 billion ChatGPT users (Part II)
快速扩展在线存储以服务超过10亿ChatGPT用户（第二部分）
ChatGPT's 12× faster reads show the engineering cost behind 1 billion users, a tradeoff in multi-tenancy and caching that infrastructure teams must weigh.
Judgment
OpenAI's Habitat storage team now decides how much per-tenant isolation and cache coherence to trade for keeping global caching in place: the 12× faster reads come from sharing cache and metadata paths across tenants while still serving 1 billion ChatGPT users.
Basis
OpenAI's own Part II engineering post makes the claim: Habitat was scaled with "reliable multi-tenancy, global caching, and Azure Cosmos DB" to deliver 12× faster reads for ChatGPT, at a scale of over 1 billion users. The mechanism is the interesting part. Multi-tenancy packs many workloads onto shared storage so cache entries and metadata lookups get reused rather than recomputed per tenant, global caching puts that reused data close to whichever request needs it, and Cosmos DB carries the metadata layer that keeps tenants addressable at that fan-out. That is where the cost sits: every unit of cross-tenant reuse is a unit of isolation and consistency you no longer get for free, so the 12× is a property of the shared design, not a free upgrade. The judgment for infrastructure leads is therefore not whether to adopt caching but where to draw the line between reuse breadth and blast radius — a call the post attributes to engineering choices, not to any stated SLA.
Unclear The source gives no cache hit ratio, no absolute latency figures, and never says how tenant isolation is actually enforced — per-tenant keys, partitioning scheme, or dedicated pools are all unstated.
Next Check OpenAI's next Habitat post or talk for a reported cache hit rate or an explicit per-tenant isolation mechanism.