Framework
A parallel world
On the outside, a digital ecosystem for millions of users. On the inside, classic infrastructure bought in 2026.
From the outside, this is a group of companies with a mass-market digital product. The app opens an account in a couple of minutes, the service works without queues or branches, and the users number in the millions. It is the showcase of a digital ecosystem in the most modern sense.
On the inside, classic infrastructure is at work: a three-tier architecture, VMware on a VCF subscription, an FC fabric on Brocade, a Cisco network, backup to Data Domain, and a multi-cabinet tape library. Every new need is met with a new class of equipment from a tier-one vendor. Saving on components is not an accepted method here.
The portrait below is a composite in form but quite specific in substance: everything described is available in open sources. We didn’t build this fleet and we don’t run it. We want to understand its logic, not judge someone else’s choices, so we write without irony. This world has its own consistent logic, and it deserves a close look.
Compute
Hosts in batches
Hundreds of hosts, waves of several dozen, one configuration from two manufacturers at once.
The virtualization fleet is measured in hundreds of hosts, and it grows in waves of several dozen identical servers. A typical host from the latest wave is 1U, with two 32-core processors, about 1.5 TB of memory, and not a single data drive. It boots from a mirrored M.2 pair, and two dual-port 32G HBAs and two dual-port 25GbE adapters face outward.
Three details in this description matter.
The first is a host without disks. Data lives on the arrays; the host only computes. A failed server is taken out of the cluster and replaced with a new one with the same profile within an hour. In other words, the host here is a consumable, not a place where data is stored.
A diskless host is a consumable. It gets swapped without a second thought, because the data doesn’t live on it.
The second is that the same configuration is requested from two manufacturers at once. A single purchase contains practically identical specifications for Dell PowerEdge and HPE ProLiant: the same processor, the same amount of memory, the same set of adapters. Whoever offers the better terms and delivery time wins. The VMware cluster doesn’t care whose nameplate is on the chassis, and for purchasing this means protection against shortages and against dependence on a single vendor.
The third is choosing the middle on cores. In issue 08 we wrote that licensing by physical cores pulls toward processors with fewer cores, and in issue 10 that power and density pull the other way. Here the choice is made in the middle: 32 cores per socket, 64 per host. The VCF subscription counts a minimum of 16 cores per processor, so for a wave of 50 hosts the range runs from 1,600 subscription cores on the most modest processors to 9,600 on 96-core ones. The chosen middle gives 3,200, and that is just one wave.
Memory belongs to the same decision. A host has 64 cores and about 1.5 TB of memory, roughly 24 GB per core. The subscription is counted by cores and memory isn’t included, so it pays to max out memory and keep cores in the middle. In issue 08 we called this an unchanged licensing perimeter: VMs get more resources, and the subscription bill doesn’t grow.
Storage network
A fabric that never gets switched off
FC here is not a legacy but a choice: it is being expanded, not run into the ground.
Storage is connected to the hosts over Fibre Channel through two independent fabrics on Brocade switches. The fabric isn’t being rebuilt or replaced with Ethernet; it is being expanded. New switches are current-generation with 32G optics, and ports are added with licenses, 8 at a time. In other words, what grows is the capacity of the same topology that operations, monitoring, and procedures are already used to.
A detail that shows the logic well: the new switches belong to the 64G generation but arrive with 32G optics to match the HBAs installed in the hosts. The fabric speed can be raised later by replacing optics and adapters, without touching the switches themselves. The fabric is bought with room to grow, but you pay for the speed you need now.
In issue 09 we showed that when leaving VMware, the dedicated FC fabric can often be removed and its bandwidth handed over to Ethernet at 25G and 100G. Here the logic is the opposite, and it is consistent. FC delivers predictable latency, physically separates storage traffic from the rest of the network, and relies on skills the team already has and that are easy to find on the market. As long as the core platform is three-tier, there is no reason to give up FC.
This does come at a price, however. First, two networks instead of one: separate switches, separate cabling, and separate support contracts. Second, a separate body of knowledge that has to be kept in the team. This world pays for both deliberately.
Storage
A shelf for every task
A new need is not squeezed into an existing array — a new class is bought for it.
This world has dozens of arrays, and the range of generations runs from hybrid systems with spinning disks to the newest all-NVMe. A new need isn’t squeezed into an existing array: a new one of the right class is bought for it. Judging by open job descriptions, arrays from at least two manufacturers work side by side along with a separate S3 object store, so the heterogeneity here is not only across generations but across vendors too.
The “shelf for every task” principle is also visible in the structure of the group. Each company within it gets its own block array and its own platform for files and objects. In the latest purchases, this is Pure Storage: FlashArray as a pair for block workloads and FlashBlade as a pair for object and file. The top model is installed right away with external fabric modules so that it can later grow to several chassis. Separately, there is a tier on local NVMe — servers with ten 7.68 TB drives each, most likely for vSAN.
VMware is what manages the heterogeneity here. Storage vMotion moves VMs between arrays without downtime, storage policies choose the media class, and the hypervisor acts as the common denominator for everything below it. Without this layer, dozens of arrays would turn into dozens of islands.
The price of this is dozens of lifecycles. Each array has its own support term, its own firmware compatibility matrix, its own console, and its own vendor engineer for escalations. In issue 11 we wrote about a fleet that builds up in layers. Here are the same layers, except each one was bought new and is under support.
Second site
Two replication speeds
Synchronous where a mistake costs seconds of transactions. Asynchronous for everything else.
The second site receives the bulk of the data asynchronously. RPO is measured in minutes, the link doesn’t determine write latency at the primary site, and the distance between sites can be chosen based on risk rather than on the physics of synchronous writes. In issue 07 we calculated the limit of a synchronous design at 50 and at 600 km. For the main fleet, that question simply doesn’t arise here.
Next to it runs a synchronous tier. Arrays with ActiveCluster keep data on two sites at once where losing even a few seconds of transactions costs more than the added write latency. This is the part for which the user opens the app in the first place.
Why does the synchronous tier rely on the arrays rather than on the virtualization layer’s own tools? VCF 9.1 does have a stretched Metro cluster on vSAN ESA. But replication to a remote site with orchestrated failover isn’t part of the base subscription: it requires a separate recovery product or the ACC add-on, which is only available for VCF. When storage already lives on external FC arrays and the arrays can run a synchronous pair themselves, it makes more sense to leave replication to them. The hypervisor keeps VM restarts and failover plans.
In other words, replication is split by the cost of a mistake. The synchronous design is expensive in both hardware and links, so it is applied to the tier where RPO must be zero. Everything else lives on the asynchronous design with clear recovery plans. Money would allow making everything synchronous, but this world doesn’t do that, and that’s a good sign: the split here is driven by engineering logic, not by budget.
Backup
Backup by the book
Backup software, Data Domain at every site, and multi-cabinet tape — the whole protection ladder.
Backup is built the way vendors draw it in their training courses. Backup software takes VM snapshots and writes backups to Data Domain via DD Boost. Deduplication starts on the source side, and only unique blocks go over the network. Each site has its own appliance, and DD Replication sends already-deduplicated data between them. Capacity is bought with perpetual licenses, support for 5 years ahead. The appliances are installed without an internal HA cluster: the backup is the second site, not a second controller.
Long-term retention goes to tape. A large multi-cabinet library is the last step of the ladder we wrote about in issue 13. It is the cheapest per terabyte, the slowest to restore from, and physically separated from everything that can be encrypted remotely.
In issue 13 we left open the question of whether dedupe appliances are needed as a class when deduplication already exists on the media server. In this world, the answer is clear: they are. If the scale is hundreds of hosts and the budget allows, an appliance provides global deduplication across all jobs, predictable ingest speed, and replication that doesn’t need to be managed separately. It’s not a solution for everyone, but within its own logic it is coherent.
Security and delivery
The perimeter and delivery are on the hypervisor too
Firewalls, load balancers, WAF, and guest Linux — all of it rests on VMware.
At the edge are hardware firewalls from the same manufacturer as the network, and their central management server is deployed as a VM on VMware. Load balancers and WAF were bought as virtual appliances and also run on VMware. Guest Linux is licensed on a “per hypervisor host” model: one subscription covers any number of VMs on a dual-socket server.
In other words, VMware here is not just a hypervisor. Security, application delivery, guest OS licensing, and network management all rest on it. Leaving it would be a multi-year program touching every layer, not a VM migration project. That is why this world isn’t leaving: if you count not only the subscription but everything that sits on it, the decision to stay comes out cheaper than the decision to go.
When the perimeter, load balancing, and guest OS licenses live on the hypervisor, leaving is not a migration but a rebuild.
Environments
Test behind a wall
The boundary between test and production is drawn not in a VLAN but in the cabinets.
The test zone is physically separated from the main one: a separate cabinet, its own hosts, its own switches. Logical separation through VLANs and separate clusters is not considered sufficient here.
The logic is clear. A physical boundary rules out test traffic accidentally crossing into production and simplifies audits and conversations with the regulator. Changes go through a test bench that doesn’t share a single port with production.
This boundary does have a price, however. Hardware that is only partly loaded most of the time takes up space, power, and support contracts on a par with production. With hundreds of production hosts, the test cabinet is a small share of the fleet, and this world pays for that share without a second thought.
Practices
Modern inside the classic
Terraform and vanilla Kubernetes on top of VMware: modern doesn’t have to mean new.
The most unexpected detail of the portrait is how this world handles change. Infrastructure is described as code: Terraform with the vSphere provider creates VMs, networks, and policies. Container workloads run in vanilla Kubernetes deployed on ordinary VMs on top of VMware, and storage for pods is provided through vSphere CSI. No Tanzu and no VKS, just clusters on VMs.
There is a practical explanation for this choice. As we wrote in issue 14, standalone Tanzu Kubernetes Grid ended at version 2.5.4, and VKS runs only on top of Supervisor, which ties it to how VMware develops its container layer. Vanilla Kubernetes on ordinary VMs doesn’t depend on that: it can be updated at its own pace and, if desired, moved to another platform together with its manifests and Terraform code.
In issue 14 we argued that Terraform and Ansible survive leaving VMware and that the world doesn’t end with VMware. Here the same point is visible from the other side: modern practices don’t require a new platform. Declarative infrastructure and container orchestration work perfectly well on classic infrastructure. The link between “modern” and “new” is weaker than people tend to think.
AI workloads
Straight to the flagship for AI
No pilot and no intermediate generations — straight to a B300 cluster and a separate 400/800G fabric.
When compute for AI was needed, this world didn’t go through the intermediate steps. The usual path runs from a couple of cards in an ordinary server through a pilot node to a cluster. Here there was no pilot and no H100 or H200 generation: a cluster of fewer than ten nodes with 8 B300 accelerators each is purchased right away, with a choice of two supply options — Cisco UCS and Supermicro.
A separate Ethernet fabric on Cisco Nexus is built for the cluster. The front end runs at 400G in a spine-leaf design, and the existing data center switches join it as leaves. The back end for traffic between accelerators uses 800G optics. Each node gets about 3 TB of memory, local NVMe, and 200G adapters, plus DPUs in the Cisco option.
So two network speeds run in one data center. The main fleet’s hosts are connected over 25GbE, and that’s enough for them because storage goes over the separate FC fabric. The AI cluster lives on 400G and 800G, and its fabric adds nothing to the main fleet apart from the shared leaf switches.
The logic of leapfrogging generations is this. The gap between H100 and B300 in memory and bandwidth is too large for an intermediate purchase to pay off before the next one. Those who buy step by step pay twice and rebuild power twice. Those who can take the flagship right away do it once.
The price comes in kilowatts. In issue 10 we wrote that an air-cooled 8-accelerator node grew from 3.5 to 8.6 kW. By our estimate, a B300 node already draws more than 12 kW, and manufacturers plan for up to 70 kW per rack with such nodes. Multiply by the number of nodes, and a cluster of fewer than ten such machines comes to about 100 kW — more than the four-rack site with five generations of equipment that we examined in issue 11 consumes. We know that far from every in-house or commercial data center has that kind of power feed per rack. For this world, it is a solvable engineering task, not a constraint.
What to adopt
What this world gets right
Predictability, a dual source of supply, a support horizon, and separation by class.
If you strip away everything related to budget, this world still has several decisions worth adopting by anyone.
Predictability over elegance. Every layer is understandable, documented by the vendor, and familiar to any engineer you could hire. There is no exotica here that only one person knows.
Comparison on a single specification. The configuration is put together in advance, for the workload, and only then goes to manufacturers. Bidders compete on delivery time and terms, not on who can cut the specification more. What matters here is not the number of vendors but the fact that the same thing is being compared.
The support horizon is bought up front. Five years of support at the time of purchase cover 3 years of primary operation and almost all of the second round and remove the post-warranty question we wrote about in issue 06. And purchasing waves keep most of the fleet under 6 years old, where on our scale confidence in the equipment doesn’t drop below 85–90%.
Separation by class. A shelf for every task, synchronous replication only for the zero-RPO tier, a physical boundary for test — all of these are examples of one principle: don’t mix what doesn’t have to be mixed.
Modern practices without changing platforms. IaC and containers have been introduced where people already work, without waiting for a big migration.
Limits
Where this world hits its limits
The limits show even with an unlimited budget: cores, heterogeneity, power feed, and cash flow.
Licenses grow with the waves. Each batch of hosts increases the VCF license base in proportion to cores. Under a subscription model, this is an ongoing rather than a one-time expense, and if the vendor revises the price, this expense grows faster than the fleet itself.
Heterogeneity accumulates anyway. A dual source of supply protects purchasing but leads to a mixed fleet: two server lines, two sets of firmware and management tools. Add dozens of arrays of different generations, and you get a situation where each layer is uniform within itself but their combination is complex. Part of the fleet is already addressing this: the new Pure arrays were bought with an Evergreen subscription, where support comes at a fixed rate and controllers and capacity are upgraded without downtime and without replacing the whole array. But this doesn’t extend to the dozens of systems from previous years.
Kilowatts. Two networks instead of one, a physically separate test environment, a multi-cabinet library, and an AI cluster drawing close to a hundred kilowatts make this world constrained by power feed sooner than by space. In issue 10 we put it this way: the rack runs out first. Here, the power feed runs out first.
Dependence on cash flow. In issue 12 we described infrastructure debt. This world has no debt as long as it keeps paying: everything is on subscription, everything is under support, everything is refreshed in waves. But exactly this model accumulates debt faster than any other if the flow stops. Subscriptions expire at the same time, array support ends in a cascade, and new waves stop coming. The resilience of this world is a function of budget, not architecture.
This world has no debt as long as it keeps paying. And no safety margin if it stops.
Checklist
Checklist: what to take without such a budget
Ten points that work even without hundreds of hosts on a subscription.
Summary
In lieu of a conclusion
The parallel world hasn’t fallen behind the market. It has chosen a different axis of optimization: predictability instead of cost of ownership, its own class of equipment for each task instead of standardization, expanding what already works instead of migration.
In one of our upcoming issues we will describe the opposite pole — a fleet assembled from the cheapest equipment that formally meets the requirements. Real customers live between these poles, and almost every one of them will find something to take from both sides.
If your fleet is closer to this world than to that one, and you want to understand where it rests on architecture and where only on budget, let’s work through it together.