Despite the greater efficiency of the newer networks of the hyperscalers, as their infrastructure continues to grow in size, they will gradually start grappling with similar problems (although not necessarily even close to the same degree) as traditional ISPs. Even in early 2016, we wrote: “[T]he emphasis on DCI ‘throwaways’ may be exaggerated – and that [optical] overbuilding may be a means to keep capital costs down, as there would continue to be growth with the older generation deployments. For example, we have learned that certain transport gear installed about three years ago in Google’s network has not been removed, yet.” Besides dealing with multiple layers of networking, another concern for operators, such as Verizon, has been to avoid “putting too many eggs in one basket,” which especially hit home after 9/11. One of Microsoft’s ideas, which still remains only in the investigation stage, may be slightly ironic, given its historical affinity to a certain extent towards a conventional mindset. In fibeReality’s opinion, to flatten the spine-leaf architecture, comparable disastrous consequences in case of a failure cannot be completely ruled out. We will also look at the situation that although all of the big four hyperscalers have different requirements, the demands of Microsoft at times are by far the most strict and distinctive.
The reason to be at least a little bit skeptical of Microsoft’s plan, is one of the advantages of having the Top of Rack switch with lots of spines (and then the super-spine layers as well), is that the blast radius of the ToR is really low. While the corporation’s conception results in simplification, in going say, from eight switches down to only two devices, there still seems to be a lot of capacity going into one spot.
Ordinarily, if a ToR switch fails, it is not considered that big of a deal. In contrast, with the flatter scheme, it could be a very large event with so much bandwidth at stake — even potentially creating a major black hole in the network does not appear to be an impossibility. In addition, while there is impressive power and latency reduction, it is not clear at all that the model results in any capital outlay initially being saved. Certainly, the eventual replacement of the ToR switches with patch panels would substantially lower expenditures.
Although we have made the point that if the direction of Microsoft was compelling to the other big hyperscalers, they would likely follow, there is an ample number of precedents that the software goliath has been alone in moving in particular directions, such as apparently its development efforts with COBO. Also, the other three players have not deployed Inphi’s COLORZ offering. Evidently, the massive push by Jeff Cox and his network architecture team to move the industry in the direction of such stand-alone, optical modules was not persuasive (and perhaps partially, there was not enough of a critical mass of short-reach applications at the other customers to encourage purchases of these components).
Moreover, Microsoft tends to be extremely exacting when it comes to particular features, such as its preference for pigtails in general, or specific to this flatter architecture notion, a desire for a 16-fiber ribbon matrix cable. We suspect that the modular 8 and modular 16 would be satisfactory solutions for the others.
If Microsoft continues to seriously consider a plan to flatten its switch architecture, we would expect the company to release an RFQ concerning the design in the first half of 2019. It could be over two years before the concept would become a reality at the firm. As to if the risk of so much capacity in a single location becomes an issue or whether ample contingency plans for redundancy would need to be developed at Microsoft, remains to be seen.
As always, fibeReality does not recommend any securities, and this writer does not invest in any companies being analyzed by us.
To follow us on our totally separate, quick update company blog, which is exclusively on fibeReality’s LinkedIn page, please click here.
[written by Mark Lutkowitz]
SHARE THIS