Most multi-tenant data centers are built in phases, either with multiple data rooms within a single building or with multiple buildings on a campus. This is done for efficient use of capital while also allowing faster access to the site for customers. Staged development brings in revenue sooner to offset the initial capital costs for the operator while also addressing the needs of customers to address the ever-increasing demand for data services.
This planned development and growth of a data center location also creates a need for a scalable cross connect fabric – providing connectivity for a small number of initial customers with the ability to grow to many thousand cross connects upon full use of the site. As data center operators increasingly view automation of the cross connect as a desired capability to avoid the typical problems of manual patch panels, it becomes important to find an automated cross connect system that meets the scalability requirements of the data center.
Since staged growth is done to minimize the initial capital outlay, the same criteria can be applied to the automated cross connect system. The most economical automated system will include a pay-as-you grow feature. This allows the data center operator to match their capital outlay on the automated cross connects with the current demand for interconnection. This pay-as-you-grow design is especially helpful with a large port count system that allows the user to initially populate the system with a subset of the number of ports in the full system.
It is very important to understand the difference between a pay-as-you-grow design in a large port count automated solution with stacking together multiple lower-port-count systems. Four 250 port systems stacked together in a rack do not offer the same functionality as a single 1,000 port solution. The main function of an automated cross-connect system is to provide connectivity between all the ports – allowing any customer to be connected to any other customer. Because the smaller systems stacked together are isolated and do not have any connectivity between them, only customers connected within the individual units can be interconnected. A customer on system 1 has no way to be connected to a customer on other systems stacked in the same rack.
It is possible to use a leaf-spine network to form a larger connectivity matrix from smaller systems. To create a network of 1,000 any-to-any ports using a 250-port system requires twelve systems, 8 as leaf nodes and 4 spine nodes. Each system has 125 customer facing ports and 31 ports connected to each of the 4 spine nodes. This is shown in the figure contrasting this with the simplicity of a 1,000 node system providing any-to-any connectivity.
As can be seen in the figure, it takes 3 times the overall number of ports in the smaller system to equal the same any-to-any connectivity capability as the single larger system. Companies with lower port count systems try to create marketing gimmicks to hide this shortcoming, such as the term “Smart CLOS” network. But Math trumps Marketing. Creating cute marketing phrases doesn’t overcome the limitations of the low port count systems.
To highlight the marketing misinformation touted by some manufacturers, information on a website claims that the product achieves non-blocking end-to-end connections with high efficiency using “inter-switch-links.” The design is to use 20% of the ports in a system to provide “inter-switch links” with systems daisy chained to each other. This is clearly less efficient than the traditional leaf-spine connection because connections between system 1 and system 4 require 3 links. But more importantly, while the marketing claims this approach achieves non-blocking end-to-end connections, calculating the blocking probability for Clos networks gives a > 95% blocking probability when just 50% of the customer connections are active [1]. This can be understood intuitively since for a random distribution of connections, only 25% of the connections will stay within the same system while 75% will be connected to the other systems. With only 20% of the links available for inter-system connections, it is clear that there will be a very high blocking probability when even just a fraction of the customer connections are active.
Fortunately, there is a system that offers what customers want and doesn’t need to create confusion through marketing. The Telescent Network Topology Manager (NTM) offers the largest port count available in an automated cross connect system, a pay-as-you-grow model and has proven reliability through NEBS Level 3 certification. The NTM offers 1,008 ports per system with a patented algorithm to allow any-to-any connectivity at any time in the future. Customers can start with as few as 48 ports and grow in units of 96 ports per module while still maintaining the ability to connect any of the initial ports to any of the newly added ports. And with a leaf-spine Clos network design the Telescent NYM can scale to 10,000 ports or more. The Telescent system can be used to automate connections, disconnections and reconfigurations through a simple Rest API or CLI interface. The Telescent system is manufactured by Flex, one of the top 3 contract manufacturers in the world and is used in production networks today. Please contact Telescent today to learn more about automating cross-connects in your data centers in a way that fits your growth.