A recent white paper from data center networks leader Dell (News - Alert) Force10 addresses the fact that “as data centers scale to support thousands of servers, IT managers are seeking better ways to network those servers while reducing costs and power consumption.”

Web search engines require parallel communication with every node in the cluster to provide the most relevant results, the paper explains, adding that Web servers “may require interaction with hundreds of sub-services running on remote nodes.” This paper describes the use of Dell Force10’s core switching system, the Z9000, in a distributed core architecture to address these and other issues. Dell Force10’s Z9000 is a core switch that is purpose-built for distributed core architectures and cost-optimized for scaleout fabric solutions of any size.

Much of the problem, as the paper identifies, is that websites, portals, search engines and analytical applications are dealing with extremely large data sets: “big data,” beyond the ability of commonly used software tools to capture, manage, and process within a tolerable elapsed time.

We’re talking anywhere from a few dozen terabytes to many petabytes of data in a single data set, which require processing methods such as massively parallel processing (MPP) databases, data-mining grids, Apache Hadoop Framework, distributed file systems and others.

“Networking vendors are reacting to these demands with new networking fabric configurations that support large compute clusters,” the paper explains, noting that “traditional hierarchical network designs are useful for certain types of data centers, but they’re not well suited for big data compute-cluster applications.”

There are several advantages to a distributed core architecture in such situations, the paper says. They’re cost-effective because the core can be massively scaled through the use of multiple, low-cost Ethernet switches vs. traditional and expensive chassis-based systems, for one thing, and they’re high-performance with full bisectional bandwidth for any-to-any communication.

Such configurations also tend to be ultra-resilient, since nodes can be restarted or replaced without losing the entire switching fabric.

As Dell Force10 officials say, distributed core architectures provide “greater scalability, higher bandwidth, and higher resiliency as networking foundation for data centers that handle massive amounts of data and employ large-scale compute clusters.”