Abstract:
Multicast data dissemination is the performance bottleneck for high-performance data analytics applications in cluster computing, because terabytes of data need to be dis...Show MoreMetadata
Abstract:
Multicast data dissemination is the performance bottleneck for high-performance data analytics applications in cluster computing, because terabytes of data need to be distributed routinely from a single data source to hundreds of computing servers. The state-of-the-art solutions for delivering these massive data sets all rely on application-layer overlays, which suffer from inherent performance limitations. This paper presents Blast, a system for accelerating data analytics applications by optical multicast. Blast leverages passive optical power splitting to duplicate data at line rate on a physical-layer broadcast medium separate from the packet-switched network core. We implement Blast on a small-scale hardware testbed. Multicast transmission can start 33ms after an application issues the request, resulting in a very small control overhead. We evaluate Blast's performance at the scale of thousands of servers through simulation. Using only a 10Gbps optical uplink per rack, Blast achieves upto 102× better performance than the state-of-the-art solutions even when they are used over a non-blocking core network with a 400Gbps uplink per rack.
Date of Conference: 26 April 2015 - 01 May 2015
Date Added to IEEE Xplore: 24 August 2015
Electronic ISBN:978-1-4799-8381-0
Print ISSN: 0743-166X
Keywords assist with retrieval of results and provide a means to discovering other relevant content. Learn more.
- IEEE Keywords
- Index Terms
- High-performance Applications ,
- Data Analytics Applications ,
- Optical Power ,
- Core Network ,
- Power Splitting ,
- Network Packets ,
- Small Overhead ,
- Group Membership ,
- Power Grid ,
- Control Network ,
- Beam Splitter ,
- Optical Path ,
- Optical Transmittance ,
- Traffic Volume ,
- Optical Amplifier ,
- Output Ports ,
- Data Block ,
- Packet Loss ,
- Optical Switching ,
- Hadoop Distributed File System ,
- Control Plane ,
- Optical Networks ,
- Traffic Demand ,
- Optical Channel ,
- Matching Problem ,
- Transmission Speed ,
- Node Failure ,
- Link Budget ,
- Background Flow
Keywords assist with retrieval of results and provide a means to discovering other relevant content. Learn more.
- IEEE Keywords
- Index Terms
- High-performance Applications ,
- Data Analytics Applications ,
- Optical Power ,
- Core Network ,
- Power Splitting ,
- Network Packets ,
- Small Overhead ,
- Group Membership ,
- Power Grid ,
- Control Network ,
- Beam Splitter ,
- Optical Path ,
- Optical Transmittance ,
- Traffic Volume ,
- Optical Amplifier ,
- Output Ports ,
- Data Block ,
- Packet Loss ,
- Optical Switching ,
- Hadoop Distributed File System ,
- Control Plane ,
- Optical Networks ,
- Traffic Demand ,
- Optical Channel ,
- Matching Problem ,
- Transmission Speed ,
- Node Failure ,
- Link Budget ,
- Background Flow