Abstract:
Graph Neural Networks (GNN) have evolved as powerful models for graph representation learning. Sampling-based training methods have been introduced to train large graphs ...Show MoreMetadata
Abstract:
Graph Neural Networks (GNN) have evolved as powerful models for graph representation learning. Sampling-based training methods have been introduced to train large graphs without compromising accuracy. However, it is challenging for the existing GNN systems to effectively utilize multi-core accelerators, especially GPUs, due to a large number of atomic operations and unbalanced workload originating from the serial execution of multiple GNN processing stages. In this paper, we propose a combination of optimization techniques to accelerate the end-to-end performance of the sampling-based GNN training process. Specifically, we propose an adaptive shared memory-based sampling technique and a degree-guided thread block scheduling strategy to optimize the graph sampling. Further, based on the observations of resource demand in different training stages, we propose an asynchronous pipeline-based scheduling method, which accelerates the GNN training by decoupling different training stages into a pipeline and therefore improves the GPU resource utilization significantly. The experimental results show that compared with the existing work, the proposed methods can achieve up to 5.6X performance speedup in the end-to-end performance.
Published in: IEEE Transactions on Computers ( Volume: 72, Issue: 9, 01 September 2023)
Funding Agency:
Keywords assist with retrieval of results and provide a means to discovering other relevant content. Learn more.
- IEEE Keywords
- Index Terms
- Graph Neural Networks ,
- Graph Neural Networks Training ,
- Resource Utilization ,
- Training Stage ,
- Demand For Resources ,
- Large Graphs ,
- Scheduling Method ,
- Linearizable ,
- Sampling-based Methods ,
- Sample Processing ,
- Batch Size ,
- Low Use ,
- Sample Stage ,
- Nodes In The Graph ,
- Transfer Characteristics ,
- Vertices ,
- Serialized ,
- Vertex Degree ,
- Global Operations ,
- Performance Bottleneck ,
- Shared Memory ,
- Graph Neural Network Model ,
- Resource Contention ,
- Adaptive Optimization ,
- Edge List ,
- Individual Streams ,
- CPU Memory ,
- Load Imbalance ,
- Memory Bandwidth ,
- Memory Capacity
- Author Keywords
Keywords assist with retrieval of results and provide a means to discovering other relevant content. Learn more.
- IEEE Keywords
- Index Terms
- Graph Neural Networks ,
- Graph Neural Networks Training ,
- Resource Utilization ,
- Training Stage ,
- Demand For Resources ,
- Large Graphs ,
- Scheduling Method ,
- Linearizable ,
- Sampling-based Methods ,
- Sample Processing ,
- Batch Size ,
- Low Use ,
- Sample Stage ,
- Nodes In The Graph ,
- Transfer Characteristics ,
- Vertices ,
- Serialized ,
- Vertex Degree ,
- Global Operations ,
- Performance Bottleneck ,
- Shared Memory ,
- Graph Neural Network Model ,
- Resource Contention ,
- Adaptive Optimization ,
- Edge List ,
- Individual Streams ,
- CPU Memory ,
- Load Imbalance ,
- Memory Bandwidth ,
- Memory Capacity
- Author Keywords