A dynamic load balance on GPU cluster for fork-join search

As a result of that every computer can have different CPUs, memory size, GPU devices and so on, they are heterogeneous and unreliable, dynamic load balancing is a difficult problem for a GPU cluster system needs to solve. In this paper, we discuss a method that can dispatch the appropriate tasks to each node to achieve load balancing. We assume that each node has an initial capability of hyper-computing, according to number of completed tasks in each cycle; this capability of each node will be updated dynamically. We will also show that how the tasks resend when some nodes disconnect to improve the system's reliability. In our experiments, the load of each computing node can be balanced within a few minutes, and if some nodes disconnect, the computing tasks can be completed normally.