Resource-aware distributed scheduling strategies for large-scale computational cluster/grid systems

In this paper, we propose distributed algorithms, referred to as Resource Aware Dynamic Incremental Scheduling (RADIS) strategies. Our strategies are specifically designed to handle large volumes of computationally intensive arbitrarily divisible loads submitted for processing at Cluster/Grid systems involving multiple sources and sinks (processing nodes). We consider a reallife scenario wherein buffer space (memory) available at the sinks (required for holding and processing the loads) vary over time and the loads have deadlines, and propose efficient "pull-based" scheduling strategies with admission control policy that ensures that the admitted loads are processed satisfying their deadline requirements. The design of our proposed strategies adopts the divisible load paradigm, referred to as divisible load theory (DLT), which is shown to be efficient in handling large volume loads. We demonstrate detailed workings of the proposed algorithms via a simulation study using real-life parameters obtained from a major physics experiment.