DRGC: A Dynamic Redundant Gradient Coding Method

To solve the Straggler problem which seriously affects computational efficiency in distributed gradient descent (DGD), we design a dynamic redundant gradient coding method(DRGC) based on Reed-Solomon coding with dynamic code words and heartbeat delay-aware mechanism that can monitor the computational progress of workers. Traditional coding method ensures the straggler tolerance through a large of redundant computations, but can’t adapt to the change of cluster state. In contrast, DRGC can flexibly adapt to the change by changing the code words base on the cluster state that heartbeat delay-aware mechanism has monitored and determined. Experiments prove that compared with the RS gradient coding, DRGC can further reduce training time.