A Fundamental Storage-Communication Tradeoff for Distributed Computing with Straggling Nodes - Equipe Communications numériques Accéder directement au contenu
Article Dans Une Revue IEEE Transactions on Communications Année : 2020

A Fundamental Storage-Communication Tradeoff for Distributed Computing with Straggling Nodes

Résumé

Placement delivery arrays for distributed computing (Comp-PDAs) have recently been proposed as a framework to construct universal computing schemes for MapReduce-like systems. In this work, we extend this concept to systems with straggling nodes, i.e., to systems where a subset of the nodes cannot accomplish the assigned map computations in due time. Unlike most previous works that focused on computing linear functions, our results are universal and apply for arbitrary map and reduce functions. Our contributions are as follows. Firstly, we show how to construct a universal coded computing scheme for MapReduce-like systems with straggling nodes from any given Comp-PDA. We also characterize the storage and communication loads of the resulting scheme in terms of the Comp-PDA parameters. Then, we prove an information-theoretic converse bound on the storage-communication (SC) tradeoff achieved by universal computing schemes with straggling nodes. We show that the information-theoretic bound matches the performance achieved by the coded computing schemes with straggling nodes corresponding to the Maddah-Ali and Niesen (MAN) PDAs, i.e., to the Comp-PDAs describing Maddah-Ali and Niesen's coded caching scheme. Interestingly, the MAN-PDAs are optimal for any number of straggling nodes. This implies that the map phase of optimal coded computing schemes does not need to be adapted to the number of stragglers in the system. We show that the points that lie exactly on the fundamental SC tradeoff cannot be achieved with Comp-PDAs that require smaller number of files than the MAN-PDAs. This is however possible for some of the points that lie close to the SC tradeoff. For these latter points, the decrease in the requested number of files can be exponential in the number of nodes of the system. We also model the total execution time, and numerically show that the active set size should be chosen to balance the duration of the map phase and the durations of the shuffle and reduce phases.
Fichier principal
Vignette du fichier
A Fundamental Storage-Communication Tradeoff for Distributed Computing with Straggling Nodes.pdf (844.32 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-02940396 , version 1 (16-09-2020)

Identifiants

Citer

Qifa Yan, Michèle Wigger, Sheng Yang, Xiaohu Tang. A Fundamental Storage-Communication Tradeoff for Distributed Computing with Straggling Nodes. IEEE Transactions on Communications, 2020, ⟨10.1109/TCOMM.2020.3020549⟩. ⟨hal-02940396⟩
48 Consultations
52 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More