In reply to Thomas Ponweiser.

There has been quite a bit of research over the years about redundant computation; such work also has touches upon distributed computing (vs. parallel computing -- both are similar, but have different characteristics). The idea of computing some unit of work N times and then deciding which is the "right" answer, or perhaps only *conditionally* re-computing work when it fails to be performed within a timeout (e.g., a manager/worker-style model) -- these are both common fault tolerant schemes.

At least one problem with these approaches, however, is that they tend to be expensive: e.g., you have to buy additional hardware to perform the same amount of work. There can also be scalability issues (e.g., to scale manager/worker models, you really need multiple managers, and then the managers need to coordinate with each other, and depending on the coherence algorithms used, they may have difficulty scaling, too).

The problem is that there is no silver bullet that works with all problems. A distributed manager/worker model with a trivial coherence protocol can potentially scale to arbitrary numbers of workers, but data movement can then become the scalability bottleneck. Additionally, not all computational problems can fit into a manager/worker paradigm.