Parallel computing with R using foreach, future, and other packages - Bryan Lewis

January 30, 2020 Bryan Lewis
Steve Weston's foreach package defines a simple but powerful framework for map/reduce and list-comprehension-style parallel computation in R. One of its great innovations is the ability to support many interchangeable back-end computing systems so that *the same R code* can run sequentially, in parallel on your laptop, or across a supercomputer. Recent new packages like future package define elegant new programming approaches that can use the foreach framework to run across a wide variety of parallel computing systems. This talk introduces the basics of foreach and future packages with examples using a variety of back-end systems including MPI, Redis and R's default parallel package clusters.

About the Author

Bryan Lewis

A mathematician well-known to the R and other open-source software communities, Bryan has worked on many applied math projects in computational finance, health care, genomics, and other fields over the years. Bryan often thinks about methods that help simplify computation of large-scale problems and is the coauthor with Taylor Arnold and Michael Kane of the CRC Press textbook "A Computational Approach to Statistical Learning." Bryan is also an avid kayaker, amateur mycologist and forager. His R packages include irlba and threejs.

More Content by Bryan Lewis
Previous Video
Future: Simple Async, Parallel & Distributed Processing in R - What's Next? - Henrik Bengtsson
Future: Simple Async, Parallel & Distributed Processing in R - What's Next? - Henrik Bengtsson

Future is a minimal and unifying framework for asynchronous, parallel, and distributed computing in R.

Next Video
Getting things logged - Gergely Daroczi
Getting things logged - Gergely Daroczi

One of the greatest strength of R is the ease and speed of developing a prototype (let it be a report or da...