Parallel computing with R using foreach, future, and other packages

Steve Weston's foreach package defines a simple but powerful framework for map/reduce and list-comprehension-style parallel computation in R.

rstudio::conf 2020 programming

Parallel computing with R using foreach, future, and other packages

January 30, 2020
Steve Weston's foreach package defines a simple but powerful framework for map/reduce and list-comprehension-style parallel computation in R. One of its great innovations is the ability to support many interchangeable back-end computing systems so that *the same R code* can run sequentially, in parallel on your laptop, or across a supercomputer. Recent new packages like future package define elegant new programming approaches that can use the foreach framework to run across a wide variety of parallel computing systems. This talk introduces the basics of foreach and future packages with examples using a variety of back-end systems including MPI, Redis and R's default parallel package clusters.

About the speaker

Bryan Lewis
/

A mathematician well-known to the R and other open-source software communities, Bryan has worked on many applied math projects in computational finance, health care, genomics, and other fields over the years. Bryan often thinks about methods that help simplify computation of large-scale problems and is the coauthor with Taylor Arnold and Michael Kane of the CRC Press textbook “A Computational Approach to Statistical Learning.” Bryan is also an avid kayaker, amateur mycologist and forager. His R packages include irlba and threejs.