Skip to content

Use multiprocessing.Pool for parallel I/O

Duncan Macleod requested to merge github/fork/duncanmmacleod/table-read-nproc into master

This PR modifies gwpy.io.mp.read_multi to use multiprocessing.Pool (instead of ProcessQueue) for parallel file reading, enabling trivial parallel I/O for astropy registered readers (e.g. ASCII). Files are now passed through the pool one-by-one, so readers in gwpy.table.io only need to be built to handle a single file at a time.

Merge request reports