ClearMap.ParallelProcessing#

This sub-package provides functions for parallel processing of data for ClearMap.

The main functionality is to dsitribute Blocks of data to be processed in parallel manged by ClearMap.ParallelProcessing.BlockProcessing.

The DataProcessing sub-module contains cython implementations for a set of functions to be applied to large arrays in parallel with out sub-dividing the data into blocks but using memory mapping and the buffer interface.

Sub-packages and modules#

ClearMap.ParallelProcessing.Block

The Block module provides a Source class used in parallel processing of very large arrays in ClearMap.ParallelProcessing.BlockProcessing.

ClearMap.ParallelProcessing.BlockProcessing

Module to process data in parallel for large data sets

ClearMap.ParallelProcessing.DataProcessing

This sub-package provides functions for parallel processing of large data for functions that cannot be split into blocks or that have a more optimal way of parallelization.

ClearMap.ParallelProcessing.ParallelTraceback

Decorator to traceback errors in parallel processes.

ClearMap.ParallelProcessing.ProcessWriter

Provides formatting tool to print text with parallel process header.

ClearMap.ParallelProcessing.SharedMemoryArray

Shared ctype memory arrays.

ClearMap.ParallelProcessing.SharedMemoryManager

Shared memory array manager for parallel processing using ctype shared arrays in SharedMemoryArray.

`ClearMap.ParallelProcessing.Block`	The Block module provides a `Source` class used in parallel processing of very large arrays in `ClearMap.ParallelProcessing.BlockProcessing`.
`ClearMap.ParallelProcessing.BlockProcessing`	Module to process data in parallel for large data sets
`ClearMap.ParallelProcessing.DataProcessing`	This sub-package provides functions for parallel processing of large data for functions that cannot be split into blocks or that have a more optimal way of parallelization.
`ClearMap.ParallelProcessing.ParallelTraceback`	Decorator to traceback errors in parallel processes.
`ClearMap.ParallelProcessing.ProcessWriter`	Provides formatting tool to print text with parallel process header.
`ClearMap.ParallelProcessing.SharedMemoryArray`	Shared ctype memory arrays.
`ClearMap.ParallelProcessing.SharedMemoryManager`	Shared memory array manager for parallel processing using ctype shared arrays in `SharedMemoryArray`.