IO

IO interface to read files as sources.

This is the main module to distribute the reading and writing of individual data formats to the specialized submodules.

See ClearMap.IO for details.

as_source(source_, slicing=None, *args, **kwargs)

Convert source specification to a Source class.

Arguments

source_object

The source specification.

Returns

sourceSource class

The source class.

buffer(source_)

Returns an io buffer of the data array of a source for use with e.g. cython.

Arguments

source_source specification

The source specification.

Returns

bufferarray or memmap

A buffer to read and write data.

convert(source_, sink, processes=None, verbose=False, **kwargs)

Transforms a source into another format.

Arguments

source_source specification

The source or list of sources.

sinksource specification

The sink or list of sinks.

Returns

sinksink specification

The sink or list of sinks.

convert_files(filenames, extension=None, path=None, processes=None, verbose=False, workspace=None, verify=False)

Transforms list of files to their sink format in parallel.

Arguments

filenameslist of str

The filenames to convert

extensionstr

The new file format extension.

pathstr or None

Optional path specification.

processesint, ‘serial’ or None

The number of processes to use for parallel conversion.

verbosebool

If True, print progress information.

Returns

filenameslist of str

The new file names.

create(source_, *args, **kwargs)

Create a data source on disk.

Arguments

source_str, pathlib.Path, array, Source class

The source to write data to.

Returns

sinkstr, array or Source class

The sink to which the data was written.

dtype(source_)

Returns dtype of a source.

Arguments

source_str, array or Source

The source specification.

Returns

dtypedtype

The data type of the source.

element_strides(source_)

Returns the strides of the data array of a source.

Arguments

source_str, array, dtype or Source

The source specification.

Returns

stridestuple of int

The strides of the source.

file_list(expression=None, file_list=None, sort=True, verbose=False)

Returns the list of files that match the tag expression.

Arguments

expression :str

The regular expression the file names should match.

sortbool

If True, sort files naturally.

verbosebool

If True, print warning if no files exists.

Returns

file_listlist of str

The list of files that matched the expression.

filename_to_module(filename)

Returns the IO module associated with a filename.

Arguments

filenamestr

The file name.

Returns

modulemodule

The module that handles the IO of the file.

get_info(d_type)

Get the numpy info object for a data type. (automatically determines if integer or float)

Parameters:

d_type (dtype) – The data type to get the info for.

Returns

info: numpy info object

The info object for the data type.

get_value(source_, value_type)

Get the minimal or maximal value of a source data type.

Parameters:

• source (str, array, dtype or Source) – The source specification.

• value_type (str) – The value type to get, either ‘min’ or ‘max’.

Returns

value: number

The value of the data type.

initialize(source_=None, shape=None, dtype=None, order=None, location=None, memory=None, like=None, hint=None, **kwargs)

Initialize a source with specified properties.

Note

The source is created on disk or in memory if it does not exist so processes can start writing into it.

Arguments

source_str, array, Source class

The source to write data to.

shapetuple or None

The desired shape of the source. If None, inferred from existing file or from the like parameter. If not None and source has a valid shape shapes are tested to match.

dtypetype, str or None

The desired dtype of the source. If None, inferred from existing file or from the like parameter. If not None and source has a valid dtype the types are tested to match.

order‘C’, ‘F’ or None

The desired order of the source. If None, inferred from existing file or from the like parameter. If not None and source has a valid order the orders are tested to match.

locationstr or None

The desired location of the source. If None, inferred from existing file or from the like parameter. If not None and source has a valid location the locations need to match.

memory‘shared’ or None

The memory type of the source. If ‘shared’ a shared array is created.

likestr, array or Source class

Infer the source parameter from this source.

hintstr, array or Source class

If parameters for source creation are missing use the ones from this hint source.

Returns

source: Source class

The initialized source.

initialize_buffer(source_, shape=None, dtype=None, order=None, location=None, memory=None, like=None, **kwargs)

Initialize a buffer with specific properties.

Arguments

source_str, array, Source class

The source to write data to.

shapetuple or None

The desired shape of the source. If None, inferred from existing file or from the like parameter. If not None and source has a valid shape shapes are tested to match.

dtypetype, str or None

The desired dtype of the source. If None, inferred from existing file or from the like parameter. If not None and source has a valid dtype the types are tested to match.

order‘C’, ‘F’ or None

The desired order of the source. If None, inferred from existing file or from the like parameter. If not None and source has a valid order the orders are tested to match.

locationstr or None

The desired location of the source. If None, inferred from existing file or from the like parameter. If not None and source has a valid location the locations need to match.

memory‘shared’ or None

The memory type of the source. If ‘shared’ a shared array is created.

likestr, array or Source class

Infer the source parameter from this source.

Returns

bufferarray

The initialized buffer to use tih e.g. cython.

Note

The buffer is created if it does not exist.

is_source(source_, exists=True)

Checks if source_ is a valid Source.

Arguments

source_object

Source to check.

existsbool

If True, check if source exists in case it has a location.

Returns

is_sourcebool

True if source is a valid source.

location(source_)

Returns the location of a source.

Arguments

source_str, array or Source

The source specification.

Returns

locationstr or None

The location of the source.

location_to_module(location_)

Returns the IO module associated with a location string.

Arguments

location_str or te.Expression or pathlib.Path

Location of the source.

Returns

modulemodule

The module that handles the IO of the source specified by its location.

max_value(source_)

Returns the maximal value of a source data type.

Arguments

source_str, array, dtype or Source

The source specification.

Returns

max_valuenumber

The maximal value for the data type of the source

memory(source_)

Returns the memory type of source_.

Arguments

source_str, array or Source

The source specification.

Returns

memorystr or None

The memory type of the source.

min_value(source_)

Returns the minimal value of a source data type.

Arguments

source_str, array, dtype or Source

The source specification.

Returns

min_valuenumber

The minimal value for the data type of the source

ndim(source_)

Returns number of dimensions of a source.

Arguments

source_str, array or Source

The source specification.

Returns

ndimint

The number of dimensions in the source.

order(source_)

Returns order of a source.

Arguments

source_str, array or Source

The source specification.

Returns

order‘C’, ‘F’, or None

The order of the source data items.

read(source_, *args, **kwargs)

Read data from a data source.

Arguments

source_str, pathlib.Path, array, Source class

The source to read the data from.

Returns

dataarray

The data of the source.

shape(source_)

Returns shape of a source.

Arguments

source_: str, array or Source

The source specification.

Returns

shapetuple of ints

The shape of the source.

size(source_)

Returns size of a source.

Arguments

source_str, array or Source

The source specification.

Returns

sizeint

The size of the source.

source(source_, slicing=None, *args, **kwargs)

Convert source specification to a Source class.

Arguments

source_object

The source specification.

Returns

sourceSource class

The source class.

source_to_module(source_)

Returns IO module associated with a source.

Arguments

source_object

The source specification.

Returns

typemodule

The module that handles the IO of the source.

write(sink, data, *args, **kwargs)

Write data to a data source.

Arguments

sinkstr, pathlib.Path, array, Source class

The source to write data to.

dataarray

The data to write to the sink.

slicingslice specification or None

Optional sub-slice to write data to.

Returns

sinkstr, array or Source class

The sink to which the data was written.

source_modules = [<module 'ClearMap.IO.NPY' from '/home/crousseau/code/icm/ClearMap2-lf003/ClearMap/IO/NPY.py'>, <module 'ClearMap.IO.TIF' from '/home/crousseau/code/icm/ClearMap2-lf003/ClearMap/IO/TIF.py'>, <module 'ClearMap.IO.MMP' from '/home/crousseau/code/icm/ClearMap2-lf003/ClearMap/IO/MMP.py'>, <module 'ClearMap.IO.SMA' from '/home/crousseau/code/icm/ClearMap2-lf003/ClearMap/IO/SMA.py'>, <module 'ClearMap.IO.FileList' from '/home/crousseau/code/icm/ClearMap2-lf003/ClearMap/IO/FileList.py'>, <module 'ClearMap.IO.NRRD' from '/home/crousseau/code/icm/ClearMap2-lf003/ClearMap/IO/NRRD.py'>, <module 'ClearMap.IO.MHD' from '/home/crousseau/code/icm/ClearMap2-lf003/ClearMap/IO/MHD.py'>, <module 'ClearMap.IO.CSV' from '/home/crousseau/code/icm/ClearMap2-lf003/ClearMap/IO/CSV.py'>, <module 'ClearMap.IO.GT' from '/home/crousseau/code/icm/ClearMap2-lf003/ClearMap/IO/GT.py'>]

The valid source modules.