Source#

This module provides the base class for data sources and sinks.

class AbstractSource(source=None, shape=None, dtype=None, order=None, location=None, name=None, mode=None)[source]#

Bases: Source

Abstract source to handle data sources without data in memory.

Note

This class handles essential info about a source and to how access its data.

as_buffer()[source]#

as_real()[source]#

as_virtual()[source]#

Return virtual source without array data to pass in parallel processing.

Returns

sourceSource class: The source class without array data.

property dtype#

The data type of the source.

Returns

dtypedtype: The data type of the source.

property location#

The location of the source’s data.

Returns

locationstr or None: Returns the location of the data source or None if there is none.

property order#

The contiguous order of the data array of the source.

Returns

orderstr: Returns ‘C’ for C and ‘F’ for fortran contiguous arrays, None otherwise.

property shape#

The shape of the source.

Returns

shapetuple: The shape of the source.

class Source(name=None, mode=None)[source]#

Bases: object

Base abstract source class.

as_buffer()[source]#

as_memory()[source]#

as_real()[source]#

as_virtual()[source]#

Return virtual source without array data to pass in parallel processing.

Returns

sourceSource class: The source class without array data.

exists()[source]#

read(*args, **kwargs)[source]#

write(*args, **kwargs)[source]#

property dtype#

The data type of the source.

Returns

dtypedtype: The data type of the source.

property is_persistable#: True if changes can be written back to disk (mode ‘r+’ or ‘w+’).

property is_writable#: True if in-memory writes are permitted (mode != ‘r’).

property location#

The location where the data of the source is stored.

Returns

locationstr or None: Returns the location of the data source or None if this source lives in memory only.

property mode#

property name#

The name of this source.

Returns

namestr: Name of this source.

property ndim#

The number of dimensions of the source.

Returns

ndimint: The number of dimension of the source.

property order#

The contiguous order of the underlying data array.

Returns

orderstr: Returns ‘C’ for C contiguous and ‘F’ for Fortran contiguous, None otherwise.

property shape#

The shape of the source.

Returns

shapetuple: The shape of the source.

property size#

The size of the source.

Returns

sizeint: The number of data items in the source.

class VirtualSource(source=None, shape=None, dtype=None, order=None, location=None, name=None, mode=None)[source]#

Bases: AbstractSource

Virtual source to handle data sources without data in memory.

Note

This class is fast to serialize and useful as a source pointer in parallel processing.

as_buffer()[source]#

as_real()[source]#: Default: reopen from location with mode. Override in modules that need extra constructor args.

as_virtual()[source]#

Return virtual source without array data to pass in parallel processing.

Returns

sourceSource class: The source class without array data.

read(*args, **kwargs)[source]#

write(*args, **kwargs)[source]#

property name#

The name of this source.

Returns

namestr: Name of this source.