Skip to content

Codec

Zarr v3 codecs for array-to-array, array-to-bytes, and bytes-to-bytes transforms.

ArrayToArrayCodec

A Zarr v3 array-to-array codec.

config property 

config: JSONValue | None

The codec's Zarr v3 configuration as a dict, if any.

name property 

name: str | None

The codec's Zarr v3 name (e.g. "transpose"), if any.

decode 

decode(
    value: ArrayBytes,
    /,
    shape: list[int],
    data_type: DataType,
    fill_value: FillValue,
) -> ArrayBytes

Decode chunk bytes with this codec.

Parameters:

  • value (ArrayBytes) –

    The encoded chunk bytes.

  • shape (list[int]) –

    The shape of the encoded chunk, in elements along each dimension.

  • data_type (DataType) –

    The data type of the encoded chunk.

  • fill_value (FillValue) –

    The fill value of the encoded chunk.

Returns:

Raises:

  • CodecError –

    If value does not agree with shape and data_type, or if the codec cannot decode the chunk.

decoded_shape 

decoded_shape(encoded_shape: list[int]) -> list[int] | None

Return the chunk shape that decodes to encoded_shape.

Parameters:

  • encoded_shape (list[int]) –

    The shape of the encoded chunk, in elements along each dimension.

Returns:

  • list[int] | None –

    The shape of the decoded chunk, or None if the codec cannot determine it.

encode 

encode(
    value: ArrayBytes,
    /,
    shape: list[int],
    data_type: DataType,
    fill_value: FillValue,
) -> ArrayBytes

Encode chunk bytes with this codec.

Parameters:

  • value (ArrayBytes) –

    The decoded chunk bytes.

  • shape (list[int]) –

    The shape of the decoded chunk, in elements along each dimension.

  • data_type (DataType) –

    The data type of the decoded chunk.

  • fill_value (FillValue) –

    The fill value of the decoded chunk.

Returns:

Raises:

  • CodecError –

    If value does not agree with shape and data_type, or if the codec cannot encode the chunk.

encoded_data_type 

encoded_data_type(decoded_data_type: DataType) -> DataType

Return the data type that this codec produces when it encodes.

Parameters:

  • decoded_data_type (DataType) –

    The data type of the decoded chunk.

Returns:

  • DataType –

    The data type of the encoded chunk.

encoded_fill_value 

encoded_fill_value(
    decoded_data_type: DataType, decoded_fill_value: FillValue
) -> FillValue

Return the fill value that this codec produces when it encodes.

Parameters:

  • decoded_data_type (DataType) –

    The data type of the decoded chunk.

  • decoded_fill_value (FillValue) –

    The fill value of the decoded chunk.

Returns:

  • FillValue –

    The fill value of the encoded chunk.

encoded_shape 

encoded_shape(decoded_shape: list[int]) -> list[int]

Return the chunk shape that this codec produces when it encodes.

Parameters:

  • decoded_shape (list[int]) –

    The shape of the decoded chunk, in elements along each dimension.

Returns:

  • list[int] –

    The shape of the encoded chunk.

Raises:

  • CodecError –

    If decoded_shape has a number of dimensions that the codec does not support.

from_config staticmethod 

from_config(metadata: JSONValue) -> ArrayToArrayCodec

Construct a codec from its Zarr v3 metadata.

Parameters:

  • metadata (JSONValue) –

    The Zarr v3 metadata of the codec, for example {"name": "transpose", "configuration": {"order": [1, 0]}}.

Returns:

Raises:

  • PluginCreateError –

    If the metadata names an unsupported codec, or if the configuration is not valid for that codec.

ArrayToBytesCodec

A Zarr v3 array-to-bytes codec (the "serializer").

config property 

config: JSONValue | None

The codec's Zarr v3 configuration as a dict, if any.

name property 

name: str | None

The codec's Zarr v3 name (e.g. "bytes", "sharding_indexed"), if any.

from_config staticmethod 

from_config(metadata: JSONValue) -> ArrayToBytesCodec

Construct a codec from its Zarr v3 metadata.

Parameters:

  • metadata (JSONValue) –

    The Zarr v3 metadata of the codec, for example {"name": "bytes", "configuration": {"endian": "little"}}.

Returns:

Raises:

  • PluginCreateError –

    If the metadata names an unsupported codec, or if the configuration is not valid for that codec.

BytesToBytesCodec

A Zarr v3 bytes-to-bytes codec.

config property 

config: JSONValue | None

The codec's Zarr v3 configuration as a dict, if any.

name property 

name: str | None

The codec's Zarr v3 name (e.g. "blosc"), if any.

encode 

encode(decoded_value: bytes) -> bytes

Encode chunk bytes with this codec.

Parameters:

  • decoded_value (bytes) –

    The decoded chunk bytes.

Returns:

  • bytes –

    The encoded chunk bytes.

from_config staticmethod 

from_config(metadata: JSONValue) -> BytesToBytesCodec

Construct a codec from its Zarr v3 metadata.

Parameters:

  • metadata (JSONValue) –

    The Zarr v3 metadata of the codec, for example {"name": "gzip", "configuration": {"level": 5}}.

Returns:

Raises:

  • PluginCreateError –

    If the metadata names an unsupported codec, or if the configuration is not valid for that codec.

CodecChain

A full Zarr v3 codec pipeline.

A codec chain holds:

  • the array-to-array codecs ("filters")
  • one array-to-bytes codec ("serializer")
  • the bytes-to-bytes codecs ("compressors")

The chain applies them in that order to encode, and in the reverse order to decode.

An array carries the codec chain that encodes its chunks. To decode chunk bytes with it, use EncodedChunk, which holds the bytes and the chain together.

compressors property 

compressors: list[BytesToBytesCodec]

The bytes-to-bytes codecs ("compressors").

filters property 

filters: list[ArrayToArrayCodec]

The array-to-array codecs ("filters").

serializer property 

serializer: ArrayToBytesCodec

The array-to-bytes codec ("serializer").

For a sharded array, this is the sharding_indexed codec.

bitround 

bitround(keepbits: int) -> ArrayToArrayCodec

Construct a bit-rounding codec that keeps keepbits mantissa bits.

Parameters:

  • keepbits (int) –

    The number of mantissa bits to keep.

Returns:

Raises:

blosc 

blosc(
    cname: BloscCompressor,
    clevel: int,
    shuffle_mode: BloscShuffle,
    *,
    blocksize: int | None = None,
    typesize: int | None = None
) -> BytesToBytesCodec

Construct a blosc codec from its parameters.

Parameters:

  • cname (BloscCompressor) –

    The name of the internal compressor.

  • clevel (int) –

    The compression level, from 0 (no compression) to 9 (most compression).

  • shuffle_mode (BloscShuffle) –

    The shuffle mode to apply before compression.

Keyword Arguments:

  • blocksize (int | None) –

    The block size in bytes. Give None or 0 to let blosc choose the block size.

  • typesize (int | None) –

    The size of one element in bytes. This must be a positive integer if shuffle_mode is not "noshuffle".

Returns:

Raises:

  • ValueError –

    If cname is not a known compressor, or if clevel is outside the range 0 to 9.

  • PluginCreateError –

    If shuffle_mode is not "noshuffle" and typesize is None or 0.

crc32c 

crc32c() -> BytesToBytesCodec

Construct a crc32c codec.

The codec appends a CRC32C checksum to the encoded bytestream.

Returns:

gzip 

gzip(level: int) -> BytesToBytesCodec

Construct a gzip codec.

Parameters:

  • level (int) –

    The compression level, from 0 (no compression) to 9 (most compression).

Returns:

Raises:

transpose 

transpose(order: list[int]) -> ArrayToArrayCodec

Construct a transpose codec with the given axis order.

Parameters:

  • order (list[int]) –

    The new order of the axes. This must be a permutation of the axis indices, from 0 to one less than the number of dimensions.

Returns:

Raises:

zstd 

zstd(level: int, checksum: bool) -> BytesToBytesCodec

Construct a zstd codec.

Parameters:

  • level (int) –

    The compression level. The codec does not check the range, and zstd clamps the value to the range that it supports.

  • checksum (bool) –

    Whether to write a checksum to the encoded bytestream. The codec verifies this checksum when it decodes the bytestream.

Returns:

zarrista.codec.CodecOptions

Bases: TypedDict

Per-operation codec options for encoding and decoding.

These control runtime behaviour such as concurrency limits and checksum validation. They are passed as keyword arguments to the array read/write methods, e.g. arr.retrieve_chunk([0, 0], validate_checksums=False). All keys are optional; omitted keys fall back to the defaults noted below.

Not importable at runtime

To use this type hint in your code, import it within a TYPE_CHECKING block:

from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
    from zarrista.codec import CodecOptions

chunk_concurrent_minimum instance-attribute 

chunk_concurrent_minimum: int

Preferred minimum chunk concurrency for multi-chunk operations. The concurrency of internal codecs is adjusted to accommodate the chunk concurrency in accordance with concurrent_target. Defaults to 4.

concurrent_target instance-attribute 

concurrent_target: int

Preferred number of concurrent operations. Defaults to the number of threads in the Rust thread pool.

experimental_partial_encoding instance-attribute 

experimental_partial_encoding: bool

Whether to use experimental partial encoding. Defaults to False.

store_empty_chunks instance-attribute 

store_empty_chunks: bool

Whether to store chunks that are entirely the fill value. Defaults to False.

validate_checksums instance-attribute 

validate_checksums: bool

Whether to validate checksums when decoding. Defaults to True.

Codec configuration types

zarrista.codec._bytes_to_bytes._blosc.BloscCompressor module-attribute 

BloscCompressor: TypeAlias = Literal[
    "blosclz", "lz4", "lz4hc", "snappy", "zlib", "zstd"
]

A blosc compressor name.

zarrista.codec._bytes_to_bytes._blosc.BloscShuffle module-attribute 

BloscShuffle: TypeAlias = Literal['noshuffle', 'shuffle', 'bitshuffle']

A blosc shuffle mode.