Concurrency¶
zarrista does CPU-bound work, such as decoding, on a global Rust thread pool.
One pool serves the whole process. To set its size, use the
RAYON_NUM_THREADS environment variable.
To run the work on a separate pool instead, construct a
ThreadPool and pass it to
EncodedChunk.decode_async.
To limit how much of the pool one operation claims, pass concurrent_target
as a codec option. See CodecOptions.
zarrista.ThreadPool ¶
A dedicated Rust thread pool for CPU-bound work.
You do not need this class for normal use. By default,
EncodedChunk.decode_async runs on
the global Rust thread pool. zarrista shares that same pool for all of its
internal parallel work, so one pool serves the whole process. To set its
size, use the RAYON_NUM_THREADS environment variable.
Construct a ThreadPool only when you want to isolate decode work from the
global pool. Each ThreadPool adds threads to the process, so two pools
that each have one thread per core can compete for the CPU.
Examples:
Decode on a pool of four threads:
pool = zarrista.ThreadPool(4)
chunk = array.retrieve_encoded_chunk([0, 0])
decoded = await chunk.decode_async(pool=pool)