Skip to content

obstore

PyPI Conda Version PyPI - Downloads

Simple, fast integration with object storage services like Amazon S3, Google Cloud Storage, Azure Blob Storage, and S3-compliant APIs like Cloudflare R2.

  • Sync and async API.
  • Streaming downloads with configurable chunking.
  • Streaming uploads from async or sync iterators.
  • Streaming list, with no need to paginate.
  • File-like object API and fsspec integration.
  • Support for conditional put ("put if not exists"), as well as custom tags and attributes.
  • Automatically uses multipart uploads under the hood for large file objects.
  • Optionally return list results as Arrow, which is faster than materializing Python dict/list objects.
  • Easy to install with no required Python dependencies.
  • The underlying Rust library is production quality and used in large scale production systems, such as the Rust package registry crates.io.
  • Zero-copy data exchange between Rust and Python in get_range, get_ranges, GetResult.bytes, and put via the Python buffer protocol.
  • Simple API with static type checking.
  • Helpers for constructing from environment variables and boto3.Session objects

Installation

To install obstore using pip:

pip install obstore

Obstore is on conda-forge and can be installed using conda, mamba, or pixi. To install obstore using conda:

conda install -c conda-forge obstore

Documentation

Full documentation is available on the website.

Head to Getting Started to dig in.