Memory¶

obstore.store.MemoryStore ¶

Bases: ObjectStoreMethods, MemoryStore

A fully in-memory implementation of ObjectStore.

Create a new in-memory store:

store = MemoryStore()

copy ¶

copy(from_: str, to: str, *, overwrite: bool = True) -> None

Copy an object from one path to another in the same object store.

Refer to the documentation for copy.

copy_async `async` ¶

copy_async(from_: str, to: str, *, overwrite: bool = True) -> None

Call copy asynchronously.

Refer to the documentation for copy_async.

delete ¶

delete(paths: str | Sequence[str]) -> None

Delete the object at the specified location(s).

Refer to the documentation for delete.

delete_async `async` ¶

delete_async(paths: str | Sequence[str]) -> None

Call delete asynchronously.

Refer to the documentation for delete_async.

get ¶

get(path: str, *, options: GetOptions | None = None) -> GetResult

Return the bytes that are stored at the specified location.

Refer to the documentation for get.

get_async `async` ¶

get_async(path: str, *, options: GetOptions | None = None) -> GetResult

Call get asynchronously.

Refer to the documentation for get_async.

get_range ¶

get_range(
    path: str, *, start: int, end: int | None = None, length: int | None = None
) -> Bytes

Return the bytes stored at the specified location in the given byte range.

Refer to the documentation for get_range.

get_range_async `async` ¶

get_range_async(
    path: str, *, start: int, end: int | None = None, length: int | None = None
) -> Bytes

Call get_range asynchronously.

Refer to the documentation for get_range_async.

get_ranges ¶

get_ranges(
    path: str,
    *,
    starts: Sequence[int],
    ends: Sequence[int] | None = None,
    lengths: Sequence[int] | None = None,
    coalesce: int = 1024 * 1024
) -> list[Bytes]

Return the bytes stored at the specified location in the given byte ranges.

Refer to the documentation for get_ranges.

get_ranges_async `async` ¶

get_ranges_async(
    path: str,
    *,
    starts: Sequence[int],
    ends: Sequence[int] | None = None,
    lengths: Sequence[int] | None = None,
    coalesce: int = 1024 * 1024
) -> list[Bytes]

Call get_ranges asynchronously.

Refer to the documentation for get_ranges_async.

head ¶

head(path: str) -> ObjectMeta

Return the metadata for the specified location.

Refer to the documentation for head.

head_async `async` ¶

head_async(path: str) -> ObjectMeta

Call head asynchronously.

Refer to the documentation for head_async.

list ¶

list(
    prefix: str | None = None,
    *,
    offset: str | None = None,
    chunk_size: int = 50,
    return_arrow: Literal[True]
) -> ListStream[RecordBatch]

list(
    prefix: str | None = None,
    *,
    offset: str | None = None,
    chunk_size: int = 50,
    return_arrow: Literal[False] = False
) -> ListStream[Sequence[ObjectMeta]]

list(
    prefix: str | None = None,
    *,
    offset: str | None = None,
    chunk_size: int = 50,
    return_arrow: bool = False
) -> ListStream[RecordBatch] | ListStream[Sequence[ObjectMeta]]

List all the objects with the given prefix.

Refer to the documentation for list.

list_async ¶

list_async(
    prefix: str | None = None,
    *,
    offset: str | None = None,
    chunk_size: int = 50,
    return_arrow: Literal[True]
) -> ListStream[RecordBatch]

list_async(
    prefix: str | None = None,
    *,
    offset: str | None = None,
    chunk_size: int = 50,
    return_arrow: Literal[False] = False
) -> ListStream[Sequence[ObjectMeta]]

list_async(
    prefix: str | None = None,
    *,
    offset: str | None = None,
    chunk_size: int = 50,
    return_arrow: bool = False
) -> ListStream[RecordBatch] | ListStream[Sequence[ObjectMeta]]

List all the objects with the given prefix.

Refer to the documentation for list.

Note

This is an alias for list, provided to match the ListAsync protocol in obspec. There is no difference in functionality between this and the list method.

list_with_delimiter ¶

list_with_delimiter(
    prefix: str | None = None, *, return_arrow: Literal[True]
) -> ListResult[Table]

list_with_delimiter(
    prefix: str | None = None, *, return_arrow: Literal[False] = False
) -> ListResult[Sequence[ObjectMeta]]

list_with_delimiter(
    prefix: str | None = None, *, return_arrow: bool = False
) -> ListResult[Table] | ListResult[Sequence[ObjectMeta]]

List objects with the given prefix and an implementation specific delimiter.

Refer to the documentation for list_with_delimiter.

list_with_delimiter_async `async` ¶

list_with_delimiter_async(
    prefix: str | None = None, *, return_arrow: Literal[True]
) -> ListResult[Table]

list_with_delimiter_async(
    prefix: str | None = None, *, return_arrow: Literal[False] = False
) -> ListResult[Sequence[ObjectMeta]]

list_with_delimiter_async(
    prefix: str | None = None, *, return_arrow: bool = False
) -> ListResult[Table] | ListResult[Sequence[ObjectMeta]]

Call list_with_delimiter asynchronously.

Refer to the documentation for list_with_delimiter_async.

put ¶

put(
    path: str,
    file: (
        IO[bytes] | Path | bytes | Buffer | Iterator[Buffer] | Iterable[Buffer]
    ),
    *,
    attributes: Attributes | None = None,
    tags: dict[str, str] | None = None,
    mode: PutMode | None = None,
    use_multipart: bool | None = None,
    chunk_size: int = 5 * 1024 * 1024,
    max_concurrency: int = 12
) -> PutResult

Save the provided bytes to the specified location.

Refer to the documentation for put.

put_async `async` ¶

put_async(
    path: str,
    file: (
        IO[bytes]
        | Path
        | bytes
        | Buffer
        | AsyncIterator[Buffer]
        | AsyncIterable[Buffer]
        | Iterator[Buffer]
        | Iterable[Buffer]
    ),
    *,
    attributes: Attributes | None = None,
    tags: dict[str, str] | None = None,
    mode: PutMode | None = None,
    use_multipart: bool | None = None,
    chunk_size: int = 5 * 1024 * 1024,
    max_concurrency: int = 12
) -> PutResult

Call put asynchronously.

Refer to the documentation for put_async.

rename ¶

rename(from_: str, to: str, *, overwrite: bool = True) -> None

Move an object from one path to another in the same object store.

Refer to the documentation for rename.

rename_async `async` ¶

rename_async(from_: str, to: str, *, overwrite: bool = True) -> None

Call rename asynchronously.

Refer to the documentation for rename_async.