Skip to content

List

obspec.List

Bases: Protocol

list 

list(
    prefix: str | None = None,
    *,
    offset: str | None = None,
    chunk_size: int = 50,
) -> ListIterator[Sequence[ObjectMeta]]

List all the objects with the given prefix.

Prefixes are evaluated on a path segment basis, i.e. foo/bar/ is a prefix of foo/bar/x but not of foo/bar_baz/x. List is recursive, i.e. foo/bar/more/x will be included.

Examples:

Synchronously iterate through list results:

import obstore as obs
from obstore.store import MemoryStore

store = MemoryStore()
for i in range(100):
    obs.put(store, f"file{i}.txt", b"foo")

stream = obs.list(store, chunk_size=10)
for list_result in stream:
    print(list_result[0])
    # {'path': 'file0.txt', 'last_modified': datetime.datetime(2024, 10, 23, 19, 19, 28, 781723, tzinfo=datetime.timezone.utc), 'size': 3, 'e_tag': '0', 'version': None}
    break

Note

The order of returned ObjectMeta is not guaranteed

Parameters:

  • prefix (str | None, default: None ) –

    The prefix within ObjectStore to use for listing. Defaults to None.

Other Parameters:

  • offset (str | None) –

    If provided, list all the objects with the given prefix and a location greater than offset. Defaults to None.

  • chunk_size (int) –

    The number of items to collect per chunk in the returned (async) iterator. All chunks except for the last one will have this many items.

Returns:

obspec.ListAsync

Bases: Protocol

list_async 

list_async(
    prefix: str | None = None,
    *,
    offset: str | None = None,
    chunk_size: int = 50,
) -> ListStream[Sequence[ObjectMeta]]

List all the objects with the given prefix.

Note that this method itself is not async. It's a synchronous method but returns an async iterator.

Refer to obspec.List for more information about list semantics.

Examples:

Asynchronously iterate through list results. Just change for to async for:

stream = obs.list_async(store, chunk_size=10)
async for list_result in stream:
    print(list_result[2])
    # {'path': 'file10.txt', 'last_modified': datetime.datetime(2024, 10, 23, 19, 21, 46, 224725, tzinfo=datetime.timezone.utc), 'size': 3, 'e_tag': '10', 'version': None}
    break

Note

The order of returned ObjectMeta is not guaranteed

Parameters:

  • prefix (str | None, default: None ) –

    The prefix within ObjectStore to use for listing. Defaults to None.

Other Parameters:

  • offset (str | None) –

    If provided, list all the objects with the given prefix and a location greater than offset. Defaults to None.

  • chunk_size (int) –

    The number of items to collect per chunk in the returned (async) iterator. All chunks except for the last one will have this many items.

Returns:

obspec.ListWithDelimiter

Bases: Protocol

list_with_delimiter 

list_with_delimiter(
    prefix: str | None = None,
) -> ListResult[Sequence[ObjectMeta]]

List objects with the given prefix and an implementation specific delimiter.

Returns common prefixes (directories) in addition to object metadata.

Prefixes are evaluated on a path segment basis, i.e. foo/bar/ is a prefix of foo/bar/x but not of foo/bar_baz/x. This list is not recursive, i.e. foo/bar/more/x will not be included.

Note

Any prefix supplied to this prefix parameter will not be stripped off the paths in the result.

Parameters:

  • prefix (str | None, default: None ) –

    The prefix within ObjectStore to use for listing. Defaults to None.

Returns:

obspec.ListWithDelimiterAsync

Bases: Protocol

list_with_delimiter_async async 

list_with_delimiter_async(
    prefix: str | None = None,
) -> ListResult[Sequence[ObjectMeta]]

Call list_with_delimiter asynchronously.

Refer to the documentation for ListWithDelimiter.

obspec.ListResult

Bases: TypedDict, Generic[ListChunkType_co]

Result of a list_with_delimiter call.

Includes objects, prefixes (directories) and a token for the next set of results. Individual result sets may be limited to 1,000 objects based on the underlying object storage's limitations.

common_prefixes instance-attribute 

common_prefixes: Sequence[str]

Prefixes that are common (like directories)

objects instance-attribute 

objects: ListChunkType_co

Object metadata for the listing

obspec.ListIterator

Bases: Protocol[ListChunkType_co]

A stream of ObjectMeta that can be polled synchronously.

__iter__ 

__iter__() -> Self

Return Self as an async iterator.

__next__ 

__next__() -> ListChunkType_co

Return the next chunk of ObjectMeta in the stream.

obspec.ListStream

Bases: Protocol[ListChunkType_co]

A stream of ObjectMeta that can be polled asynchronously.

__aiter__ 

__aiter__() -> Self

Return Self as an async iterator.

__anext__ async 

__anext__() -> ListChunkType_co

Return the next chunk of ObjectMeta in the stream.

obspec.ListChunkType_co module-attribute 

ListChunkType_co = TypeVar('ListChunkType_co', covariant=True)

The data structure used for holding list results.