ObjectStore

obstore.store.from_url

from_url(
    url: str,
    *,
    config: S3Config | None = None,
    client_options: ClientConfig | None = None,
    retry_config: RetryConfig | None = None,
    credential_provider: S3CredentialProvider | None = None,
    **kwargs: Unpack[S3Config]
) -> ObjectStore

from_url(
    url: str,
    *,
    config: GCSConfig | None = None,
    client_options: ClientConfig | None = None,
    retry_config: RetryConfig | None = None,
    credential_provider: GCSCredentialProvider | None = None,
    **kwargs: Unpack[GCSConfig]
) -> ObjectStore

from_url(
    url: str,
    *,
    config: AzureConfig | None = None,
    client_options: ClientConfig | None = None,
    retry_config: RetryConfig | None = None,
    credential_provider: AzureCredentialProvider | None = None,
    **kwargs: Unpack[AzureConfig]
) -> ObjectStore

from_url(
    url: str,
    *,
    config: None = None,
    client_options: None = None,
    retry_config: None = None,
    automatic_cleanup: bool = False,
    mkdir: bool = False
) -> ObjectStore

from_url(
    url: str,
    *,
    config: S3Config | GCSConfig | AzureConfig | None = None,
    client_options: ClientConfig | None = None,
    retry_config: RetryConfig | None = None,
    credential_provider: (
        S3CredentialProvider
        | GCSCredentialProvider
        | AzureCredentialProvider
        | None
    ) = None,
    **kwargs: Any
) -> ObjectStore

Easy construction of store by URL, identifying the relevant store.

This will defer to a store-specific from_url constructor based on the provided url. E.g. passing "s3://bucket/path" will defer to S3Store.from_url.

Any path on the URL will be assigned as the prefix for the store. So if you pass s3://bucket/path/to/directory, the store will be created with a prefix of path/to/directory, and all further operations will use paths relative to that prefix.

Supported formats:

• file:///path/ -> LocalStore
• memory:/// -> MemoryStore
• s3://bucket/path -> S3Store (also supports s3a)
• gs://bucket/path -> GCSStore
• az://account/container/path -> AzureStore (also supports adl, azure, abfs, abfss)
• http://mydomain/path -> HTTPStore
• https://mydomain/path -> HTTPStore

There are also special cases for AWS and Azure for https://{host?}/path paths:

• dfs.core.windows.net, blob.core.windows.net, dfs.fabric.microsoft.com, blob.fabric.microsoft.com -> AzureStore
• amazonaws.com -> S3Store
• r2.cloudflarestorage.com -> S3Store

Note

For best static typing, use the constructors on individual store classes directly.

Parameters:

• url (str) –

  well-known storage URL.

Other Parameters:

• config (S3Config | GCSConfig | AzureConfig | None) –

  per-store Configuration. Values in this config will override values inferred from the url. Defaults to None.

• client_options (ClientConfig | None) –

  HTTP Client options. Defaults to None.

• retry_config (RetryConfig | None) –

  Retry configuration. Defaults to None.

• credential_provider (S3CredentialProvider | GCSCredentialProvider | AzureCredentialProvider | None) –

  A callback to provide custom credentials to the underlying store classes.

• kwargs (Any) –

  per-store configuration passed down to store-specific builders.

obstore.store.ObjectStore module-attribute

ObjectStore: TypeAlias = Union[
    AzureStore, GCSStore, HTTPStore, S3Store, LocalStore, MemoryStore
]

All supported ObjectStore implementations.

obstore.store.ObjectStoreMethods

Shared methods implemented by all obstore store classes.

This is the concrete method surface common to S3Store, AzureStore, GCSStore, HTTPStore, LocalStore, and MemoryStore.

It is exposed for documentation and for isinstance checks against obstore's own stores.

This implements the protocols defined in obspec.

Note

Do not subclass this to build your own store. To implement a custom object store backend, implement the protocols in obspec instead.