ObjectStore

obstore.store.from_url

from_url(
    url: str,
    *,
    config: S3Config | None = None,
    client_options: ClientConfig | None = None,
    retry_config: RetryConfig | None = None,
    credential_provider: S3CredentialProvider | None = None,
    **kwargs: Unpack[S3Config],
) -> ObjectStore

from_url(
    url: str,
    *,
    config: GCSConfig | None = None,
    client_options: ClientConfig | None = None,
    retry_config: RetryConfig | None = None,
    credential_provider: GCSCredentialProvider | None = None,
    **kwargs: Unpack[GCSConfig],
) -> ObjectStore

from_url(
    url: str,
    *,
    config: AzureConfig | None = None,
    client_options: ClientConfig | None = None,
    retry_config: RetryConfig | None = None,
    credential_provider: AzureCredentialProvider | None = None,
    **kwargs: Unpack[AzureConfig],
) -> ObjectStore

from_url(
    url: str,
    *,
    config: None = None,
    client_options: None = None,
    retry_config: None = None,
    automatic_cleanup: bool = False,
    mkdir: bool = False,
) -> ObjectStore

from_url(
    url: str,
    *,
    config: S3Config | GCSConfig | AzureConfig | None = None,
    client_options: ClientConfig | None = None,
    retry_config: RetryConfig | None = None,
    credential_provider: S3CredentialProvider
    | GCSCredentialProvider
    | AzureCredentialProvider
    | None = None,
    **kwargs: Any,
) -> ObjectStore

Easy construction of store by URL, identifying the relevant store.

This will defer to a store-specific from_url constructor based on the provided url. E.g. passing "s3://bucket/path" will defer to S3Store.from_url.

Any path on the URL will be assigned as the prefix for the store. So if you pass s3://bucket/path/to/directory, the store will be created with a prefix of path/to/directory, and all further operations will use paths relative to that prefix.

Supported formats:

• file:///path/ -> LocalStore
• memory:/// -> MemoryStore
• s3://bucket/path -> S3Store (also supports s3a)
• gs://bucket/path -> GCSStore
• az://account/container/path -> AzureStore (also supports adl, azure, abfs, abfss)
• http://mydomain/path -> HTTPStore
• https://mydomain/path -> HTTPStore

There are also special cases for AWS and Azure for https://{host?}/path paths:

• dfs.core.windows.net, blob.core.windows.net, dfs.fabric.microsoft.com, blob.fabric.microsoft.com -> AzureStore
• amazonaws.com -> S3Store
• r2.cloudflarestorage.com -> S3Store

Note

For best static typing, use the constructors on individual store classes directly.

Parameters:

• url (str) –

  well-known storage URL.

Other Parameters:

• config (S3Config | GCSConfig | AzureConfig | None) –

  per-store Configuration. Values in this config will override values inferred from the url. Defaults to None.

• client_options (ClientConfig | None) –

  HTTP Client options. Defaults to None.

• retry_config (RetryConfig | None) –

  Retry configuration. Defaults to None.

• credential_provider (S3CredentialProvider | GCSCredentialProvider | AzureCredentialProvider | None) –

  A callback to provide custom credentials to the underlying store classes.

• kwargs (Any) –

  per-store configuration passed down to store-specific builders.

obstore.store.ObjectStore module-attribute

ObjectStore: TypeAlias = Union[
    AzureStore, GCSStore, HTTPStore, S3Store, LocalStore, MemoryStore
]

All supported ObjectStore implementations.