Skip to content

Microsoft Azure

obstore.store.AzureStore

Interface to a Microsoft Azure Blob Storage container.

All constructors will check for environment variables. All environment variables starting with AZURE_ will be evaluated. Names must match keys from AzureConfig. Only upper-case environment variables are accepted.

Some examples of variables extracted from environment:

  • AZURE_STORAGE_ACCOUNT_NAME: storage account name
  • AZURE_STORAGE_ACCOUNT_KEY: storage account master key
  • AZURE_STORAGE_ACCESS_KEY: alias for AZURE_STORAGE_ACCOUNT_KEY
  • AZURE_STORAGE_CLIENT_ID -> client id for service principal authorization
  • AZURE_STORAGE_CLIENT_SECRET -> client secret for service principal authorization
  • AZURE_STORAGE_TENANT_ID -> tenant id used in oauth flows

client_options property 

client_options: ClientConfig | None

Get the store's client configuration.

config property 

config: AzureConfig

Get the underlying Azure config parameters.

prefix property 

prefix: str | None

Get the prefix applied to all operations in this store, if any.

retry_config property 

retry_config: RetryConfig | None

Get the store's retry configuration.

__init__ 

__init__(
    container: str | None = None,
    *,
    prefix: str | None = None,
    config: AzureConfig | AzureConfigInput | None = None,
    client_options: ClientConfig | None = None,
    retry_config: RetryConfig | None = None,
    credential_provider: AzureCredentialProvider | None = None,
    **kwargs: Unpack[AzureConfigInput],
) -> None

Construct a new AzureStore.

Parameters:

  • container (str | None, default: None ) –

    the name of the container.

Other Parameters:

  • prefix (str | None) –

    A prefix within the bucket to use for all operations.

  • config (AzureConfig | AzureConfigInput | None) –

    Azure Configuration. Values in this config will override values inferred from the url. Defaults to None.

  • client_options (ClientConfig | None) –

    HTTP Client options. Defaults to None.

  • retry_config (RetryConfig | None) –

    Retry configuration. Defaults to None.

  • credential_provider (AzureCredentialProvider | None) –

    A callback to provide custom Azure credentials.

  • kwargs (Unpack[AzureConfigInput]) –

    Azure configuration values. Supports the same values as config, but as named keyword args.

Returns:

  • None –

    AzureStore

from_url classmethod 

from_url(
    url: str,
    *,
    prefix: str | None = None,
    config: AzureConfig | AzureConfigInput | None = None,
    client_options: ClientConfig | None = None,
    retry_config: RetryConfig | None = None,
    credential_provider: AzureCredentialProvider | None = None,
    **kwargs: Unpack[AzureConfigInput],
) -> AzureStore

Construct a new AzureStore with values populated from a well-known storage URL.

The supported url schemes are:

  • abfs[s]://<container>/<path> (according to fsspec)
  • abfs[s]://<file_system>@<account_name>.dfs.core.windows.net/<path>
  • abfs[s]://<file_system>@<account_name>.dfs.fabric.microsoft.com/<path>
  • az://<container>/<path> (according to fsspec)
  • adl://<container>/<path> (according to fsspec)
  • azure://<container>/<path> (custom)
  • https://<account>.dfs.core.windows.net
  • https://<account>.blob.core.windows.net
  • https://<account>.blob.core.windows.net/<container>
  • https://<account>.dfs.fabric.microsoft.com
  • https://<account>.dfs.fabric.microsoft.com/<container>
  • https://<account>.blob.fabric.microsoft.com
  • https://<account>.blob.fabric.microsoft.com/<container>

Parameters:

  • url (str) –

    well-known storage URL.

Other Parameters:

  • prefix (str | None) –

    A prefix within the bucket to use for all operations.

  • config (AzureConfig | AzureConfigInput | None) –

    Azure Configuration. Values in this config will override values inferred from the url. Defaults to None.

  • client_options (ClientConfig | None) –

    HTTP Client options. Defaults to None.

  • retry_config (RetryConfig | None) –

    Retry configuration. Defaults to None.

  • credential_provider (AzureCredentialProvider | None) –

    A callback to provide custom Azure credentials.

  • kwargs (Unpack[AzureConfigInput]) –

    Azure configuration values. Supports the same values as config, but as named keyword args.

Returns:

obstore.store.AzureConfigInput

Bases: TypedDict

Configuration parameters for AzureStore.

There are duplicates of many parameters, and parameters can be either upper or lower case. Not all parameters are required.

Not importable at runtime

To use this type hint in your code, import it within a TYPE_CHECKING block:

from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
    from obstore.store import AzureConfigInput

ACCESS_KEY instance-attribute 

ACCESS_KEY: str

Master key for accessing storage account

ACCOUNT_KEY instance-attribute 

ACCOUNT_KEY: str

Master key for accessing storage account

ACCOUNT_NAME instance-attribute 

ACCOUNT_NAME: str

The name of the azure storage account

AUTHORITY_ID instance-attribute 

AUTHORITY_ID: str

Tenant id used in oauth flows

AZURE_AUTHORITY_ID instance-attribute 

AZURE_AUTHORITY_ID: str

Tenant id used in oauth flows

AZURE_CLIENT_ID instance-attribute 

AZURE_CLIENT_ID: str

Service principal client id for authorizing requests

AZURE_CLIENT_SECRET instance-attribute 

AZURE_CLIENT_SECRET: str

Service principal client secret for authorizing requests

AZURE_CONTAINER_NAME instance-attribute 

AZURE_CONTAINER_NAME: str

Container name

AZURE_DISABLE_TAGGING instance-attribute 

AZURE_DISABLE_TAGGING: bool

Disables tagging objects

AZURE_ENDPOINT instance-attribute 

AZURE_ENDPOINT: str

Override the endpoint used to communicate with blob storage

AZURE_FEDERATED_TOKEN_FILE instance-attribute 

AZURE_FEDERATED_TOKEN_FILE: str

File containing token for Azure AD workload identity federation

AZURE_IDENTITY_ENDPOINT instance-attribute 

AZURE_IDENTITY_ENDPOINT: str

Endpoint to request a imds managed identity token

AZURE_MSI_ENDPOINT instance-attribute 

AZURE_MSI_ENDPOINT: str

Endpoint to request a imds managed identity token

AZURE_MSI_RESOURCE_ID instance-attribute 

AZURE_MSI_RESOURCE_ID: str

Msi resource id for use with managed identity authentication

AZURE_OBJECT_ID instance-attribute 

AZURE_OBJECT_ID: str

Object id for use with managed identity authentication

AZURE_SKIP_SIGNATURE instance-attribute 

AZURE_SKIP_SIGNATURE: bool

Skip signing requests

AZURE_STORAGE_ACCESS_KEY instance-attribute 

AZURE_STORAGE_ACCESS_KEY: str

Master key for accessing storage account

AZURE_STORAGE_ACCOUNT_KEY instance-attribute 

AZURE_STORAGE_ACCOUNT_KEY: str

Master key for accessing storage account

AZURE_STORAGE_ACCOUNT_NAME instance-attribute 

AZURE_STORAGE_ACCOUNT_NAME: str

The name of the azure storage account

AZURE_STORAGE_AUTHORITY_ID instance-attribute 

AZURE_STORAGE_AUTHORITY_ID: str

Tenant id used in oauth flows

AZURE_STORAGE_CLIENT_ID instance-attribute 

AZURE_STORAGE_CLIENT_ID: str

Service principal client id for authorizing requests

AZURE_STORAGE_CLIENT_SECRET instance-attribute 

AZURE_STORAGE_CLIENT_SECRET: str

Service principal client secret for authorizing requests

AZURE_STORAGE_ENDPOINT instance-attribute 

AZURE_STORAGE_ENDPOINT: str

Override the endpoint used to communicate with blob storage

AZURE_STORAGE_MASTER_KEY instance-attribute 

AZURE_STORAGE_MASTER_KEY: str

Master key for accessing storage account

AZURE_STORAGE_SAS_KEY instance-attribute 

AZURE_STORAGE_SAS_KEY: str

Shared access signature.

The signature is expected to be percent-encoded, muchlike they are provided in the azure storage explorer or azure portal.

AZURE_STORAGE_SAS_TOKEN instance-attribute 

AZURE_STORAGE_SAS_TOKEN: str

Shared access signature.

The signature is expected to be percent-encoded, muchlike they are provided in the azure storage explorer or azure portal.

AZURE_STORAGE_TENANT_ID instance-attribute 

AZURE_STORAGE_TENANT_ID: str

Tenant id used in oauth flows

AZURE_STORAGE_TOKEN instance-attribute 

AZURE_STORAGE_TOKEN: str

Bearer token

AZURE_STORAGE_USE_EMULATOR instance-attribute 

AZURE_STORAGE_USE_EMULATOR: bool

Use object store with azurite storage emulator

AZURE_TENANT_ID instance-attribute 

AZURE_TENANT_ID: str

Tenant id used in oauth flows

AZURE_USE_AZURE_CLI instance-attribute 

AZURE_USE_AZURE_CLI: bool

Use azure cli for acquiring access token.

Defaults to True.

AZURE_USE_FABRIC_ENDPOINT instance-attribute 

AZURE_USE_FABRIC_ENDPOINT: bool

Use object store with url scheme account.dfs.fabric.microsoft.com

BEARER_TOKEN instance-attribute 

BEARER_TOKEN: str

Bearer token

CLIENT_ID instance-attribute 

CLIENT_ID: str

Service principal client id for authorizing requests

CLIENT_SECRET instance-attribute 

CLIENT_SECRET: str

Service principal client secret for authorizing requests

CONTAINER_NAME instance-attribute 

CONTAINER_NAME: str

Container name

DISABLE_TAGGING instance-attribute 

DISABLE_TAGGING: bool

Disables tagging objects

ENDPOINT instance-attribute 

ENDPOINT: str

Override the endpoint used to communicate with blob storage

FEDERATED_TOKEN_FILE instance-attribute 

FEDERATED_TOKEN_FILE: str

File containing token for Azure AD workload identity federation

IDENTITY_ENDPOINT instance-attribute 

IDENTITY_ENDPOINT: str

Endpoint to request a imds managed identity token

MASTER_KEY instance-attribute 

MASTER_KEY: str

Master key for accessing storage account

MSI_ENDPOINT instance-attribute 

MSI_ENDPOINT: str

Endpoint to request a imds managed identity token

MSI_RESOURCE_ID instance-attribute 

MSI_RESOURCE_ID: str

Msi resource id for use with managed identity authentication

OBJECT_ID instance-attribute 

OBJECT_ID: str

Object id for use with managed identity authentication

SAS_KEY instance-attribute 

SAS_KEY: str

Shared access signature.

The signature is expected to be percent-encoded, muchlike they are provided in the azure storage explorer or azure portal.

SAS_TOKEN instance-attribute 

SAS_TOKEN: str

Shared access signature.

The signature is expected to be percent-encoded, muchlike they are provided in the azure storage explorer or azure portal.

SKIP_SIGNATURE instance-attribute 

SKIP_SIGNATURE: bool

Skip signing requests

TENANT_ID instance-attribute 

TENANT_ID: str

Tenant id used in oauth flows

TOKEN instance-attribute 

TOKEN: str

Bearer token

USE_AZURE_CLI instance-attribute 

USE_AZURE_CLI: bool

Use azure cli for acquiring access token.

Defaults to True.

USE_EMULATOR instance-attribute 

USE_EMULATOR: bool

Use object store with azurite storage emulator

USE_FABRIC_ENDPOINT instance-attribute 

USE_FABRIC_ENDPOINT: bool

Use object store with url scheme account.dfs.fabric.microsoft.com

access_key instance-attribute 

access_key: str

Master key for accessing storage account

account_key instance-attribute 

account_key: str

Master key for accessing storage account

account_name instance-attribute 

account_name: str

The name of the azure storage account

authority_id instance-attribute 

authority_id: str

Tenant id used in oauth flows

azure_authority_id instance-attribute 

azure_authority_id: str

Tenant id used in oauth flows

azure_client_id instance-attribute 

azure_client_id: str

Service principal client id for authorizing requests

azure_client_secret instance-attribute 

azure_client_secret: str

Service principal client secret for authorizing requests

azure_container_name instance-attribute 

azure_container_name: str

Container name

azure_disable_tagging instance-attribute 

azure_disable_tagging: bool

Disables tagging objects

azure_endpoint instance-attribute 

azure_endpoint: str

Override the endpoint used to communicate with blob storage

azure_federated_token_file instance-attribute 

azure_federated_token_file: str

File containing token for Azure AD workload identity federation

azure_identity_endpoint instance-attribute 

azure_identity_endpoint: str

Endpoint to request a imds managed identity token

azure_msi_endpoint instance-attribute 

azure_msi_endpoint: str

Endpoint to request a imds managed identity token

azure_msi_resource_id instance-attribute 

azure_msi_resource_id: str

Msi resource id for use with managed identity authentication

azure_object_id instance-attribute 

azure_object_id: str

Object id for use with managed identity authentication

azure_skip_signature instance-attribute 

azure_skip_signature: bool

Skip signing requests

azure_storage_access_key instance-attribute 

azure_storage_access_key: str

Master key for accessing storage account

azure_storage_account_key instance-attribute 

azure_storage_account_key: str

Master key for accessing storage account

azure_storage_account_name instance-attribute 

azure_storage_account_name: str

The name of the azure storage account

azure_storage_authority_id instance-attribute 

azure_storage_authority_id: str

Tenant id used in oauth flows

azure_storage_client_id instance-attribute 

azure_storage_client_id: str

Service principal client id for authorizing requests

azure_storage_client_secret instance-attribute 

azure_storage_client_secret: str

Service principal client secret for authorizing requests

azure_storage_endpoint instance-attribute 

azure_storage_endpoint: str

Override the endpoint used to communicate with blob storage

azure_storage_master_key instance-attribute 

azure_storage_master_key: str

Master key for accessing storage account

azure_storage_sas_key instance-attribute 

azure_storage_sas_key: str

Shared access signature.

The signature is expected to be percent-encoded, muchlike they are provided in the azure storage explorer or azure portal.

azure_storage_sas_token instance-attribute 

azure_storage_sas_token: str

Shared access signature.

The signature is expected to be percent-encoded, muchlike they are provided in the azure storage explorer or azure portal.

azure_storage_tenant_id instance-attribute 

azure_storage_tenant_id: str

Tenant id used in oauth flows

azure_storage_token instance-attribute 

azure_storage_token: str

Bearer token

azure_storage_use_emulator instance-attribute 

azure_storage_use_emulator: bool

Use object store with azurite storage emulator

azure_tenant_id instance-attribute 

azure_tenant_id: str

Tenant id used in oauth flows

azure_use_azure_cli instance-attribute 

azure_use_azure_cli: bool

Use azure cli for acquiring access token.

Defaults to True.

azure_use_fabric_endpoint instance-attribute 

azure_use_fabric_endpoint: bool

Use object store with url scheme account.dfs.fabric.microsoft.com

bearer_token instance-attribute 

bearer_token: str

Bearer token

client_id instance-attribute 

client_id: str

Service principal client id for authorizing requests

client_secret instance-attribute 

client_secret: str

Service principal client secret for authorizing requests

container_name instance-attribute 

container_name: str

Container name

disable_tagging instance-attribute 

disable_tagging: bool

Disables tagging objects

endpoint instance-attribute 

endpoint: str

Override the endpoint used to communicate with blob storage

federated_token_file instance-attribute 

federated_token_file: str

File containing token for Azure AD workload identity federation

identity_endpoint instance-attribute 

identity_endpoint: str

Endpoint to request a imds managed identity token

master_key instance-attribute 

master_key: str

Master key for accessing storage account

msi_endpoint instance-attribute 

msi_endpoint: str

Endpoint to request a imds managed identity token

msi_resource_id instance-attribute 

msi_resource_id: str

Msi resource id for use with managed identity authentication

object_id instance-attribute 

object_id: str

Object id for use with managed identity authentication

sas_key instance-attribute 

sas_key: str

Shared access signature.

The signature is expected to be percent-encoded, muchlike they are provided in the azure storage explorer or azure portal.

sas_token instance-attribute 

sas_token: str

Shared access signature.

The signature is expected to be percent-encoded, muchlike they are provided in the azure storage explorer or azure portal.

skip_signature instance-attribute 

skip_signature: bool

Skip signing requests

tenant_id instance-attribute 

tenant_id: str

Tenant id used in oauth flows

token instance-attribute 

token: str

Bearer token

use_azure_cli instance-attribute 

use_azure_cli: bool

Use azure cli for acquiring access token.

Defaults to True.

use_emulator instance-attribute 

use_emulator: bool

Use object store with azurite storage emulator

use_fabric_endpoint instance-attribute 

use_fabric_endpoint: bool

Use object store with url scheme account.dfs.fabric.microsoft.com

obstore.store.AzureConfig

Bases: TypedDict

Configuration parameters returned from AzureStore.config.

Note that this is a strict subset of the keys allowed for input into the store, see AzureConfigInput.

Not importable at runtime

To use this type hint in your code, import it within a TYPE_CHECKING block:

from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
    from obstore.store import AzureConfig

azure_container_name instance-attribute 

azure_container_name: str

Container name

azure_disable_tagging instance-attribute 

azure_disable_tagging: bool

Disables tagging objects

azure_federated_token_file instance-attribute 

azure_federated_token_file: str

File containing token for Azure AD workload identity federation

azure_msi_endpoint instance-attribute 

azure_msi_endpoint: str

Endpoint to request a imds managed identity token

azure_msi_resource_id instance-attribute 

azure_msi_resource_id: str

Msi resource id for use with managed identity authentication

azure_object_id instance-attribute 

azure_object_id: str

Object id for use with managed identity authentication

azure_skip_signature instance-attribute 

azure_skip_signature: bool

Skip signing requests

azure_storage_account_key instance-attribute 

azure_storage_account_key: str

Master key for accessing storage account

azure_storage_account_name instance-attribute 

azure_storage_account_name: str

The name of the azure storage account

azure_storage_client_id instance-attribute 

azure_storage_client_id: str

Service principal client id for authorizing requests

azure_storage_client_secret instance-attribute 

azure_storage_client_secret: str

Service principal client secret for authorizing requests

azure_storage_endpoint instance-attribute 

azure_storage_endpoint: str

Override the endpoint used to communicate with blob storage

azure_storage_sas_key instance-attribute 

azure_storage_sas_key: str

Shared access signature.

The signature is expected to be percent-encoded, muchlike they are provided in the azure storage explorer or azure portal.

azure_storage_tenant_id instance-attribute 

azure_storage_tenant_id: str

Tenant id used in oauth flows

azure_storage_token instance-attribute 

azure_storage_token: str

Bearer token

azure_storage_use_emulator instance-attribute 

azure_storage_use_emulator: bool

Use object store with azurite storage emulator

azure_use_azure_cli instance-attribute 

azure_use_azure_cli: bool

Use azure cli for acquiring access token

azure_use_fabric_endpoint instance-attribute 

azure_use_fabric_endpoint: bool

Use object store with url scheme account.dfs.fabric.microsoft.com

obstore.store.AzureAccessKey

Bases: TypedDict

A shared Azure Storage Account Key.

learn.microsoft.com/en-us/rest/api/storageservices/authorize-with-shared-key

Not importable at runtime

To use this type hint in your code, import it within a TYPE_CHECKING block:

from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
    from obstore.store import AzureAccessKey

access_key instance-attribute 

access_key: str

Access key value.

expires_at instance-attribute 

expires_at: datetime | None

Expiry datetime of credential. The datetime should have time zone set.

If None, the credential will never expire.

obstore.store.AzureSASToken

Bases: TypedDict

A shared access signature.

learn.microsoft.com/en-us/rest/api/storageservices/delegate-access-with-shared-access-signature

Not importable at runtime

To use this type hint in your code, import it within a TYPE_CHECKING block:

from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
    from obstore.store import AzureSASToken

expires_at instance-attribute 

expires_at: datetime | None

Expiry datetime of credential. The datetime should have time zone set.

If None, the credential will never expire.

sas_token instance-attribute 

sas_token: str | list[tuple[str, str]]

SAS token.

obstore.store.AzureBearerToken

Bases: TypedDict

An authorization token.

learn.microsoft.com/en-us/rest/api/storageservices/authorize-with-azure-active-directory

Not importable at runtime

To use this type hint in your code, import it within a TYPE_CHECKING block:

from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
    from obstore.store import AzureBearerToken

expires_at instance-attribute 

expires_at: datetime | None

Expiry datetime of credential. The datetime should have time zone set.

If None, the credential will never expire.

token instance-attribute 

token: str

Bearer token.

obstore.store.AzureCredential module-attribute 

AzureCredential: TypeAlias = AzureAccessKey | AzureSASToken | AzureBearerToken

A type alias for supported azure credentials to be returned from AzureCredentialProvider.

Not importable at runtime

To use this type hint in your code, import it within a TYPE_CHECKING block:

from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
    from obstore.store import AzureCredential

obstore.store.AzureCredentialProvider

Bases: Protocol

A type hint for a synchronous or asynchronous callback to provide custom Azure credentials.

This should be passed into the credential_provider parameter of AzureStore.

Not importable at runtime

To use this type hint in your code, import it within a TYPE_CHECKING block:

from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
    from obstore.store import AzureCredentialProvider

__call__ staticmethod 

__call__() -> AzureCredential | Coroutine[Any, Any, AzureCredential]

Return an AzureCredential.