Skip to content

Google Cloud Storage

obstore.store.GCSStore

Interface to Google Cloud Storage.

All constructors will check for environment variables. All environment variables starting with GOOGLE_ will be evaluated. Names must match keys from GCSConfig. Only upper-case environment variables are accepted.

Some examples of variables extracted from environment:

  • GOOGLE_SERVICE_ACCOUNT: location of service account file
  • GOOGLE_SERVICE_ACCOUNT_PATH: (alias) location of service account file
  • SERVICE_ACCOUNT: (alias) location of service account file
  • GOOGLE_SERVICE_ACCOUNT_KEY: JSON serialized service account key
  • GOOGLE_BUCKET: bucket name
  • GOOGLE_BUCKET_NAME: (alias) bucket name

If no credentials are explicitly provided, they will be sourced from the environment as documented here.

client_options property 

client_options: ClientConfig | None

Get the store's client configuration.

config property 

config: GCSConfig

Get the underlying GCS config parameters.

prefix property 

prefix: str | None

Get the prefix applied to all operations in this store, if any.

retry_config property 

retry_config: RetryConfig | None

Get the store's retry configuration.

__init__ 

__init__(
    bucket: str | None = None,
    *,
    prefix: str | None = None,
    config: GCSConfig | GCSConfigInput | None = None,
    client_options: ClientConfig | None = None,
    retry_config: RetryConfig | None = None,
    credential_provider: GCSCredentialProvider | None = None,
    **kwargs: Unpack[GCSConfigInput],
) -> None

Construct a new GCSStore.

Parameters:

  • bucket (str | None, default: None ) –

    The GCS bucket to use.

Other Parameters:

  • prefix (str | None) –

    A prefix within the bucket to use for all operations.

  • config (GCSConfig | GCSConfigInput | None) –

    GCS Configuration. Values in this config will override values inferred from the environment. Defaults to None.

  • client_options (ClientConfig | None) –

    HTTP Client options. Defaults to None.

  • retry_config (RetryConfig | None) –

    Retry configuration. Defaults to None.

  • credential_provider (GCSCredentialProvider | None) –

    A callback to provide custom Google credentials.

  • kwargs (Unpack[GCSConfigInput]) –

    GCS configuration values. Supports the same values as config, but as named keyword args.

Returns:

  • None –

    GCSStore

from_url classmethod 

from_url(
    url: str,
    *,
    prefix: str | None = None,
    config: GCSConfig | GCSConfigInput | None = None,
    client_options: ClientConfig | None = None,
    retry_config: RetryConfig | None = None,
    credential_provider: GCSCredentialProvider | None = None,
    **kwargs: Unpack[GCSConfigInput],
) -> GCSStore

Construct a new GCSStore with values populated from a well-known storage URL.

The supported url schemes are:

  • gs://<bucket>/<path>

Parameters:

  • url (str) –

    well-known storage URL.

Other Parameters:

  • prefix (str | None) –

    A prefix within the bucket to use for all operations.

  • config (GCSConfig | GCSConfigInput | None) –

    GCS Configuration. Values in this config will override values inferred from the url. Defaults to None.

  • client_options (ClientConfig | None) –

    HTTP Client options. Defaults to None.

  • retry_config (RetryConfig | None) –

    Retry configuration. Defaults to None.

  • credential_provider (GCSCredentialProvider | None) –

    A callback to provide custom Google credentials.

  • kwargs (Unpack[GCSConfigInput]) –

    GCS configuration values. Supports the same values as config, but as named keyword args.

Returns:

obstore.store.GCSConfigInput

Bases: TypedDict

Configuration parameters for GCSStore.

There are duplicates of many parameters, and parameters can be either upper or lower case. Not all parameters are required.

Not importable at runtime

To use this type hint in your code, import it within a TYPE_CHECKING block:

from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
    from obstore.store import GCSConfigInput

BUCKET instance-attribute 

BUCKET: str

Bucket name.

BUCKET_NAME instance-attribute 

BUCKET_NAME: str

Bucket name.

GOOGLE_APPLICATION_CREDENTIALS instance-attribute 

GOOGLE_APPLICATION_CREDENTIALS: str

Application credentials path.

See cloud.google.com/docs/authentication/provide-credentials-adc.

GOOGLE_BUCKET instance-attribute 

GOOGLE_BUCKET: str

Bucket name.

GOOGLE_BUCKET_NAME instance-attribute 

GOOGLE_BUCKET_NAME: str

Bucket name.

GOOGLE_SERVICE_ACCOUNT instance-attribute 

GOOGLE_SERVICE_ACCOUNT: str

Path to the service account file.

GOOGLE_SERVICE_ACCOUNT_KEY instance-attribute 

GOOGLE_SERVICE_ACCOUNT_KEY: str

The serialized service account key

GOOGLE_SERVICE_ACCOUNT_PATH instance-attribute 

GOOGLE_SERVICE_ACCOUNT_PATH: str

Path to the service account file.

SERVICE_ACCOUNT instance-attribute 

SERVICE_ACCOUNT: str

Path to the service account file.

SERVICE_ACCOUNT_KEY instance-attribute 

SERVICE_ACCOUNT_KEY: str

The serialized service account key

SERVICE_ACCOUNT_PATH instance-attribute 

SERVICE_ACCOUNT_PATH: str

Path to the service account file.

bucket_name instance-attribute 

bucket_name: str

Bucket name.

google_application_credentials instance-attribute 

google_application_credentials: str

Application credentials path.

See cloud.google.com/docs/authentication/provide-credentials-adc.

google_bucket instance-attribute 

google_bucket: str

Bucket name.

google_bucket_name instance-attribute 

google_bucket_name: str

Bucket name.

google_service_account instance-attribute 

google_service_account: str

Path to the service account file.

google_service_account_key instance-attribute 

google_service_account_key: str

The serialized service account key

google_service_account_path instance-attribute 

google_service_account_path: str

Path to the service account file.

service_account instance-attribute 

service_account: str

Path to the service account file.

service_account_key instance-attribute 

service_account_key: str

The serialized service account key

service_account_path instance-attribute 

service_account_path: str

Path to the service account file.

obstore.store.GCSConfig

Bases: TypedDict

Configuration parameters returned from GCSStore.config.

Note that this is a strict subset of the keys allowed for input into the store, see GCSConfigInput.

Not importable at runtime

To use this type hint in your code, import it within a TYPE_CHECKING block:

from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
    from obstore.store import GCSConfig

google_application_credentials instance-attribute 

google_application_credentials: str

Application credentials path.

See cloud.google.com/docs/authentication/provide-credentials-adc.

google_bucket instance-attribute 

google_bucket: str

Bucket name.

google_service_account instance-attribute 

google_service_account: str

Path to the service account file.

google_service_account_key instance-attribute 

google_service_account_key: str

The serialized service account key

obstore.store.GCSCredential

Bases: TypedDict

A Google Cloud Storage Credential.

Not importable at runtime

To use this type hint in your code, import it within a TYPE_CHECKING block:

from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
    from obstore.store import GCSCredential

expires_at instance-attribute 

expires_at: datetime | None

Expiry datetime of credential. The datetime should have time zone set.

If None, the credential will never expire.

token instance-attribute 

token: str

An HTTP bearer token.

obstore.store.GCSCredentialProvider

Bases: Protocol

A type hint for a synchronous or asynchronous callback to provide custom Google Cloud Storage credentials.

This should be passed into the credential_provider parameter of GCSStore.

Not importable at runtime

To use this type hint in your code, import it within a TYPE_CHECKING block:

from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
    from obstore.store import GCSCredentialProvider

__call__ staticmethod 

__call__() -> GCSCredential | Coroutine[Any, Any, GCSCredential]

Return a GCSCredential.