hydraters

Hydrate Python dictionaries with Rust. A general-purpose algorithm, used in pgstac to reduce the size of the items table.

import hydraters

base = {"a": "first", "b": "second", "c": {"d": "third"}}
item = {"c": {"e": "fourth", "f": "fifth"}}
result = hydraters.hydrate(base, item)
assert result == {
    "a": "first",
    "b": "second",
    "c": {"d": "third", "e": "fourth", "f": "fifth"},
}

If you want to clean out any lingering DO_NOT_MERGE_MARKER entries after hydration, pass strip_unmatched_markers=True. This will call hydraters.strip_unmatched_markers on the hydrated item before it is returned and will emit the same warning when markers are stripped.

import warnings
import hydraters

item = {"a": hydraters.DO_NOT_MERGE_MARKER, "b": {"c": "value"}}

with warnings.catch_warnings():
    warnings.simplefilter("ignore")
    hydraters.hydrate({}, item, strip_unmatched_markers=True)

assert item == {"b": {"c": "value"}}

Strip existing data back to a clean state by removing any keys whose value is DO_NOT_MERGE_MARKER. A warning lists every stripped path using JSONPath dot notation (for example $.assets[0].href).

item = {
    "a": hydraters.DO_NOT_MERGE_MARKER,
    "b": {"c": hydraters.DO_NOT_MERGE_MARKER, "d": 1},
}

hydraters.strip_unmatched_markers(item)
assert item == {"b": {"d": 1}}

Installation

python -m pip install hydraters

Or, if you're using uv:

uv add hydraters

Developing

Get Rust and uv. Then:

git clone git@github.com:developmentseed/hydraters.git
cd hydraters
uv sync
uv run pre-commit install

To run tests:

uv run pytest

Background

The code for this package was taken from pypgstac. It came from some benchmarking that determined it was much faster to do this operation in Rust than in pure Python.

License

MIT