Blog post cover image

There has never been a better time to be working with earth observation data.

Every year brings more satellites, richer datasets, and better open source tools for analyzing our planet. Cloud-native data formats and AI-assisted programming have made it easier than ever for scientists, developers, and analysts to turn data into insight.

Yet one challenge remains surprisingly difficult: finding the right dataset in the first place.

Before you can decide which dataset to use, you first need to know where to look. With hundreds of public STAC APIs, each containing dozens to thousands of collections, simply discovering what's available can become a research project of its own.

We believe there is an opportunity to improve this experience in two ways:

  • Broader adoption of Collection Search in STAC APIs
  • Federated search interfaces that let users search across many catalogs at once

We encountered this challenge while working with scientists on the [Multi-Mission Algorithm and Analysis Platform (MAAP), who regularly combine datasets from NASA, ESA, and many other providers.

Why finding data is still hard

Today, a scientist trying to answer a question like "Identify forest harvesting events in the US South during 2025" often follows a workflow like this:

  1. Determine what types of data could answer the question.
  2. Search documentation, research papers, Google, AI tools, forums, and catalogs to identify candidate datasets.
  3. Evaluate which sources provide the necessary processing level, cloud access, and APIs.
  4. Finally begin working with the data.

Even after all that work, there's no guarantee you've found the best dataset.

Maybe you're a student who doesn't yet know the major data providers. Maybe you've overlooked a catalog that contains exactly what you need. Or perhaps you've found SAR imagery but missed that another provider offers radiometrically terrain-corrected data that would save significant preprocessing effort.

Finding data shouldn't depend on already knowing where to search.

Federated STAC collection search for MAAP

The Multi-Mission Algorithm and Analysis Platform (MAAP) is a collaborative open-science environment jointly developed by NASA and ESA.

MAAP scientists routinely combine datasets from many organizations, including NASA's Harmonized Landsat Sentinel-2 (HLS), ICESat-2, ESA's Biomass mission, and many others. While these datasets are increasingly available through STAC APIs, discovering complementary collections and comparing available options across providers still requires considerable time and effort.

To help address this challenge, we worked with MAAP scientists to improve data discoverability by building open source tools for federated STAC collection search.

These tools make it possible to search collections across multiple STAC APIs simultaneously without first knowing which catalog contains the data you're looking for.

STAC Collection Discovery

Preview of STAC Collection Discovery website
  • stac-fastapi-collection-discovery provides a lightweight STAC API implementation focused exclusively on collection search. It forwards queries to multiple upstream STAC APIs and combines the results into a single response.
  • the STAC Collection Discovery web app provides a web interface that lets users visually search collections across many STAC APIs using that federated search service.

How it works

The underlying technology is intentionally simple. Rather than maintaining its own copy of collection metadata, stac-fastapi-collection-discovery forwards each search request to a configurable set of upstream STAC APIs, combines the responses, and returns the results directly to the user. Because the service remains stateless, results always reflect the current state of the upstream catalogs without requiring synchronization or scheduled indexing. The searches are idempotent and are performed in parallel so requests can be processed very quickly.

The service also inspects each upstream STAC API to determine which search capabilities are supported, allowing it to expose only the parameters that are common across the selected catalogs.

Since the service presents a standard STAC API interface, it can also be used with existing tools like STAC Browser and pystac-client.

Collection Details

Preview of collection details popup

In addition to collection metadata, the interface includes code hints that help users quickly start querying items directly from the source API.

Code Hints

Code hint example showing how to start an item search

Once you have found the collection(s) you need, the STAC Collection Discovery app has done its job and it is time to pick up your item search tool of choice to track down the specific items/assets that you need.

You can explore the application yourself at https://discover.maap-project.org.

Toward Federated STAC Discovery

Data discoverability is a pillar of the FAIR Data Principles (F is for Findable) that many data providers aim to uphold, but how findable is a dataset if a user needs to know a specific website or search interface in order to find it? The rise of the SpatioTemporal Asset Catalog (STAC) metadata specification and the STAC APIs as the catalog interface of choice for many data providers has helped reduce the cognitive load on users once they have identified a data source by enabling them to use a common set of tools to access data from many providers. However, to get to this point a user needs to know which STAC API and data collection they want to use out of the hundreds of public STAC APIs out in the world, each containing dozens to thousands of data collections.

Imagine a world in which all earth observation data collections are represented in STAC APIs. Since all of these STAC APIs follow the STAC API specification and implement the collection-search and free-text search extensions you could theoretically run a single search for data across all of these APIs from a single Federated STAC API. If this were possible, it would pave the way for powerful natural language search interfaces that could efficiently trawl through the sea of collections available in public STAC APIs.

This vision is already beginning to take shape. There is one existing project that has collected a vast array of collections from all of the known sources in stacindex.org: https://atlas.stacindex.org/. The STAC Atlas is built by periodically scanning all of the registered STAC APIs and static catalogs and incorporating the collection records into a central location which is searchable as a STAC API with collection search and free-text search enabled.

Where We Go From Here

Federated search depends on rich, searchable metadata. As more organizations publish data through STAC APIs, enabling Collection Search and Free Text Search extensions will make those datasets easier for both people and applications to discover.

These extensions make the collections in your STAC API searchable via free-text search and they are already implemented in two of the most widely used STAC API backend frameworks: (stac-fastapi-pgstac and stac-fastapi-elasticsearch-opensearch) so if you are using one of those frameworks you can upgrade to a recent version and make sure those extensions are enabled.

Enabling these extensions will make the collections in your STAC API searchable which means users and client applications will be able to perform targeted spatial, temporal, and text-based searches on the collections instead of requesting all of the records and sifting through them for relevant results. We're excited to continue contributing to the open tooling that makes this possible. If you're building Earth observation data platforms or thinking about improving discoverability across your own catalogs, we'd be happy to compare notes.

What we're doing.

Latest