A Modern Imagery Processing Pipeline


3 min read

Satellite data is a tremendously powerful resource for governments and development organizations. We built a suite of tools to make open Landat data more accessible and useable. These allow our development partners to process imagery and perform analysis quicker, and that can make all the difference in rapidly evolving situations.

Often our partners need commercial imagery with greater resolution and refresh times than what Landsat 8 offers. We have great partnerships with commercial imagery providers, to offer all sorts of imagery. Too often receiving and processing commercial imagery is a huge pain point that slows us down and makes it harder to make use of the data. As developers we know it could be better.

Astro Digital gave us an opportunity to rebuild this workflow from the ground up. We’ve worked closely with their team to build a satellite imagery pipeline for developers and end users. We just launched a browsing and publishing platform with Astro Digital to allow anyone to discover, process, and share satellite imagery in an incredibly quick and intuitive manner. A process that previously could take days has now been cut down to minutes.

API first

We built an end-to-end data processing pipeline that feeds a powerful data API that unlocks possibilities for others. We broke down the fundamental goals of the Platform and built API calls around each. Those goals were to search, process, and publish.

We built and exposed a perfomant Elasticsearch-powered endpoint, based on our previous landsat-api work, that will allow for complex queries to find exactly the data that is needed.

`$ curl [https://api.astrodigital.com/v1/search?search=cloudCoverageFull:[0+TO+20]`](https://api.astrodigital.com/v1/search?search=cloudCoverageFull:[0+TO+20])

But how to process the imagery? We extended our existing open source landsat-util tool to handle varying band combinations and the API offers several including true color, vegation health false color and urban false color.

`$ curl [https://api.astrodigital.com/v1/methods`](https://api.astrodigital.com/v1/methods)

And finally, there is a simple request that can be made to process the imagery and receive a tiled map URL. This URL can be used with tools like Mapbox or Leaflet to build upon the processed imagery in any way. Full documentation, including interactive samples, can be found at docs.astrodigital.com.

`$ curl -X POST --data "sceneID=LC80430332014262LGN00&process=urbanFalse&satellite=l8" [https://api.astrodigital.com/v1/publish`](https://api.astrodigital.com/v1/publish)

{"status":"Image is being processed."}

Frictionless publishing

We are using this data pipeline to power an extremely easy and visual imagery browser and publishing tool. We started with Libra as a base and modified it to meet the Astro Digital specific workflow. Libra was already designed to be quick and intuitive. We added a simple publish workflow that will process and publish images and email a link to the tiled map after processing has completed. For images processed within this visual workflow, the email contains a link to an embeddable map that can be used anywhere across the web.

Working with Astro Digital, we built a modern publishing pipeline that we hope will push the entire industry to build more usable tools. This is good for the industry and good for users, particularly the small governments and development organizations that are the next wave of power satellite data users.

What we're doing.