NASA's pioneering platform for Earth Observation Data. The cloud-native, open-source VEDA platform revolutionizes open science and maximizes the return on investment in the commercial cloud by offering highly reusable tools in your browser for more efficient and scalable data interaction, analysis, visualization, processing, and collaboration. Designed to meet the diverse needs of the Earth observation community, VEDA facilitates optimized, user-friendly data exploration, enabling scientists and stakeholders to drive forward their research and missions together.
Overview
VEDA's primary goal is to increase efficiency and reduce compute costs for the migration of NASA’s Earth science data to the cloud by optimizing data formats and data delivery, and leveraging open standards to promote data interoperability. Enabling scientists to collaborate with each other through cloud-native solutions accelerates open science and provides a single point of entry when connecting with stakeholders.
Challenge
NASA has invested significant resources in storing its vast data archive in the cloud, but the data isn't always accessible in cloud-native, optimized formats. The Earth observation community needed a platform offering a unified, interoperable data catalog and a collaborative computing environment for efficient data analysis, processing, and sharing.
Outcome
A centralized, but modular platform for cloud-hosted data that enables stakeholders to maximize cloud capabilities for accessing, analyzing, processing, and communicating scientific data and information. In collaboration with NASA’s Office of Data Science and Informatics (ODSI), this project continues to innovate new technologies for optimized data access, analysis, and visualization through a standards-based architecture and an interoperable data catalog, expanding the capabilities to support the mission of federal and commercial partners who utilize NASA Earth science data. This set of highly reusable components and capabilities are drawing attention from other science teams eager to accelerate their own missions.
VEDA Goals
The VEDA Platform has two primary goals:
- Demonstrate the benefits of cloud-native data publication and access for open science to help official data providers transition to cloud-native infrastructures
- Support the NASA mission in building more modern, scalable, and resilient data infrastructures, leveraging the benefits of the cloud, and enabling collaborative open science on Earth observation data
OOur primary hypothesis for how to achieve these goals is to create a reusable cloud data system that enables a reinforcing loop connecting the work of scientists with the interests of stakeholders.
Open Science lifecycle using VEDA
Like much of the Earth observation (EO) community, NASA has a wealth of data and talented individuals who know how to derive and share insights from this data. However, traditional data access patterns do not leverage the benefits of the commercial cloud to increase scalability and efficiency in scientific research. Off-the-shelf software provides limited solutions, and building a new platform for every use case isn’t feasible. A gap exists in the EO community for a flexible, open-source, yet powerful platform for scientists to access data, analyze that data to create new data products, and share their work with stakeholders efficiently. VEDA seeks to fill this gap with the following guiding principles are to:
- Reuse, combine, and improve existing open-source development efforts
- Develop using the latest open-source standards
- Develop reusable components that the EO community can repurpose for their use cases
- Innovate upon existing cloud-based solutions to more efficiently support the science research life cycle
Developing a Large Open-Source Product
The VEDA project is a large project involving many stakeholders, teams, and other organizations.
- We manage this complexity with a flexible, transparent agile product development approach
- We establish a flexible design and architecture from the beginning, using user research and validation to build towards value
- The team focuses on shipping software, leaning into our core value of bias to action
Because of its size, VEDA is organized into three major components: Data Services, Dashboard, and Analytics Platform.
The method for transforming, ingesting, and accessing Analysis-Ready, Cloud-Optimized (ARCO) data is the VEDA Data Services. The ARCO datasets get published to a Spatio-Temporal Asset Catalog (STAC), which is a community-driven open-source standard and enables us to leverage other community-developed tools such as TiTiler and pySTAC. Newer technologies enable VEDA Data Services to directly access the NASA data archive, reducing the need for data duplication. This saves NASA time and money. Using community-backed standards and open source tools is vital to VEDA because, once ingested, this data must be accessed by a variety of tools for many different purposes.
The VEDA Dashboard reveals the utility of the data. The dashboard is a blueprint for science teams to use open source technology to enable their users to browse, analyze, and explore their data in the browser. It is easily configurable and reproducible, making it easier than ever for scientists to connect directly with their stakeholders without worrying about capacity building or data access issues.
Once a scientist decides to perform a deeper, more complex analysis, they can navigate to a JupyterHub environment where they can access and analyze VEDA data. This is the VEDA Analytics Platform. Over the past several years, this architecture has been prototyped, tested, and improved to ensure it is beneficial for users and stakeholders. This is vital for open science, ensuring that the science is fully reproducible.
VEDA is an open-source project, and we welcome anybody to view or contribute to these repositories. Visit the VEDA Documentation for a list of our GitHub repositories.
Results
The VEDA project has seen success by enabling scientists to visualize and share their work more efficiently, while providing program managers with a centralized location for research teams to implement the science lifecycle. The development and adoption of open-source platforms such as VEDA are powerful for the future of our planet. As the world comes to terms with the massive environmental and social challenges ahead of us, tools that allow flexible and engaging visualizations will be an important part of our collective response. Thanks to the vision and partnership from NASA ODSI, we now have an excellent example of one such tool in VEDA.
About NASA ODSI (formerly known as IMPACT)
NASA’s Marshall Space Flight Center Office of Data Science and Informatics (ODSI) is the agency’s premier center for data science innovation, driving groundbreaking scientific discoveries, and pioneering technological advancements and applications across all scientific fields. ODSI partners with Development Seed to advance NASA science through enhanced data science infrastructures and informatics, providing cutting-edge expertise, tools, and capacity building to support key programs and drive transformative scientific breakthroughs.
Have a challenging project that could use our help?
Let's connect
We'd love to hear from you.