Recent developments and a future vision for OpenStreetMap validation and infrastructure.
Billions of people around the world rely on OpenStreetMap (OSM). From supporting life-saving humanitarian projects to enabling Amazon package delivery, OSM continues to empower communities and provide a critical digital public good for humanity.
It’s more important than ever to ensure the quality of edits in OpenStreetMap. More than 3 million map features are created, modified, or deleted from the map everyday. Keeping track of these changes to ensure the map's data quality is challenging. OSMCha was created in 2015 to help the OSM community with that task. Use of OSMCha has grown exponentially since then.
OSMCha has become one of OpenStreetMap's core validation tools. In the past 12 months, OSMCha was used for more than 58,000 quality checks and is widely used by organizations like Apple, Meta, the Humanitarian OpenStreetMap Team, and the larger OpenStreetMap community.
OSMCha logs each OSM changeset in a database and provides an interface to search and visualize those edits. It allows users to see exactly what was changed and easily identify potential problems.
It’s more important than ever to ensure the quality of edits in OpenStreetMap.
Mapbox was one of the first organizations to recognize the value of a tool like OSMCha and make use of it. They adopted it as part of their QA workflow, and invested financial and engineering resources to grow the project. They supported the development, maintenance, and infrastructure costs of OSMCha for over eight years. Under their support, OSMCha grew from a prototype to being a platform capable of processing OSM data in near real-time and store information about millions of changesets. We worked closely with folks at Mapbox to help us transition OSMCha to OpenStreetMap US stewardship which they supported as an important step towards the long term sustainability and community support of the project. We are grateful for Mapbox’s commitment and support all these years.
Becoming a Community Supported Project
Development Seed has also carried on support for OSMCha given that three primary maintainers (Wille, Sajjad & Sanjay) are currently on our team. The goal has always been for OSMCha to be governed, maintained, and influenced by the OSM community. At the beginning of 2023, OSMCha became a charter project of OpenStreetMap US, a non-profit organization created to support and grow the OSM community in the United States. Meta generously supported OSM-US by covering the costs associated with moving OSMCha to OSM-US infrastructure and important database improvements. A few months ago, in partnership with Mapbox, we completed the migration of OSMCha to infrastructure owned by OSM-US. Anyone can now directly support OSMCha. This was an exciting moment in fulfilling the long-term vision for OSMCha!
Running infrastructure that keeps track of every change in OSM is hard to scale with limited resources. Through this migration, we made some key changes to the OSMCha infrastructure that put us on a path to sustainability. We upgraded OSMCha changeset processing and core services to use Kubernetes and AWS Open Data program donated storage to host the minutely changeset history that includes full features. We made many database optimizations and brought down the RDS cost by a third. We are also excited about the new pipeline that produces cloud-native OSM change data that will enable new, high-performance approaches to validating OSM data.
Figure 1: Example of a vandalism visualized in OSMCha where the name for the Taj Mahal was changed.
What's Next
OSMCha is only one piece of how we think the OSM ecosystem should be planning for a foundation of coordinated tools and processes for QA and validation in OSM. Just last year, the Taj Mahal was vandalized, identified and fixed by the OSM community in Kerala, India. This is a concrete example of targeting a significant feature on the map that could go unnoticed for weeks — the change was only caught 13 days later. We still don’t have a consolidated repository of issues, there’s no coordinated priority queue of identified issues to fix, and there are no open ML-based approaches. We need tighter integration between mapping and validation tools - a great example of what we should be working towards is how Rapid now integrates MapRoulette directly into the editor. We should develop a suspicious feature schema to publish information about features that fail validation checks which can then be prioritized by different community initiatives.
Marc Farra and I will be at State of the Map US later this week, discussing the modern OSMCha pipeline and a new application, Gradient which demonstrates how new data assets from OSMCha can scale validation and metrics of OSM data in new ways. We’re looking forward to lots of great conversation around the future of validation in OSM.
We need tighter integration between mapping and validation tools.
What we're doing.
Latest