AI + Humans: How Artificial Intelligence scales Human Expertise for Detecting Changes to the Earth
Changes in land use and land cover are crucial for understanding socioeconomic behavior, as well as supporting various commercial use cases that require the latest infrastructure and connectivity data. Commercial mapping use cases, in particular, require an extremely high degree of fidelity and confidence, which includes the ability to track changes to key area features at massive scale. Features such as buildings and roads often change well before official mapping updates, which themselves can take substantial amounts of time and manual effort. Critical feature records may therefore be out of date, and if put into use lead to faulty analyses, planning, or operations.
Satellite hardware companies that have successfully launched fully operational constellations are extremely rare and provide a differentiated “high ground” solution to address these issues. Earth observation imagery from these will offer a good first source to search for new features and updates. This is dependent, however, on sufficient coverage over wide areas, as well as resources to search for and catalog relevant updates.
Orbital Insight is able to reliably monitor wide areas by stitching together imagery from multiple satellite providers and extracting features at scale with proprietary AI.
We have seen that official mapping records can be very out of date, especially in remote, rural areas. For example, Figure 1 shows a region where some mapping records maintain less than 20% of the actual road network, as detected by our fully automated analysis of recent satellite imagery. The same value proposition can extend to fast-developing urban areas, where manual updates may not keep pace with the rate of growth. Either way, this type of effort shows where existing records are out of date, and even provides a foundational dataset with which to update them.
However, the accuracy one can derive from satellites alone may not support all commercial use case requirements. In which case, satellite-based analyses can tip-and-cue other satellite sources, non-imagery geospatial signals, and/or manual follow-on efforts. Given the financial and time commitments required for some of these use cases, commercial mapping professionals need to deploy resources to those areas where building footprints or road networks have indeed changed.
We will go through a case study demonstrating how Orbital Insight’s access to multiple earth observation satellite constellations, high-performing artificial intelligence (AI), and software/systems engineering can help solve these challenges.
This pairing offers an unprecedented foundational lift that empowers the human domain expert to accomplish their mapping updates faster, cheaper, and better than they would sans this technology.
“The key is in finding the right balance, and, as unlikely as it sounds, there are lessons from chess. After Garry Kasparov’s infamous defeat against IBM’s Deep Blue, he went on to consider the possibilities offered by playing chess in partnership with computers rather than against them.
Rather than admitting defeat, he invented a new form of the game, called Advanced Chess, where a human and AI work together. The brute force analysis of the computer system together with the more strategic thinking of the human player has taken the game to heights of skill never seen before, and it’s now an active sport around the world.”
The above anecdote is a transferable analogy about pairing AI with human professionals. Just as AI is able to detect minutiae and tactical level insights to empower human experts in chess or in medicine, our AI can scan vast quantities of satellite imagery for pertinent features that an analyst can use to enhance their existing processes and workflows.
Commercial Mapping: Challenges
The requirements for advanced commercial mapping are increasingly rigorous, and there are limited options to meet them. More specifically:
- Commercial road mapping use cases, like those supporting autonomous car technologies, demand highly exquisite and accurate maps.
- It is extremely costly for people to drive every mile of road to search for changes; end-state systems may require even more precise measurements to actually update their records, beyond just detecting that change.
- A single satellite constellation’s cadence and fidelity cannot keep pace with road changes nor provide the necessary accuracy in some of these cases. This is particularly true for rapid, hyper-local development in urban areas, and also for rural/remote regions that are imaged even less frequently, as shown in Figure 2.
The need for both feature-level accuracy and nation-wide scalability makes Orbital Insight’s GO platform the ideal solution. By ingesting, interpreting, and analyzing exquisite high-resolution imagery combined with differentiated high-cadence imagery, commercial mapping companies are able to meet these challenges.
Commercial Mapping: Solution
Orbital Insight provides a systematic solution based on the partnerships and technologies we have cultivated, all accessed through our online GO Platform. We combine our self-developed AI algorithms, with integrated and scalable software, and leverage the multiple imagery constellations at our disposal.
The first, broad sweeping search for changes leverages exclusively AI and medium-resolution imagery. Medium resolution imagery has a significantly higher revisit rate, as shown in Figure 3, covering entire countries at the speed of change and in such high volume that human analysts are hard-pressed to search it rigorously and completely.
It is financially unfeasible for a human workforce to analyze this volume of imagery, however, AI can augment domain expertise. Figure 4 underscores the totality of this challenge, with a time series showing often several hundred thousand square kilometers of imagery collected daily. By extrapolating from previous hand-marking projects, we estimate it would take a team of 10 analysts working full-time approximately 388 business days to review the 124,000 tiles referenced in Figure 3’s caption above.
Our highly-trained computer vision algorithms allow us to interpret this wide-area imagery rapidly and at scale, detecting the same essential, optically recognizable features we would get from searching it by hand. We can also compress and compare different collection time windows and search for changes between them.
- Our AI can process this scale of information and deliver structured results in a matter of days. It could take over a year for a team of human analysts to search this same volume of imagery.
As noted earlier, however, the accuracy and fidelity of imagery are not always sufficient for updating datasets with more rigorous requirements. Our proprietary software lets human analysts easily iterate over and validate those results, leveraging multiple medium and high-resolution sets of imagery as well as other geospatial sources; Figure 5 below shows a sampling of what these different imagery sources look like.
Using these in various orders and combinations, we can derive a higher confidence dataset with relevant semantic changes. This can then be used to target changes/updates at the required level of rigor (hand annotation, custom collection, ground truth/field work, etc.). We capitalize on the relative advantages of these multiple sources and have engineered systems and software for human analysts take over when and how it is most advantageous to do so. The entirety of this process is outlined in Figure 6, including a generalized concept of operations and more specific anecdotes.
This process offers an iterative system of human-machine teaming powered by a virtual constellation of multiple satellite image providers, high-performance AI, and integrated validation software/processes. This will be demonstrated through a practical Case Study on how Orbital Insight leverages those elements to help our partners get highly accurate updates to countrywide road networks in a timely, cost-effective manner.
Case Study: Finding New Roads across Japan
Tier 1: Regions of Interest and Automated Land Use Analytics
The first step is to define the region we would like to search, or persistently monitor for land use updates. We can draw boundaries as large as entire cities, counties, or countries, and start an automated process that includes: ingestion from one or more satellite image providers, proprietary pre-processing and computer vision application, and advanced data science to determine changes from one time period to another. Figures 7 and 8 demonstrate the results of Orbital Insight’s land use classification algorithm, during a specific time interval; respectively, showing all land use classes across an entire city, and one specific sub-class (roads-linestrings) across a mixed urban/unpopulated area.
Through GO, and based on user-specified custom options, results like these can be automatically compared to other time periods to determine changes across semantic land use classes. This typically results in more changes identified, and far more quickly, compared to human review of such sources.
Tier 2: Efficient Validation of Changes Detected
The next step is to quickly validate changes that were detected from one time period to another, in this case: those involving road features specifically. Our proprietary data annotation software supports this in a rapid, easy-to-use and team-scalable interface. Figure 9 below shows how we provide a system for quickly visiting each detected change, and validating it using various imagery catalogs. The success of this model is predicated on our partnerships — the ability to access multiple image providers’ products — and engineering aptitude to cross-analyze these with Tier 1 results. Multiple users can conduct this process in-tandem with one another which also helps improve the pace and scalability of such efforts.
OI’s internal software, as shown in Figures 9 and 10:
- Seamless management of multiple users and thousands of images
- Interactive GUI
- Users can toggle quickly between multiple data types
- Rapid and straightforward QA for land use management