We recently joined Esri UK to take a deep dive into the technical side of our work. Our team demonstrated the tools and techniques we used in our work for The Norwegian Refugee Council following the devastating drought that happened in Afghanistan in 2018. The demonstration included how we used spatial data science, tools such as Esri’s ArcGIS and machine learning to count tents in IDP camps.
There wasn't enough time to answer all of the questions at the event and so we have answered them below. To view the recording of the event, please click on the image or here
Could you please provide us with the link to the published work and codes? many thanks.
How much time did you get to train the detection model using ArcGIS Pro?
The time required to train a detection model in ArcGIS Pro mainly relied on the machine specs. Also, it depends on how many features you have and can be used to export training datasets. The input training dataset in the case of tents were around 1,000 feature and it took 2 hours on our powerful machine to train. This does not include the time needed to export the training dataset (~20 mins) and running the inference tools.
Madeline, fantastic presentation. Who supplied your satellite imagery and what type of resolution do you recommend for this type of analysis.
The satellite imagery was provided by Maxar (WorldView2). The suitable image resolution depends on the dimensions of the object you are going to detect. For example, tents and solar panels are relatively of a small dimension so 0.5 - 1 m resolution would be good. However, crop fields for instance are of a much greater dimension, so 5 m resolution might be a good start.
What were the results of the tent mapping and how did this help NRC?
The results of the project were that we have successfully devised a method for monitoring IDP tents from satellite imagery using ML, in a way that was time efficient, cost effective, repeatable and produced more than satisfactory results (Out of 3,447 tents, we managed to perform the segmentation with average intensity of 97% and average sensitivity of 98.6%). The NRC used the data we provided from our results to better coordinate and prioritise their aid distribution among the many camp locations, as well as to help plan the reintegration of IDP's from their camps back to their homes.
How can this capability be turned to other objects with high levels of confidence?
The golden rule of any image/instance segmentation workflow is (the more training dataset that you have, the better results you are going to get). The embedded deep learning model architectures in ArcGIS pro can be trained to detect any kind of features. You will also need images of a relatively high resolution and maybe some image processing techniques to better visualize the object of interest for a better training results and accuracy.
Are there any other scenarios where you could see this approach working effectively?
Absolutely yes! Object detection could be very helpful in many different and complex scenarios. For example, Public Health and Mosquito Control agencies are responsible for providing the highest level of protection from vectors and vector-borne diseases. Therefore, detecting the neglected pools is very essential task as they might be breeding grounds for mosquitoes. Below is a link for a Medium article explaining the whole scenario:
https://medium.com/geoai/swimming-pool-detection-and-classification-using-deep-learning-aaf4a3a5e652
Also, you can find many similar articles in Esri's GeoAI blog on Medium:
Is the precision 97% +referenced in one of the slides the recall rate of the classifier?
The figure 97% or more accurately 0.97 is actually referring to the confidence score that is defined as the probability that an anchor box contains an object. It is usually predicted by a classifier. It also can be derived from a relation between the precision and recall of the classifier and it’s calculated as an attribute field within the output of the inference tool. This can also help us define a threshold below which, detections can be neglected.
Worth mentioning that as a GIS company, we pay more attention to how features are plotted and well-delineated on the map without missing any feature of interest.
To Madeleine: What was the area classified? In your experience is the performance of the model dependent on the landscape/imagery classified being similar to the landscape and imagery the training data is derived from?
The area classified was around the town of Qala-e-Naw, which is the district centre of Badghis province. Ideally, the performance of the model is significantly affected by the varied training dataset that's fed to the neural network during the training phase. If the object of interest is usually existing within different backgrounds and surroundings, so it's better to have a well-varied training dataset in order for the model to be able to learn/recognize them.
For the tent classification problem, what other features beyond colour pixel values did your ANN take into account during training?
In fact, it's the nature of the tents that was most challenging. The tents are usually overlapping when visualized in the satellite images. Therefore, the DO-U-Net architecture was proposed in order to have two outputs (Segmentation mask and Outline mask) that can be subtracted and resulting in the non-overlapping shapes.