Semantic Segmentation of Aerial/Satellite imagery

Feasibility study:
Infrastructure segmentation for rapid emergency response coordination through deep learning

Stack:

       
Dr. Markus Meingast 2019

The Questions

  • What is semantic segmentation?
  • What is it used for?

Pixel-wise classification: Where is a dog?

raw input
semantic segmentation

Roof-top Segmentation

AIRS-Dataset

    Raw:
  • 475km2 of aerial imagery (Auckland, NZ)
  • 7.5cm/px resolution
  • 850 images/16GB
    Processed:
  • downsampled to 30cm/px
  • random brightness (+/-30% variation)
  • random uniform-blurring
  • 25.000 samples at 256x256px

U-Net Model

  • Encoder / Decoder type neural net (with skip-connections)
  • 39 layers with 2.000.000 parameters
  • heavy lifting done on Google Colab (GPU)
  • Loss function: Binary Crossentropy
  • Optimizer: Adam
  • Metrics: Accuracy / DICE

Prediction

raw input
base truth
prediction

We can segment anything!

What about roads?

Ottawa-Dataset

raw input
base truth
prediction

What about cars?

Potsdam-Dataset

raw input
base truth
prediction

Let's combine them

  • Training model on combined dataset mostly infeasible
  • Model compiled for EdgeTPU for inference
  • Model input layers share input, out is concatenated
  • TF/Keras model exported to TFLite --> compiled for EdgeTPU

Live inferencing via Google Coral EdgeTPU Accelerator

Next Steps

  • Segment infrastructure damage
  • More data, more data, more data!
  • Try different models / model complexity (GAN / pix2pix?)

Thank you for your attention