Inference#
With trained detection and classification models, we can now combine the two in stepwise manner to predict classified building facades.
The main workflow here entails:
Detect building facades in streetview images
Clip the images by detected building (using again, a buffer of 100 pixels to be consistent with what the classifier expects)
Classify building properties from the clipped image that focuses on the detected building
From this, we can attribute building facades with the properties that we trained for: completeness, condition, material, use and security.
How to run#
Please ensure you have your streetview images on hand along with the trained model weights.
The final detection model weights are located at s3://hp-deliverables-v2/detectron2_model/50000iter/model_final.pth.
The final classification model weights are located at s3://hp-deliverables-v2/classifier_models/512/final/63869a95ae444b3d87f78331/checkpoints/epoch:48-step:12838-loss:2.419-f1:4.245.ckpt.
To run the script detect_clip_classify.py, execute it from the command line with the following arguments:
IMG_DIR: Directory containing dataset images.
DET_CPKT_PATH: Path to the object detection model checkpoint file.
CLASS_CPKT_PATH: Path to the classification model checkpoint file.
OUTPUT_DIR: Directory to save output files (clipped images and predicitions csv (
detection_classification_predictions.csv)).
Usage: python detect_clip_classify.py <IMG_DIR> <DET_CPKT_PATH> <CLASS_CPKT_PATH> <OUTPUT_DIR>
Example command:
python detect_clip_classify.py images/ detector_model.pth classification_model.pth output/
The detection_classification_predictions.csv file contains fields for the original streetview image name, the predicted bounding box, the predicted bounding box score, each predicted building property, as well as the name of the clipped image name.
Visual results#
Un-clipped, with boxes and classes:#
Clipped, with classes#
