Results evaluation
The achieved results are compared with the ground truth. Below the results in terms of Overall Accuracy, F1 Score, Precision, Recall and Intersection over Union (IoU).
All the metrics are weighted.
In the Training scene column is specified the number of scenes used in the training phase.
For the Test scenes, “A” stands for A_SMG_portico scene, while “B” stands for B_SMV_chapel_27to35 scene. See Dataset page for specs.
The base configuration on the NNs features is x, y, z, r, g, b, Nx, Ny, Nz. If different features are added is specified in the table and further info (features type, class metrics, confusion matrix, hardware and so forth) can be found at the link in the last column.
| Training scenes | Test scene | Features | Overall Accuracy | IoU | Precision | Recall | F1-score | Further Info | |
|---|---|---|---|---|---|---|---|---|---|
| DGCNN | all | B | No r, g, b and Normals | 0.752 | 0.353 | 0.771 | 0.752 | 0.740 | Info |
| DGCNN | all | A | No r, g, b and Normals | 0.784 | 0.376 | 0.822 | 0.784 | 0.794 | Info |
| DGCNN-Mod | all | B | – | 0.837 | 0.470 | 0.829 | 0.837 | 0.823 | Info |
| DGCNN-Mod | all | A | – | 0.896 | 0.535 | 0.893 | 0.896 | 0.892 | Info |
| DGCNN-Mod+3Dfeat | all | B | 3D features | 0.865 | 0.556 | 0.853 | 0.864 | 0.856 | Info |
| DGCNN-Mod+3Dfeat | all | A | 3D features | 0.914 | 0.600 | 0.917 | 0.915 | 0.915 | Info |