Results evaluation

The achieved results are compared with the ground truth. Below the results in terms of Overall Accuracy, F1 Score, Precision, Recall and Intersection over Union (IoU).
All the metrics are weighted.

In the Training scene column is specified the number of scenes used in the training phase.
For the Test scenes, “A” stands for A_SMG_portico scene, while “B” stands for B_SMV_chapel_27to35 scene. See Dataset page for specs.
The base configuration on the NNs features is x, y, z, r, g, b, Nx, Ny, Nz. If different features are added is specified in the table and further info (features type, class metrics, confusion matrix, hardware and so forth) can be found at the link in the last column.

Training scenesTest sceneFeaturesOverall AccuracyIoUPrecisionRecallF1-scoreFurther Info
DGCNNallBNo r, g, b and Normals0.7520.3530.7710.7520.740Info
DGCNNallANo r, g, b and Normals0.7840.3760.8220.7840.794Info
DGCNN-Mod+3DfeatallB3D features0.8650.5560.8530.8640.856Info
DGCNN-Mod+3DfeatallA3D features0.9140.6000.9170.9150.915Info