Python影像辨識筆記(十三):計算YOLOv4 Weights的mAP

2020 / 06 /11更新重要觀念:

Ripley, B.D(1996), Pattern Recognition and Neural NetworksTraining set : A set of examples used for learning , which is to fit the parameters [i.e., weights] of the classifier.Validation set : A set of examples used to tune the parameters [i.e., architecture, not weights] of a classifier, for example to choose the number of hidden units in a neural network.Test set : A set of examples used only to assess the performance [generalization] of a fully specified classifier.簡單來說:
Training set做訓練。
Validation set在訓練時調整超參數。
Test set跟Training set完全不一樣,只拿來評估Model的mAP。
切勿將Val set拿去測試mAP,出來結果是錯誤的
不行,因為官方沒有提供test set的label,只能透過COCO的Server去評估mAP

How to evaluate AP of YOLOv4 on the MS COCO evaluation server

https://github.com/AlexeyAB/darknet/blob/6f718c257815a984253346bba8fb7aa756c55090/cfg/yolov4.cfg#L8-L9
test集圖片中的txt和jpg要擺在同一個資料夾裡面。http://images.cocodataset.org/zips/test2017.zip
https://raw.githubusercontent.com/AlexeyAB/darknet/master/scripts/testdev2017.txt
#將train、valid改成屬於自己路徑的資料夾
classes= 80
train = /yanwei.liu/darknet/coco/trainvalno5k.txt
valid = yanwei.liu/darknet/coco/testdev2017.txt #要用test set
names = /yanwei.liu/darknet/data/coco.names
backup = /yanwei.liu/darknet/backup/
eval=coco
trainvalno5k.txt的範例內容(只顯示一筆)
/yanwei.liu/darknet/coco/images/train2014/COCO_train2014_XXXXXX.jpg
testdev2017.txt的範例內容(只顯示一筆)
/yanwei.liu/darknet/coco/images/test2017/XXXXXX.jpg
mkdir results
./darknet detector valid cfg/coco-mAP.data cfg/yolov4.cfg yolov4.weights
$ cd results
$ mv coco_results.json detections_test-dev2017_yolov4_results.json
$ zip detections_test-dev2017_yolov4_results.zip detections_test-dev2017_yolov4_results.json
該網站需要註冊帳號才能使用https://competitions.codalab.org/competitions/20794#participate取得以下結果 (AP=0.435 and AP50=0.657) View scoring output logoverall performance
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.435
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.657
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.473
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.267
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.467
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.533
Average Recall (AR) @[ IoU=0.50:0.95 | area= all |
maxDets= 1 ] = 0.342
Average Recall (AR) @[ IoU=0.50:0.95 | area= all |
maxDets= 10 ] = 0.549
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.580
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.403
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.617
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.713
Done (t=334.58s)

Machine Learning | Deep Learning | https://linktr.ee/yanwei