Tutorial - deploy YOLOv5 with ncnn

https://github.com/Tencent/ncnn/discussions/4541

ncnn model制作(yolov5s.pt -> ncnn.param and ncnn.bin)

使用ncnn库编译后生成的工具

https://github.com/Tencent/ncnn/wiki/use-ncnn-with-pytorch-or-onnx

https://ncnn.readthedocs.io/en/latest/how-to-use-and-FAQ/use-ncnn-with-pytorch-or-onnx.html

pt -> onnx -> ncnn

pytorch to onnx

python export.py --weights yolov5s.pt --include torchscript onnx

onnx simplify

pip install onnxsim
python -m onnxsim resnet18.onnx resnet18-sim.onnx

or

pip install onnxslim
python -m onnxslim resnet18.onnx resnet18-slim.onnx

onnx to ncnn

onnx2ncnn resnet18-sim.onnx resnet18.param resnet18.bin

优化

https://github.com/Tencent/ncnn/wiki/use-ncnnoptimize-to-optimize-model

https://ncnn.readthedocs.io/en/latest/how-to-use-and-FAQ/use-ncnnoptimize-to-optimize-model.html

ncnnoptimize mobilenet.param mobilenet.bin mobilenet-opt.param mobilenet-opt.bin 65536 

量化

https://ncnn.readthedocs.io/en/latest/how-to-use-and-FAQ/quantized-int8-inference.html

create calibration table file

下载图片

https://github.com/nihui/imagenet-sample-images

find images/ -type f > imagelist.txt
./ncnn2table mobilenet-opt.param mobilenet-opt.bin imagelist.txt mobilenet.table mean=[104,117,123] norm=[0.017,0.017,0.017] shape=[224,224,3] pixel=BGR thread=8 method=kl

./ncnn2int8 mobilenet-opt.param mobilenet-opt.bin mobilenet-int8.param mobilenet-int8.bin mobilenet.table

使用pnnx工具

pt -> torchscript / onnx -> ncnn

https://github.com/Tencent/ncnn/wiki/use-ncnn-with-pytorch-or-onnx

https://github.com/pnnx/pnnx

https://github.com/Tencent/ncnn/discussions/4541

setup yolov5 pytorch

# checkout yolov5 v7.0 project
git clone https://github.com/ultralytics/yolov5
cd yolov5
git checkout v7.0

# install requirements
pip install -r requirements.txt --user

# download yolov5s weight
wget https://github.com/ultralytics/yolov5/releases/download/v7.0/yolov5s.pt

# test detection with pytorch weight, result saved to runs/detect/expN folder
python detect.py --source /home/nihui/nbs.jpg --weights yolov5s.pt --view-img

export torchscript and convert it to ncnn via pnnx

# export to torchscript, result saved to yolov5s.torchscript
python export.py --weights yolov5s.pt --include torchscript

# download latest pnnx from https://github.com/pnnx/pnnx/releases
wget https://github.com/pnnx/pnnx/releases/download/20230217/pnnx-20230217-ubuntu.zip
unzip pnnx-20230217-ubuntu.zip

# convert torchscript to pnnx and ncnn, result saved to yolov5s.ncnn.param yolov5s.ncnn.bin
./pnnx-20230217-ubuntu/pnnx yolov5s.torchscript inputshape=[1,3,640,640]