pytorch和deep learning技巧和bug解决方法短篇收集

有一些几句话就可以说明白的观点或者解决的的问题，小虎单独收集到这里。

torch.hub.load how does it work

下载预训练模型再载入，用程序下载链接可能失效。

model = torch.hub.load('ultralytics/yolov5', 'yolov5s')

model = torch.hub.load('ultralytics/yolov3', 'yolov3')

I found yolov3.pt in Release.
参考：
https://discuss.pytorch.org/t/using-torch-load-on-a-torch-hub-model/89365
https://docs.ultralytics.com/yolov5/tutorials/pytorch_hub_model_loading/#before-you-start

No module named ‘models’

In yolov3 ~ yolov7 repo directory,

import torch
# _model = torch.hub.load('ultralytics/yolov5', 'yolov5s') # _model is the model itself
model = torch.load("yolov3.pt", map_location='cpu')['model']
torch.save(model.state_dict(), "yolov3.pth")

But the pretrained weight seems not matched to yolov7_d2. Funny design of yolo pretrained model.

The following submodules of the model were never called during the trace of the graph.

定义的submodules没有用到，可能出现在ddp的时候。移除即可。

 The following submodules of the model were never called during the trace of the graph. They may be unused, or they were accessed by direct calls to .forward() or via other python methods. In the latter case they will have zeros for statistics, though their statistics will still contribute to their parent calling module.

Inplace update to inference tensor outside inferencemode

Use @torch.no_grad() instead of torch.inferece_mode().

Inference mode vs no_grad()
Inplace update to inference tensor outside InferenceMode is not allowed.You can make a clone to get a normal tensor before doing inplace update.See https://github.com/pytorch/rfcs/pull/17 for more details.

One difference would be that you are not allowed to set the requires_grad attribute on tensors from an inference_mode context:

with torch.no_grad():
    x = torch.randn(1)
    y = x + 1

y.requires_grad = True
z = y + 1
print(z.grad_fn)
> <AddBackward0 object at 0x7fe9c6eafdf0>

with torch.inference_mode():
    x = torch.randn(1)
    y = x + 1

y.requires_grad = True
> RuntimeError: Setting requires_grad=True on inference tensor outside InferenceMode is not allowed.

In a word, outside inference mode, inference tensor is still restricted.