很简单，自定义了一个 dataloader，出现以下不易察觉 bug
inputs 维度：[bs, 4]，这个没问题
labels 维度：正确应该是 [bs, 1]，但是 dataloader 出来是 [bs]
模型的 outputs 维度：[bs， 1]
如果用 torch.mean(torch.abs(labels - outputs)) 计算 L1 Loss / MAE
由于 pytorch 的广播机制，torch.abs(labels - outputs) 变成了一个 bs * bs 的矩阵，然后计算了这个矩阵的均值，直接变成对比学习了。
怎么解决这个问题呢：
在传入 dataloader 时，就令 labels 的维度是 [n, 1]，而不是 [n]
怎么避免这个问题呢：
使用 F.l1_loss()，会发现这样的情况会直接报错，因为它会检查两个 tensor 的维度是一致的：
所以在自己手搓 Loss 时，需要检查维度一致，避免广播。
def l1_loss(input, target, size_average=None, reduce=None, reduction='mean'):
# type: (Tensor, Tensor, Optional[bool], Optional[bool], str) -> Tensor
r"""l1_loss(input, target, size_average=None, reduce=None, reduction='mean') -> Tensor
Function that takes the mean element-wise absolute value difference.
See :class:`~torch.nn.L1Loss` for details.
"""
if not torch.jit.is_scripting():
tens_ops = (input, target)
if any([type(t) is not Tensor for t in tens_ops]) and has_torch_function(tens_ops):
return handle_torch_function(
l1_loss, tens_ops, input, target, size_average=size_average, reduce=reduce,
reduction=reduction)
if not (target.size() == input.size()):
warnings.warn("Using a target size ({}) that is different to the input size ({}). "
"This will likely lead to incorrect results due to broadcasting. "
"Please ensure they have the same size.".format(target.size(), input.size()),
stacklevel=2)
if size_average is not None or reduce is not None:
reduction = _Reduction.legacy_get_string(size_average, reduce)
if target.requires_grad:
ret = torch.abs(input - target)
if reduction != 'none':
ret = torch.mean(ret) if reduction == 'mean' else torch.sum(ret)
else:
expanded_input, expanded_target = torch.broadcast_tensors(input, target)
ret = torch._C._nn.l1_loss(expanded_input, expanded_target, _Reduction.get_enum(reduction))
return ret