Pytorch从基础数据中学习时出现巨大损失

标签：python pytorch

我正在尝试制作一个非常基本的机器学习应用程序，基本上我希望模型能够根据我提供的虚拟数据尝试预测明年的每个月，但我遭受了巨大的损失 Loss: 5206342.5000 并且预测值与我的预期相差如此之远 Predicted values for each month of next year: [-0.043424129486083984, 0.041442010551691055, -0.16847632825374603, -0.18227830529212952, 0.4619046151638031, 1.128817081451416, 0.05658513307571411, 0.1113058477640152, 0.2900705337524414, 0.21394489705562592, 0.2618725001811981, 0.3829265236854553] 我对机器学习非常陌生，为什么我会看到如此巨大的损失和如此错误的预测值？

import torch
import torch.nn as nn
import torch.optim as optim
import pandas as pd
import numpy as np

# Example historical data (replace with your actual data loading)
data = {
    'ProjectId': ['666d60f2604698f1383d94e4', '666d60e2604698f1383d9495', '666d5fb1604698f1383d8e8f'],
    'Year': ['2023/2024', '2023/2024', '2023/2024'],
    'July': [0, 0, 0],
    'August': [0, 0, 0],
    'September': [0, 0, 0],
    'October': [0, 0, 0],
    'November': [0, 0, 0],
    'December': [10000, 5000, 100000],
    'January': [0, 0, 120000],
    'February': [0, 0, 0],
    'March': [0, 0, 0],
    'April': [0, 0, 0],
    'May': [0, 0, 0],
    'June': [0, 0, 0]
}

df = pd.DataFrame(data)

# Example: Extract X (input features) and y (target values)
X = df.drop(['ProjectId', 'Year'], axis=1).values.astype(np.float32)  # Convert to float32
y = df.drop(['ProjectId', 'Year'], axis=1).values.astype(np.float32)  # Assuming y includes next year's forecast

# Example: Splitting into training and validation sets
# Assume last row for validation, rest for training
X_train = torch.tensor(X[:-1], dtype=torch.float32).unsqueeze(0)  # Add batch dimension (1 sample)
y_train = torch.tensor(y[:-1], dtype=torch.float32).unsqueeze(0)  # Add batch dimension (1 sample)
X_val = torch.tensor(X[-1:], dtype=torch.float32).unsqueeze(0)  # Example single project for validation

# Define LSTM model for forecasting
class LSTMForecast(nn.Module):
    def __init__(self, input_size, hidden_size, output_size):
        super(LSTMForecast, self).__init__()
        self.hidden_size = hidden_size
        self.lstm = nn.LSTM(input_size, hidden_size, batch_first=True)
        self.fc = nn.Linear(hidden_size, output_size)

    def forward(self, x):
        lstm_out, _ = self.lstm(x)
        out = self.fc(lstm_out[:, -1, :])  # Predict based on the last LSTM output
        return out

# Initialize the model, loss function, and optimizer
input_size = X_train.shape[-1]
hidden_size = 64
output_size = y_train.shape[-1]

model = LSTMForecast(input_size, hidden_size, output_size)
criterion = nn.MSELoss()
optimizer = optim.Adam(model.parameters(), lr=0.001)

# Training loop
num_epochs = 100
for epoch in range(num_epochs):
    optimizer.zero_grad()
    outputs = model(X_train)
    loss = criterion(outputs, y_train)
    loss.backward()
    optimizer.step()

    if (epoch+1) % 10 == 0:
        print(f'Epoch [{epoch+1}/{num_epochs}], Loss: {loss.item():.4f}')

# Example: Making predictions for next year's forecast
model.eval()
with torch.no_grad():
    forecast = model(X_val)
    forecast_values = forecast.numpy().tolist()[0]  # Convert to numpy and list

print(f'Predicted values for each month of next year: {forecast_values}')

你代码中有一些问题导致损失很大，预测不准确：

输入和目标数据相同： 你正在使用 X 作为输入特征， y 作为目标值，但两者都是从 df.drop(['ProjectId', 'Year'], axis=1) 中获取的。这意味着你的模型试图根据自身来预测月份值，这没有意义，会导致模型过度拟合训练数据，并且不能很好地推广到未见过的数据。
训练数据不足： 你只有 3 个数据点用于训练，这对于 LSTM 来说太少了，不足以学习数据中的任何有意义的模式。
数据缩放： 你的数据范围很大（从 0 到 120000）。在将数据输入 LSTM 之前，对数据进行缩放（例如，使用 MinMaxScaler 或 StandardScaler）可能会有所帮助，以提高训练稳定性。
缺少验证： 你没有使用验证集来跟踪模型在训练过程中的性能。验证集有助于检测过度拟合，并可以提供对模型在未见过的数据上的性能的更好估计。

以下是如何解决这些问题以及如何改进代码的方法：

import torch
import torch.nn as nn
import torch.optim as optim
import pandas as pd
import numpy as np
from sklearn.preprocessing import MinMaxScaler
from sklearn.model_selection import train_test_split

# Example historical data (replace with your actual data loading)
data = {
    'ProjectId': ['666d60f2604698f1383d94e4', '666d60e2604698f1383d9495', '666d5fb1604698f1383d8e8f'],
    'Year': ['2023/2024', '2023/2024', '2023/2024'],
    'July': [0, 0, 0],
    'August': [0, 0, 0],
    'September': [0, 0, 0],
    'October': [0, 0, 500],
    'November': [1000, 2000, 1500],
    'December': [10000, 5000, 100000],
    'January': [12000, 6000, 120000],
    'February': [0, 0, 0],
    'March': [0, 0, 0],
    'April': [0, 0, 0],
    'May': [0, 0, 0],
    'June': [0, 0, 0]
}

df = pd.DataFrame(data)

# Data Preprocessing
# Assuming 'July' to 'June' are current year data, 
# and we want to predict 'July' to 'June' of next year
data = df.drop(['ProjectId', 'Year'], axis=1).values.astype(np.float32)

# Scale the data
scaler = MinMaxScaler()
data = scaler.fit_transform(data)

# Create input-output pairs with a sliding window
sequence_length = 6  # Use past 6 months to predict next 6 months
X, y = [], []
for i in range(len(data) - sequence_length):
    X.append(data[i:i+sequence_length])
    y.append(data[i+sequence_length:i+2*sequence_length])
X = np.array(X)
y = np.array(y)

# Split data into training and validation sets
X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2, random_state=42)

# Convert to PyTorch tensors
X_train = torch.tensor(X_train, dtype=torch.float32)
y_train = torch.tensor(y_train, dtype=torch.float32)
X_val = torch.tensor(X_val, dtype=torch.float32)
y_val = torch.tensor(y_val, dtype=torch.float32)

# Define LSTM model for forecasting
class LSTMForecast(nn.Module):
    def __init__(self, input_size, hidden_size, output_size):
        super(LSTMForecast, self).__init__()
        self.hidden_size = hidden_size
        self.lstm = nn.LSTM(input_size, hidden_size, batch_first=True)
        self.fc = nn.Linear(hidden_size, output_size)

    def forward(self, x):
        lstm_out, _ = self.lstm(x)
        out = self.fc(lstm_out[:, -1, :])  # Predict based on the last LSTM output
        return out

# Initialize the model, loss function, and optimizer
input_size = X_train.shape[-1]
hidden_size = 64
output_size = y_train.shape[-1]

model = LSTMForecast(input_size, hidden_size, output_size)
criterion = nn.MSELoss()
optimizer = optim.Adam(model.parameters(), lr=0.001)

# Training loop
num_epochs = 100
for epoch in range(num_epochs):
    # Training
    model.train()
    outputs = model(X_train)
    loss = criterion(outputs, y_train)
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

    # Validation
    model.eval()
    with torch.no_grad():
        val_outputs = model(X_val)
        val_loss = criterion(val_outputs, y_val)

    if (epoch+1) % 10 == 0:
        print(f'Epoch [{epoch+1}/{num_epochs}], Train Loss: {loss.item():.4f}, Val Loss: {val_loss.item():.4f}')

# Example: Making predictions
model.eval()
with torch.no_grad():
    forecast = model(X_val)
    forecast = scaler.inverse_transform(forecast.numpy())  # Inverse transform to get original scale
    print(f'Predicted values: {forecast}')

改进之处：

数据预处理：
- 使用滑动窗口创建输入-输出对，其中过去 6 个月的值用于预测未来 6 个月的值。
- 使用 MinMaxScaler 对数据进行缩放，使其范围在 0 到 1 之间，从而提高训练稳定性。
训练数据： 通过使用滑动窗口，我们从现有数据中创建了更多数据点，从而为 LSTM 提供了更多学习内容。
验证集： 我们将数据分成训练集和验证集，以跟踪训练过程中的性能，并更好地估计模型在未见过的数据上的性能。
逆缩放： 在进行预测后，我们使用 inverse_transform 方法将预测值缩放回其原始比例。

这些更改应该会带来更低的损失和更准确的预测。但是，请记住，LSTM 在时间序列预测方面非常强大，但它们需要大量数据才能有效。考虑收集更多数据以进一步改进你的模型。

标签：python,pytorch
From： 78778376

Pytorch从基础数据中学习时出现巨大损失

相关文章

赞助商

阅读排行