top of page

Evaluating Performance:

Insights from Predictive Models

This project evaluated the performance of various advanced neural network architectures—LSTM, improved LSTM, CNN-LSTM hybrid with attention, and GRU models—to predict Nvidia’s stock prices based on news sentiment and historical stock data. The models were assessed using Mean Squared Error (MSE) and Mean Absolute Error (MAE), which measure the accuracy of predictions. A lower MSE indicates less deviation between actual and predicted values, while a lower MAE reflects the average absolute difference between them. The results reveal distinct strengths and limitations for each model, emphasizing the importance of model selection, feature engineering, and optimization.

1.LSTM:

  • Test Loss (MSE): 73.7891

  • Test Mean Absolute Error (MAE): 7.8635

The standard LSTM model provided a baseline for time-series predictions. While effective in capturing temporal patterns, its high MSE and MAE indicate that the model struggled with noise and complex relationships in the data. The lack of feature optimization and regularization in this version limited its accuracy.

​​​​

Screenshot 2024-12-08 at 5.36.13 PM.png
Screenshot 2024-12-08 at 5.38.40 PM.png

2.Improved LSTM Model:

  • Test Loss (MSE): 0.00047

  • Test Mean Absolute Error (MAE): 0.0162

The improved LSTM model significantly reduced errors by incorporating optimized hyperparameters, additional features like moving averages and lagged sentiment, and regularization techniques such as dropout layers. This version demonstrated how enhancements in architecture and data preparation can lead to substantial improvements in performance.

​​​​

3.CNN-LSTM Hybrid with Attention:

  • Test Loss (MSE): 0.00029

  • Test Mean Absolute Error (MAE): 0.0127

The CNN-LSTM hybrid model, with its attention mechanism, combined the strengths of convolutional layers for short-term pattern detection and LSTMs for long-term temporal relationships. The attention mechanism allowed the model to focus on the most relevant timesteps, further improving accuracy. Its performance highlights the value of hybrid approaches in financial time-series prediction.

​​​​

Screenshot 2024-12-08 at 5.41.39 PM.png
Screenshot 2024-12-08 at 5.49.08 PM.png

4.GRU Model:

  • Test Loss (MSE): 0.00022

  • Test Mean Absolute Error (MAE): 0.0098

The GRU model emerged as the most efficient and accurate architecture. By reducing complexity compared to LSTMs, GRUs achieved faster training and lower computational costs while maintaining excellent performance. This model delivered the lowest MSE and MAE, indicating its suitability for datasets with limited size and significant temporal dependencies.

​​​​

Visualizations (Predicted vs Actual

Stock Prices)

1.LSTM:

image.png

3.CNN-LSTM Hybrid with Attention:

image.png

2.Improved LSTM Model:

image.png

4.GRU Model:

image.png

Model Comparison Based on MSE

image.png

The results underscore the importance of iterative improvements in model architecture and feature engineering for financial time-series prediction. The LSTM model provided a strong foundation, but its performance was significantly enhanced through optimization in the improved LSTM version. The CNN-LSTM hybrid model showcased the potential of combining local feature extraction and sequential modeling, while the GRU model delivered superior accuracy with reduced computational overhead. The GRU model’s success suggests that its simplified architecture is well-suited for datasets with limited size and noisy features. Its ability to generalize and predict with minimal error makes it the preferred choice for this project. The results demonstrate that integrating news sentiment and stock data with advanced neural networks can yield powerful insights into market trends, enabling more informed decision-making for investors and analysts.

bottom of page