Description
This experiment aims to compare the performance of the base CryptoBERT model with a fine-tuned version of the same. The fine-tuned model is specifically adapted to work with the Impact Score of the next day as a regression task, rather than the original classification task.
Change the Output Layer: The final layer of the CryptoBERT model, currently a softmax layer for classification, needs to be changed to a single node with a linear activation function. This change is necessary because regression tasks predict a continuous output, unlike classification tasks.
Modify the Loss Function: The loss function should be changed from a classification loss function, such as Cross-Entropy Loss, to a regression loss function, such as Mean Squared Error (MSE) or Mean Absolute Error (MAE). This change is crucial as the loss function guides the optimization of the model weights.
Adjust the Labels: The labels should be continuous values suitable for regression, rather than discrete class labels. The Impact Score labeler provided in the code can be used as an example.
Data Normalization: The Impact Score labels should be normalized before using them. They should be normalized to a number between 0 and 1, where 0 represents the least impact and 1 represents the most.
Training and Evaluation Metrics: The compute metrics method should include the Mean Absolute Error. An attribute should be added to the model that sets the type of task, which could be either regression or classification. The Mean Absolute Error should only be calculated when the self.task attribute in the model is set to regression.
Dataset:
The dataset used for this experiment includes daily tweets from 2020, introduced by Zou, containing a total of 64,000 combined tweets. It also includes 2020 daily candles.
Model:
The model used for this experiment is the Hugging Face CryptoBERT model.
Metrics:
The metric used to evaluate the performance of the model is the Mean Absolute Error. This metric provides a measure of how accurately the model can predict the Impact Score of the next day.