Open
Description
The forward transfer metric in the metrics_summary.csv is the same across different methods.
Run with Offline-ER and Joint updater on Mnist.
Expected behavior
I would not expect this to be (strictly) same for different methods.
Example
Joint training
| Task ID | Average Accuracy | Forgetting | Forward Transfer | Backward Transfer |
|---|---|---|---|---|
| 1 | 0.999054 | 0.000000 | 0.000000 | 0.000000 |
| 2 | 0.968869 | 0.010875 | -0.206170 | -0.010875 |
| 3 | 0.913238 | 0.043618 | -0.103085 | -0.043618 |
| 4 | 0.853757 | 0.092997 | -0.190409 | -0.092997 |
| 5 | 0.646799 | 0.273982 | -0.142806 | -0.273982 |
Offline-ER
| Task ID | Average Accuracy | Forgetting | Forward Transfer | Backward Transfer |
|---|---|---|---|---|
| 1 | 0.998582 | 0.000000 | 0.000000 | 0.000000 |
| 2 | 0.977127 | 0.006619 | -0.206170 | -0.006619 |
| 3 | 0.956068 | 0.016186 | -0.103085 | -0.016186 |
| 4 | 0.948742 | 0.013383 | -0.190409 | -0.013383 |
| 5 | 0.932628 | 0.013746 | -0.142806 | -0.013746 |