Skip to content

Add CrossEntropyLoss#154

Merged
David-Kreplin merged 37 commits intosQUlearn:developfrom
MoritzWillmann:add_log_loss
Aug 21, 2025
Merged

Add CrossEntropyLoss#154
David-Kreplin merged 37 commits intosQUlearn:developfrom
MoritzWillmann:add_log_loss

Conversation

@MoritzWillmann
Copy link
Collaborator

@MoritzWillmann MoritzWillmann commented Sep 27, 2023

Current State

This PR adds a new loss class CrossEntropyLoss implementing the multi-class log loss (aka. cross entropy loss)

The value is calculated as

$$L = -\frac{1}{N} \sum_{i=1}^N \sum_{j=1}^K y_{ij} \cdot \log\left(f(x_i)_j\right)$$

If the task at hand is binary, a "class" is added with prediction $1-f(x)$ and label $1-y$.

The gradient is thus calculated as

$$\frac{\partial}{\partial \theta} L = -\frac{1}{N}\sum_{i=1}^N \sum_{j=1}^K \frac{y_{ij}}{f(x_i)_j} \frac{\partial f}{\partial \theta}$$

If the task at hand is binary, a "class" is added with prediction $f(x)-1$ and label $1-y$.

I test this functionality in tests/qnn/test_loss.py.

This PR solves #112

@MoritzWillmann MoritzWillmann added the help wanted Extra attention is needed label Sep 27, 2023
@MoritzWillmann MoritzWillmann linked an issue Oct 4, 2023 that may be closed by this pull request
@MoritzWillmann
Copy link
Collaborator Author

Update

I figured out that if if eps is sufficiently large, training works. The idea of eps being dependent on the learning task and model at hand, i.e. making it somewhat of a hyperparameter, seems very strange to me. Maybe we can gain some more insights on this.

@MoritzWillmann MoritzWillmann deleted the add_log_loss branch November 13, 2024 09:55
@MoritzWillmann MoritzWillmann restored the add_log_loss branch December 13, 2024 09:36
@MoritzWillmann
Copy link
Collaborator Author

Not sure, why this was closed

@rothma
Copy link
Collaborator

rothma commented Mar 28, 2025

@MoritzWillmann in my tests it works now (at least for binary)

@MoritzWillmann MoritzWillmann changed the title WIP: Add LogLoss WIP: Add CrossEntropyLoss Aug 19, 2025
@MoritzWillmann MoritzWillmann marked this pull request as ready for review August 20, 2025 10:47
@MoritzWillmann MoritzWillmann removed the help wanted Extra attention is needed label Aug 20, 2025
@MoritzWillmann
Copy link
Collaborator Author

I finally was able to fix it. LGTM 🎉

@MoritzWillmann MoritzWillmann changed the title WIP: Add CrossEntropyLoss Add CrossEntropyLoss Aug 21, 2025
@David-Kreplin David-Kreplin merged commit 11a90f2 into sQUlearn:develop Aug 21, 2025
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

QNN Class has no classification loss

3 participants