Skip to content

Questions about the loss function #22

Open
Open
Questions about the loss function#22

Description

@handsomelys
handsomelys
opened on Mar 14, 2023

I'm interested in how to implement InfoNCE in your code using torch.nn.CrossEntropyLoss() function, but I don't seem to have found good learning material, can you explain to me why InfoNCE can be implemented in this way in the provided code?

In addition, I would like to ask, how should this part of the code be understood?

# head + relation -> tail
loss = self. Criterion(logits, labels)
# tail -> head + relation
loss += self. Criterion(logits[:, :batch_size].t(), labels)

How to understand "tail -> head + relation"？
I would appreciate your help! Looking forward to your reply！

Activity

  1. intfloat commented on Mar 14, 2023

    @intfloat
    intfloat
    on Mar 14, 2023
    Owner

    You can refer to the answer here: #10 (comment)

    The InfoNCE loss is basically a cross entropy loss but the labels are not pre-defined like in text classification.

  2. handsomelys commented on Mar 14, 2023

    @handsomelys
    handsomelys
    on Mar 14, 2023
    Author

    thanks for your reply!

  3. cyjie429 commented on Mar 18, 2023

    @cyjie429
    cyjie429
    on Mar 18, 2023

    You can refer to the answer here: #10 (comment)

    The InfoNCE loss is basically a cross entropy loss but the labels are not pre-defined like in text classification.

    您好，请问是否可以仅使用head + relation -> tail？

  4. intfloat commented on Mar 18, 2023

    @intfloat
    intfloat
    on Mar 18, 2023
    Owner

    当然可以，把第二行loss += ...注释掉就行，但效果会下降一点

  5. cyjie429 commented on Mar 23, 2023

    @cyjie429
    cyjie429
    on Mar 23, 2023

    非常感谢

  6. cyjie429 commented on Mar 23, 2023

    @cyjie429
    cyjie429
    on Mar 23, 2023

    当然可以，把第二行loss += ...注释掉就行，但效果会下降一点

    请问我在训练过程中将第二行 Loss+=...注释之后，在测试过程中也需要注释backward_metrics = .....吗

  7. cyjie429 commented on Mar 23, 2023

    @cyjie429
    cyjie429
    on Mar 23, 2023

    当然可以，把第二行loss += ...注释掉就行，但效果会下降一点

    因为我每次看最终结果 好像forward metrics的效果会更好

  8. intfloat commented on Mar 24, 2023

    @intfloat
    intfloat
    on Mar 24, 2023
    Owner

    当然可以，把第二行loss += ...注释掉就行，但效果会下降一点

    请问我在训练过程中将第二行 Loss+=...注释之后，在测试过程中也需要注释backward_metrics = .....吗

    不需要，forward metrics 对应 给定头实体和关系，预测尾实体，backward metrics 对应 给定尾实体和关系，预测头实体。对大多数数据集来说，第一种预测任务更简单，所以metrics会好一些。

  9. cyjie429 commented on Mar 24, 2023

    @cyjie429
    cyjie429
    on Mar 24, 2023

    当然可以，把第二行loss += ...注释掉就行，但效果会下降一点

    请问我在训练过程中将第二行 Loss+=...注释之后，在测试过程中也需要注释backward_metrics = .....吗

    不需要，forward metrics 对应 给定头实体和关系，预测尾实体，backward metrics 对应 给定尾实体和关系，预测头实体。对大多数数据集来说，第一种预测任务更简单，所以metrics会好一些。

    懂了 谢谢

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions