fix(xjy): adding the messenger environment #405
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
puyuan1996
added
research
puyuan1996
reviewed
Aug 24, 2025
|
|
||
| if use_manual: | ||
| self.feature_merge_linear = nn.Linear(self.embedding_dim + manual_dim, self.embedding_dim) | ||
|
|
There was a problem hiding this comment.
self.feature_merge_linearh后面应该和原来的obs_embeddings执行相同的norm?
| [maunal_sentence], truncation=True, padding="max_length", max_length=self.max_seq_len, return_tensors='pt') | ||
| # ts = {k: v.to(self.device) for k, v in ts.items()} | ||
| with torch.no_grad(): | ||
| self.manual_embeds = self.manual_encoder(**tokenized_output).last_hidden_state[:,0,:].squeeze() |
There was a problem hiding this comment.
对于messenger，self.manual_embeds目前是使用的固定的手册文段通过bge编码得到的吗
There was a problem hiding this comment.
我目前发现了unizero在kv_cache中的一个bug, 可能导致时间unizero没有正确利用到kvcace，这可能是目前加了manual_embeddings性能没有明显提升的主要原因哈，这个bug我在mspacman上验证修复后你再merge然后重跑一下实验哈。不过就算是单步的，加了manual_embeddings应该相比之前性能会有提升一点才对，可能还有其他问题。
There was a problem hiding this comment.
你可以先实现探索相关的方法哈，等我把unizero的kv_cache bug修复再跑这边的实验
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
目前的问题是在Messenger_env环境交互的时候每一步得到了两个比较重要的信息一个是grid_image, 一个是manual_embeds，但是需要注意的是每次step()后grid_image会变化，但是manual_embeds每一个envs下都是保持不变的。 我期望的是在对grid_image通过编码器进行编码成obs latent的时候能够将manual_embeds进行拼接，但是这部分代码不知道以哪种方式实现比较好