Skip to content

Integrate DTW calculation in training loop #13

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

Integrate DTW calculation in training loop #13

wants to merge 7 commits into from

Conversation

RongLirr
Copy link
Collaborator

No description provided.

AmitMY
AmitMY requested changes May 24, 2025
View reviewed changes
fluent_pose_synthesis/core/training.py Outdated
try:
scores = [
self.validation_metric_calculator._pose_score(pred, ref)
for pred, ref in zip(all_predictions_poses, all_references_poses)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

self.validation_metric_calculator.corpus_score

fluent_pose_synthesis/core/training.py Outdated
use_fast=True,
default_distance=default_dtw_dist_val,
),
pose_preprocessors=[],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is a processor called osmething like - "normalizeshoulders"

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used NormalizePosesProcessor(PoseProcessor) https://github.com/sign-language-processing/pose-evaluation/blob/e4704ba6bddd28699fd7bdf8bac350838f161b94/pose_evaluation/metrics/pose_processors.py#L54

@RongLirr
Copy link
Collaborator Author

  1. Added AMP support as a configurable option.
  2. Saving a fixed set of validation predictions each epoch for consistent visualization.
  3. Computing DTW score and validation loss at every epoch.

@RongLirr
Copy link
Collaborator Author

RongLirr commented Jun 9, 2025

Main changes:

  1. In addition to the original MSE loss, velocity loss (and optionally, acceleration loss, but now it's turned off) has been incorporated for more smooth motion.

  2. During training, setting cond_mask_prob > 0. During inference, CFG sampling logic is added to the validation pipeline, using a guidance_scale parameter (set to 2 now) to amplify the influence of the conditional input.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@AmitMY AmitMY AmitMY requested changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@RongLirr @AmitMY