Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes + More Metrics #1

Open
wants to merge 1 commit into
base: more-metrics
Choose a base branch
from

Conversation

robertgshaw2-neuralmagic
Copy link
Collaborator

  • Fixed the incorrect logic for chunked prefill
  • Added more metrics
  • Updated naming schema of the stats to have the _suffix pattern

NOTE: this code does not run. I have not finished debugging it.

Could you pick it up from here @ronensc ?

…een hacking in my branch. Note: this is not complete. I have not run the code yet, I am sure there are bugs that need to be fixed.
@HarryWu99
Copy link

@robertgshaw2-neuralmagic can I pick it up from here?

@robertgshaw2-neuralmagic
Copy link
Collaborator Author

robertgshaw2-neuralmagic commented Apr 29, 2024

The current state is that ~half of this PR is merged into upstream main

so big next steps are rebasing this branch onto current upstream main and making sure all the metrics are being computed correctly

@HarryWu99
Copy link

HarryWu99 commented Apr 29, 2024

@robertgshaw2-neuralmagic when I rebase my branch to vllm/main branch, there seem to be infinite conflicts!

what should I do then? I think I should create a new branch from vllm/main and apply the changes to that branch. Is it ok?

HarryWu99#1

@robertgshaw2-neuralmagic
Copy link
Collaborator Author

Yeah I would make a new branch, just replicating the new metrics that were added here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants