fix: do not print perf stat when NaN

[leseb]

b415326 fix: do not print perf stat when NaN

commit b415326
Author: Sébastien Han seb@redhat.com
Date: Thu Nov 14 11:04:47 2024 +0100

fix: do not print perf stat when NaN

If the chat is exited or interrupted it will still print the stats with
NaN values which is unnecessary.

Signed-off-by: Sébastien Han <seb@redhat.com>

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchchat/1375

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• GLIBC not found in Nova workflows

✅ No Failures

As of commit b415326 with merge base 4697764 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[Jack-Khuu]

PR looks good, but can you share an example of when this would get triggered (i.e. when are we seeing NaN via manually kill)?

[leseb]

PR looks good, but can you share an example of when this would get triggered (i.e. when are we seeing NaN via manually kill)?

$ python3.10 torchchat.py chat llama3.1 
NumExpr defaulting to 12 threads.
PyTorch version 2.6.0.dev20241002 available.
lm_eval is not installed, GPTQ may not be usable
Using device=mps 
Loading model...
Time to load model: 15.06 seconds
-----------------------------------------------------------
Starting Interactive Chat
Entering Chat Mode. Will continue chatting back and forth with the language model until the models max context length of 8192 tokens is hit or until the user says /bye
Do you want to enter a system prompt? Enter y for yes and anything else for no. 

User: /bye
Exiting Chat.


      Average tokens/sec (total): nan                 
Average tokens/sec (first token): nan                 
Average tokens/sec (next tokens): nan