Closed
Description
Context
In torch.compile, the kernel exploration and generation happens in the first iteration. Therefore, when you compile a model, the first iteration takes more time than the rest of the training.
In compile_model function we should improve the error message to read something like: "Compiling model layers with torch.compile. Expect a relatively slower first step".
There may be some other mentions of this sprinkled throughout the codebase. These should be cleaned up so that the only reference happens in compile_model
Acceptance Criteria
Update or add a unit test that checks for this output in the logs.